We have a data source has regular data purge activities which will result data records be deleted. But I would still want to keep it downstream. Can there be an option for the individual databadse table mirror to have somthing like "Ignore delete" to not process delete for that table for the mirror operation?
1 Comment
I will expand on this ask.
Problem
Today, mirroring faithfully propagates source-system changes, including deletes, to the mirrored destination. While this is the correct behavior for most transactional workloads, it creates unnecessary overhead for a common class of databases that are effectively append-only.
Many operational systems continuously insert new records but periodically execute bulk purge jobs that delete aged-out data (for example, records older than 1, 3, or 7 years). In these scenarios, deletes do not represent business events that consumers need reflected in Fabric. They are simply a retention-management mechanism in the source system.
As a result, the mirroring engine must process potentially millions of delete operations that provide little or no analytical value.
Proposed Enhancement
Add an "Ignore Deletes" toggle to the mirroring configuration.
When enabled:
- Insert operations are mirrored normally.
- Update operations are mirrored normally.
- Delete operations detected in the source transaction log are ignored.
- The corresponding rows remain in the mirrored destination even after being removed from the source.
Example
Source ------- ID Value 1 A 2 B 3 C Nightly purge deletes ID=1 Source after purge ------------------ 2 B 3 C Mirrored destination (Ignore Deletes = ON) ------------------------------------------- 1 A 2 B 3 C
