Forum Discussion
Advocate IAzure Synapse Link for Dataverse in a Fabric architecture
Hi Luigia-Costabil ,
Thanks for reaching out to Microsoft Fabric Community and Thanks to Vinodh247 for providing meaningful insights.Converting the Append Only CSV exports into Delta tables using Fabric notebooks does not inherently reduce the benefits of Synapse Link change tracking. The main consideration is data freshness, as the Delta Bronze layer will only reflect changes after the Fabric notebook or pipeline has processed the latest exported files.
Regarding Append Only mode, Microsoft documentation states that create, update, and delete changes are written as additional records in the destination files, and delete operations can be identified through the IsDelete column. This supports incremental processing scenarios.
However, the documentation does not explicitly state that the exported files should be treated as a complete audit trail containing every intermediate change event. Therefore, while Append Only preserves the changes exported by Synapse Link, any requirements for full historical auditing should be validated against your specific business needs.
A common Fabric approach is to retain the exported CSV files as the raw layer and load them into Delta Bronze tables using Fabric Spark notebooks or pipelines, allowing downstream workloads to benefit from Delta Lake capabilities while preserving the original exports.
For Reference:
Advanced configuration options in Azure Synapse Link Link your Dataverse environment to Microsoft Fabric and unlock deep insights
Export Dataverse data in Delta Lake formatBest Regards,
Abdul Rafi
Super Useryes, teams have used azure synapse link exporting CSV only as a Bronze layer, but in practice it is usually a temp or transitional setup rather than a long-term design.
Keeping CSV as Bronze works, but it comes with clear trade-offs:
-
Performance: CSV is row-based, no indexing, no metadata pruning. Queries are slow and expensive at scale compared to Delta (no predicate pushdown, no file skipping).
-
Schema evolution: Weak. Any column change requires manual handling. No built-in versioning like Delta.
-
CDC / SCD2: You must implement everything yourself using
IsDelete, timestamps, and joins. This becomes complex and error-prone quickly. -
Query complexity: High. You need to repeatedly define schema, handle nulls, types, and deduplication logic in every downstream process.
In short: CSV is fine for raw landing, but not for anything beyond ingestion.
On cost:
-
ADLS Gen2: Cheap. You pay for storage (roughly $0.018 – $0.03 per GB/month depending on tier) plus transaction costs. CSV vs Delta does not change storage much, but Delta reduces compute cost later.
-
Spark (Synapse or Fabric): This is where cost matters. Synapse Spark pools are billed per vCore/hour even when idle (unless paused). Fabric Spark is capacity-based (FSKU), typically more cost-efficient if you are already invested in fabric.
Using Fabric Spark instead of Synapse Spark to convert CSV -> Delta is increasingly the preferred approach:
-
It removes the need for synapse workspace and dedicated Spark pools
-
Keeps everything inside Fabric(Lakehouse, OneLake, governance)
-
Lets you directly build bronze delta tables aligned with Medallion architecture
Real-world pattern that works well:
Dataverse -> Synapse Link (CSV in ADLS) -> Fabric Notebook/Dataflow Gen2 -> Bronze Delta in Lakehouse -> Silver/Gold
Advocate IThank you! I have a clarification to ask.
Suppose I have the following setup:
- Dataverse
- Azure Synapse Link (Append Only)
- ADLS Gen2 (CSV)
- Fabric
My question is about the Bronze layer.
If I create Fabric notebooks that periodically read the exported CSV files and materialize them into Delta Lake Bronze tables, do I lose part of the benefit of Synapse Link's near real-time change tracking?
When using Azure Synapse Link in Append Only mode, are the exported CSV snapshots guaranteed to preserve all historical change events (including intermediate updates and IsDelete records), making them suitable as a Bronze layer without materializing Delta tables?
v-moharafi-msftCommunity Support
Hi Luigia-Costabil ,
Thanks for reaching out to Microsoft Fabric Community and Thanks to Vinodh247 for providing meaningful insights.Converting the Append Only CSV exports into Delta tables using Fabric notebooks does not inherently reduce the benefits of Synapse Link change tracking. The main consideration is data freshness, as the Delta Bronze layer will only reflect changes after the Fabric notebook or pipeline has processed the latest exported files.
Regarding Append Only mode, Microsoft documentation states that create, update, and delete changes are written as additional records in the destination files, and delete operations can be identified through the IsDelete column. This supports incremental processing scenarios.
However, the documentation does not explicitly state that the exported files should be treated as a complete audit trail containing every intermediate change event. Therefore, while Append Only preserves the changes exported by Synapse Link, any requirements for full historical auditing should be validated against your specific business needs.
A common Fabric approach is to retain the exported CSV files as the raw layer and load them into Delta Bronze tables using Fabric Spark notebooks or pipelines, allowing downstream workloads to benefit from Delta Lake capabilities while preserving the original exports.
For Reference:
Advanced configuration options in Azure Synapse Link Link your Dataverse environment to Microsoft Fabric and unlock deep insights
Export Dataverse data in Delta Lake formatBest Regards,
Abdul Rafi