Forum Discussion
Helper IAre ACID properties supported in Lakehouse
Hi All,
I am transferring some data from Azure SQL DB to Fabric Lakehouse table via Fabric data pipeline.
I wanted to confirm if data pipeline fails for any reason in the middle of copy activity, would the data rollback take place? Or lakehouse table would contain half data due to pipeline failure?
How to ensure data sanity and quality in the above mentioned scenario?
Hello PriyaJha,
As you can see here : https://learn.microsoft.com/en-us/fabric/data-science/tutorial-data-science-ingest-data
"Delta Lake - Delta Lake is an open-source storage layer that brings ACID transactions, scalable metadata management, and batch and streaming data processing to Apache Spark. A Delta Lake table is a data table format that extends Parquet data files with a file-based transaction log for ACID transactions and scalable metadata management."
Delta Lake will provide some level of ACID support especially on your scenario, If you are looking for multi table acidity on a single transaction you will need to look at Data Warehouse insteak of Lakehouse.
But for your use case it's fully supported !
1 Reply
- charlyS
Most Valuable Professional
Hello PriyaJha,
As you can see here : https://learn.microsoft.com/en-us/fabric/data-science/tutorial-data-science-ingest-data
"Delta Lake - Delta Lake is an open-source storage layer that brings ACID transactions, scalable metadata management, and batch and streaming data processing to Apache Spark. A Delta Lake table is a data table format that extends Parquet data files with a file-based transaction log for ACID transactions and scalable metadata management."
Delta Lake will provide some level of ACID support especially on your scenario, If you are looking for multi table acidity on a single transaction you will need to look at Data Warehouse insteak of Lakehouse.
But for your use case it's fully supported !