Forum Discussion
Lakehouse supports ACID compliant
Hello Everyone,
As per this article (What is data warehousing in Microsoft Fabric? - Microsoft Fabric | Microsoft Learn)
- Data stored in Delta-parquet format enables ACID transactions and interoperability with other Fabric workloads means you don't need multiple copies of data.
which clearly says Warehouse is ACID compliant. As Lakehouse table also stores data in delta-parquet format only wont it be ACID compliant?
Can someone clarify please?
Anonymous
Regards,
Srisakthi
Hi Srisakthi ,
Yes, Lakehouses support ACID transactions. The Delta Table format stores previous and current versions of data and seamlessly transfers between them. You can also use Time Travel to restore or view previous versions of the data (until VACUUM/OPTIMIZE are ran, anyway)
From here; https://learn.microsoft.com/en-us/fabric/data-science/tutorial-data-science-ingest-dataLakehouse - A lakehouse is a collection of files/folders/tables that represent a database over a data lake used by the Spark engine and SQL engine for big data processing and that includes enhanced capabilities for ACID transactions when using the open-source Delta formatted tables.
Delta Lake - Delta Lake is an open-source storage layer that brings ACID transactions, scalable metadata management, and batch and streaming data processing to Apache Spark. A Delta Lake table is a data table format that extends Parquet data files with a file-based transaction log for ACID transactions and scalable metadata management.
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
4 Replies
Hi Srisakthi ,
Yes, Lakehouses support ACID transactions. The Delta Table format stores previous and current versions of data and seamlessly transfers between them. You can also use Time Travel to restore or view previous versions of the data (until VACUUM/OPTIMIZE are ran, anyway)
From here; https://learn.microsoft.com/en-us/fabric/data-science/tutorial-data-science-ingest-dataLakehouse - A lakehouse is a collection of files/folders/tables that represent a database over a data lake used by the Spark engine and SQL engine for big data processing and that includes enhanced capabilities for ACID transactions when using the open-source Delta formatted tables.
Delta Lake - Delta Lake is an open-source storage layer that brings ACID transactions, scalable metadata management, and batch and streaming data processing to Apache Spark. A Delta Lake table is a data table format that extends Parquet data files with a file-based transaction log for ACID transactions and scalable metadata management.
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.Hi spencer_sa ,
Thank you for your response.
But still multi-table transactions is not possible in lakehouse right?
Regards,
Srisakthi
Hi Srisakthi
Multi-table transactions are ONLY supported in the Data Warehouse. Lakehouse currently does not support this functionality.
Please check this document to further understand the use cases between the twoThanks and regards
Hi Srisakthi
Looks like multi-table transactions have only just hit Delta Lake 4.0 preview.
"Delta Lake is moving toward support for multi-statement and multi-table transactions, with Coordinated Commits, a centralized commit service that coordinates writes to Delta tables."
(https://delta.io/blog/delta-lake-4-0/)
I'm guessing that means it's coming, just not in the short term.
