Forum Discussion
Error Writing DataFrame to Warehouse via synapsesql – Worked Previously
Hi Anonymous,
here is the Doc. for OneLake SAS Key.
https://learn.microsoft.com/en-us/fabric/onelake/onelake-shared-access-signature-overview
https://learn.microsoft.com/en-us/fabric/onelake/how-to-create-a-onelake-shared-access-signature
I think that helps you.
Best regards
Super UserHi Anonymous
Due to change into synapsesql, means that synapse sql should not be used going forward ( depreciate ) we need to use Databricks Delta Lake Open Source table in broze+ silver warehouse. synapse sql is using data lake whereas databricks delta lake is using _delta_log( Transaction log ) with parquet files. You should not use spark notebook for everything.
Python --> Apache Spark ( Data Lake ) --> Databricks Delta Lake ( Pyspark initially and subsequent use Spark SQL )
Also, Lakehouse = Fabric Warehouse = Power BI / SSAS Tabular Semantic Model
Thank you, Bhavesh Patel, for your response.
However, your answer seems to be outside the scope of our current setup. We are working entirely within a Microsoft Fabric environment.
The objective is to write data from a Microsoft Fabric Lakehouse to a Microsoft Fabric Warehouse using a Fabric Notebook. Databricks is not our scope.
The synapsesql function was working correctly, and according to Microsoft, it has not been deprecated.
- BhaveshPatel
Initially, Databricks came up with Delta Lake Open Source and subsequently Microsoft came up with same Delta Engine and called it Microsoft Fabric.
Lakehouse = Fabric Warehouse = Power BI Semantic Model.
All three are same and doing the same purpose of Data Engineering, Data Analytics and Data Science.
synapsesql function is using only parquet files whereas Delta Lake is using parquet file + _delta_log.