Forum Discussion
Reading data from Lakehouse Delta table
Hi ,
Can we read data from Lakehouse Delta table with Fabric capacity turned off .
Now in Architecture I am planning to keep Silver layer in Lakehouse so that ML model can refer it .
But I would like to Confirm Delta is open source format other Platforms can read it but does Capacity needs to in running state ?
Hi Umasankar_Sane , thank you for reaching out to the Microsoft Fabric Community Forum.
It appears that the ability to access the Lakehouse Delta tables is indeed tied to the Fabric capacity being active. The error message "Unable to complete this action because this fabric capacity is currently not active" indicates that the infrastructure managing the data needs to be running for access to occur, even though Delta Lake itself is an open-source format.
Please consider below steps to consider to ensure that your client's data isn't vendor-locked:
- Regularly export the Delta tables to an open format such as Parquet, CSV, or JSON. These formats can be easily read by other platforms without relying on specific infrastructure. Automate this process using tools like Apache Spark or Databricks to ensure data is always available in open formats.
- Employ Delta Lake tools such as Delta Standalone, Delta Rust, or Delta Lake on Apache Spark. These tools allow you to access Delta tables without depending on proprietary systems. Ensure that these tools are tested and integrated into your architecture.
- Design your architecture to support data portability. This includes using open-source data processing frameworks and avoiding proprietary dependencies as much as possible. Regularly back up and test the migration of data to other systems to ensure seamless transitions if needed.
If this helps, please consider marking it 'Accept as Solution' so others with similar queries may find it more easily. If not, please share the details.
Thank you.
4 Replies
Hi Umasankar_Sane - if the capacity is turned off you should be able to query the lakehouse data using SQL clients or other services that support SQL connections by using the SQL endpoint. What service are you using for your ML model? Is it outside of Fabric?
Hi Umasankar_Sane , thank you for reaching out to the Microsoft Fabric Community Forum.
Data from a Lakehouse Delta table can be read even if Fabric capacity is turned off. SQL clients or other services supporting SQL connections can utilize the SQL endpoint for access.Delta Lake, being an open-source format, allows other platforms to read Delta tables as long as they support Delta Lake.
However, Optimal performance and seamless data access are ensured by keeping the Fabric capacity running. This helps maintain the health and performance of Delta tables through regular maintenance operations such as optimization and vacuuming.
If this helps, please consider marking it 'Accept as Solution' so others with similar queries may find it more easily. If not, please share the details.
Thank you.Thank you for quick response.
I could not respond as I had to test this Scenario by turning off ( PAUSE ) fabric capacity.
Gettig below when I tried to access Lake House from SSMS.
Let me know If I have to try to something else . My main point was to check that my client data should not be venor locked ( Microsoft in this case ) . 🙂
Hi Umasankar_Sane , thank you for reaching out to the Microsoft Fabric Community Forum.
It appears that the ability to access the Lakehouse Delta tables is indeed tied to the Fabric capacity being active. The error message "Unable to complete this action because this fabric capacity is currently not active" indicates that the infrastructure managing the data needs to be running for access to occur, even though Delta Lake itself is an open-source format.
Please consider below steps to consider to ensure that your client's data isn't vendor-locked:
- Regularly export the Delta tables to an open format such as Parquet, CSV, or JSON. These formats can be easily read by other platforms without relying on specific infrastructure. Automate this process using tools like Apache Spark or Databricks to ensure data is always available in open formats.
- Employ Delta Lake tools such as Delta Standalone, Delta Rust, or Delta Lake on Apache Spark. These tools allow you to access Delta tables without depending on proprietary systems. Ensure that these tools are tested and integrated into your architecture.
- Design your architecture to support data portability. This includes using open-source data processing frameworks and avoiding proprietary dependencies as much as possible. Regularly back up and test the migration of data to other systems to ensure seamless transitions if needed.
If this helps, please consider marking it 'Accept as Solution' so others with similar queries may find it more easily. If not, please share the details.
Thank you.
