Exposing metadata of the lakehouse tables

[smpa01]

If I am creating a table/view in warehouse (dw_silver) based on the tables ingested with pyspark in a lakehouse table (lh_bronze), is there any way for the sql endpoint to grab the file level metadata of the lakehouse (metadata from the underlying files in a lakehouse in the same way one can with SpakSQL)?

[BhaveshPatel]

Yes, you can get all file-level metadata from Lakehouse tables using azure storage explorer in Microsoft Azure. Go to ADLS Gen 2 and search for relevalt details of the metadata. or You can use Delta Analyzer as listed below:
https://github.com/microsoft/Analysis-Services/tree/master/DeltaAnalyzer

[Shahid12523]

SQL endpoints in Fabric can’t access file-level metadata from Lakehouse tables like SparkSQL can. To work around this, use PySpark to extract metadata (e.g., file size, history, schema), then save it as a table or view that SQL can query. It’s a manual bridge—but totally doable.

[v-prasare]

Hi @smpa01,

We are following up once again regarding your query. Could you please confirm if you have raised ideas/feedback with Microsoft?
we kindly request you to post in ideas forum and upvote this. If we don’t hear back, we’ll go ahead and close this thread.
Should you need further assistance in the future, we encourage you to reach out via the Microsoft Fabric Community Forum and create a new thread. We’ll be happy to help.

Thank you for your understanding and participation.

[v-prasare]

Hi @smpa01,

Direct access to Delta metadata is NOT available in SQL Endpoints. i suggest using Use Spark Notebooks to extract metadata and store it in a structured table if SQL access is needed.

I’d encourage you to submit your detailed feedback and ideas via Microsoft's official feedback channels, such as the Microsoft Fabric Ideas.

Feedback submitted here is often reviewed by the product teams and can lead to meaningful improvement.

Thanks,

Prashanth Are

MS Fabric community support