Forum Discussion
INFORMATION_SCHEMA
Hi all,
I’m currently migrating tables from Oracle into Microsoft Fabric (Lakehouse) and I’m trying to show progress / coverage of the migration to management.
What I’m trying to achieve The requirement is straightforward:
- Take a list of all Oracle tables (source of truth)
- Take a list of tables currently landed in Fabric
- Compare the two to show: What has already been migrated What remains
What I’ve done so far
On the Oracle side, I’ve extracted table metadata using:
SELECT DISTINCT owner, table_name FROM all_tab_cols;
That result is loaded into a dbo table in my Fabric Lakehouse and works well.
On the Fabric side, I’m querying the SQL analytics endpoint using:
SELECT * FROM INFORMATION_SCHEMA.TABLES;
This correctly returns the Lakehouse tables and matches what I see in the Explorer.
The problem
I can select from INFORMATION_SCHEMA.TABLES, but I can’t seem to do anything useful with it:
Joining it to my Oracle metadata table fails (distributed processing errors)
- CTAS / INSERT / materialising it into a table fails
- Using it in notebooks (Spark SQL / DataFrames) isn’t possible — Spark can’t see INFORMATION_SCHEMA
Same behaviour whether I use Lakehouse SQL or Warehouse - So effectively, INFORMATION_SCHEMA feels read‑only and isolated, usable only for inspection, not comparison.
What I’m trying to understand
Is this:
An expected Fabric design / control‑plane limitation?
Something others have run into when doing governance or migration tracking?
A case where the recommended pattern is: Dataflow Gen2? Pipelines? JDBC from a notebook? Warehouse‑only metadata?
I’m not looking for workarounds for the sake of it — I’d really like to understand what the supported approach is for comparing source‑system metadata with Lakehouse contents in Fabric.
Any insights or real‑world patterns would be greatly appreciated.
Thanks!
Hi cadams123
INFORMATION_SCHEMA in Fabric’s SQL analytics endpoint is a read-only system catalog that can’t be joined to Lakehouse Delta tables or materialized via CTAS . that’s by design, not a bug.
The standard workaround is to snapshot the metadata into a Delta table so you can join it with your Oracle metadata normally.
The spark.catalog.listTables() approach in a notebook remains your best bet for snapshotting Fabric metadata into a joinable Delta table.
3 Replies
- nilendraFabricSuper User
Hi cadams123
INFORMATION_SCHEMA in Fabric’s SQL analytics endpoint is a read-only system catalog that can’t be joined to Lakehouse Delta tables or materialized via CTAS . that’s by design, not a bug.
The standard workaround is to snapshot the metadata into a Delta table so you can join it with your Oracle metadata normally.
The spark.catalog.listTables() approach in a notebook remains your best bet for snapshotting Fabric metadata into a joinable Delta table.
- v-priyankataCommunity Support
Hi cadams123
Thank you for reaching out to the Microsoft Fabric Forum Community.
nilendraFabric Thanks for the inputs.
I hope the information provided by user was helpful. If you still have questions, please don't hesitate to reach out to the community.
- v-priyankataCommunity Support
Hi cadams123
Hope everything’s going smoothly on your end. I wanted to check if the issue got sorted. if you have any other issues please reach community.