The ultimate Fabric, Power BI, SQL, and AI community-led learning event. Save €200 with code FABCOMM.
Get registeredAsk the Fabric Databases & App Development teams anything! Live on Reddit on August 26th. Learn more.
I am using data warehouse tables in notebooks. My initial setup was to attach the warehouse to the notebook and read the table. This is no longer possible and I am therefore creating lakehouse schema shortcut (using preview feature with lakehouse schemas) to data warehouse schema. This setup has been working until I started refreshing the sql endpoints of the lakehouse shortcuts (because they otherwise have outdated data or are not working when using sql endpoint) with
uri = f"/v1.0/myorg/lhdatamarts/{sqlendpoint_id}"
payload = {"commands":[{"$type":"MetadataRefreshExternalCommand"}]}
response = client.post(uri, json=payload)
Now I get this error: [DELTA_FEATURES_PROTOCOL_METADATA_MISMATCH] Unable to operate on this table because the following table features are enabled in metadata but not listed in protocol: invariants.
The sql endpoint of the lakehouse shortcut works fine. I can also preview the data in the lakehouse (not sql endpoint). I have tried reading with pyspark and t-sql in the notebook. They both throw the same error.
Any idea how to fix this?
Solved! Go to Solution.
Hi @mrtnhd ,
If your goal is to read/write data from/to the warehouse then check this spark connector for Datawarehouse!
https://learn.microsoft.com/en-us/fabric/data-engineering/spark-data-warehouse-connector?tabs=pyspar...
You don't have to create shortcuts on top of your warehouse tables!
Hope this is helpful, If so please accept this answer!
Hi @mrtnhd ,
If your goal is to read/write data from/to the warehouse then check this spark connector for Datawarehouse!
https://learn.microsoft.com/en-us/fabric/data-engineering/spark-data-warehouse-connector?tabs=pyspar...
You don't have to create shortcuts on top of your warehouse tables!
Hope this is helpful, If so please accept this answer!
Thank you so much! That seems to solve my issue. It works fine in at least one notebook.
User | Count |
---|---|
4 | |
2 | |
2 | |
2 | |
2 |
User | Count |
---|---|
13 | |
8 | |
7 | |
6 | |
6 |