Supplies are limited. Contact info@espc.tech right away to save your spot before the conference sells out.
Get your discountScore big with last-minute savings on the final tickets to FabCon Vienna. Secure your discount
I am using data warehouse tables in notebooks. My initial setup was to attach the warehouse to the notebook and read the table. This is no longer possible and I am therefore creating lakehouse schema shortcut (using preview feature with lakehouse schemas) to data warehouse schema. This setup has been working until I started refreshing the sql endpoints of the lakehouse shortcuts (because they otherwise have outdated data or are not working when using sql endpoint) with
uri = f"/v1.0/myorg/lhdatamarts/{sqlendpoint_id}"
payload = {"commands":[{"$type":"MetadataRefreshExternalCommand"}]}
response = client.post(uri, json=payload)
Now I get this error: [DELTA_FEATURES_PROTOCOL_METADATA_MISMATCH] Unable to operate on this table because the following table features are enabled in metadata but not listed in protocol: invariants.
The sql endpoint of the lakehouse shortcut works fine. I can also preview the data in the lakehouse (not sql endpoint). I have tried reading with pyspark and t-sql in the notebook. They both throw the same error.
Any idea how to fix this?
Solved! Go to Solution.
Hi @mrtnhd ,
If your goal is to read/write data from/to the warehouse then check this spark connector for Datawarehouse!
https://learn.microsoft.com/en-us/fabric/data-engineering/spark-data-warehouse-connector?tabs=pyspar...
You don't have to create shortcuts on top of your warehouse tables!
Hope this is helpful, If so please accept this answer!
Hi @mrtnhd ,
If your goal is to read/write data from/to the warehouse then check this spark connector for Datawarehouse!
https://learn.microsoft.com/en-us/fabric/data-engineering/spark-data-warehouse-connector?tabs=pyspar...
You don't have to create shortcuts on top of your warehouse tables!
Hope this is helpful, If so please accept this answer!
Thank you so much! That seems to solve my issue. It works fine in at least one notebook.
User | Count |
---|---|
4 | |
4 | |
2 | |
2 | |
2 |
User | Count |
---|---|
10 | |
8 | |
7 | |
6 | |
6 |