Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
thedatacrew
Frequent Visitor

SQL Endpoint for Lakehouse Broken in MS Fabric

Given the following scenario

 

1. Create a table in Databricks with column mapping enabled.

 

 

%python
spark.conf.set("spark.databricks.delta.properties.defaults.minWriterVersion", 5)
spark.conf.set("spark.databricks.delta.properties.defaults.minReaderVersion", 2)
spark.conf.set("spark.databricks.delta.properties.defaults.columnMapping.mode", "name")

 

 

2. Add shortcut tables from Databricks to Synapse Lakehouse. The data shows correctly.

3. Go to the SQL Endpoint. Tables fail to load. with the error.  Corrective Action: Recreate the table without column mapping property.

4. This has a knock-on effect with data failing to feed into the datasets and Power BI.

 

Is the SQL Endpoint for Lakehouse going to support name column mapping mode?

 

Thanks

5 REPLIES 5
tissonmathew
New Member

+1 vote for the idea / fix

thedatacrew
Frequent Visitor
tissonmathew
New Member

I get the same error - using Databricks with Unity Catalog enabled. I think Fabric needs to update its Delta version?

Exactly. The Spark Engine (Lakehouse) can read it ok, it's the SQL Endpoint that is not working.

 

It's also a bit of a pain with Unity using GUID's for its storage layout. Not a massive problem but it would be super nice if Synapse Lakehouse shortcuts could read the Unity Catalog Metadata and we can just click and add tables.

Helpful resources

Announcements
Europe Fabric Conference

Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

PBI_Carousel_NL_June

Fabric Community Update - June 2024

Get the latest Fabric updates from Build 2024, key Skills Challenge voucher deadlines, top blogs, forum posts, and product ideas.

MayFBCUpdateCarousel

Fabric Monthly Update - May 2024

Check out the May 2024 Fabric update to learn about new features.