Fetching schema in Edit semantic model loads "forever"

[filip_ai]

I have a Lakehouse with delta tables, and have created a custom semantic model.

Everytime I want to add a new table, or a new column in a table already in the semantic model, through the Edit tables option, the Fetching schema part takes a really long time (10-15 min). After a long wait it shows the selectable tables, as usual, and I can either add the new table or the new column.

The issue is that fetching schema is taking much longer to complete, than I find feasible. One minute is OK, but longer than that does not make sense to me. The added table/column is queryable through the Lakehouse, but for some reason this schema update feeding the semantic model is much slower to reflect the change.

Any input on possible solutions to speed up the schema fetch or similar experiences?

[SnoekLaurens]

I'm experiencing the same issue. Never had a problem until now. suddenly editing the semantic model in Service is taking forever

[RubyM]

@brewed @RyanHaunfelder @SnoekLaurens  Logged a support ticket and Microsoft (or Mindtree) is saying the fix won't be happening until September 9th. 😑

[brewed]

Same issue here

[RubyM]

Getting the same error as well.  Only when the semantic model is connected to a DW.  It works fine if the semantic model is connected to a lakehouse.

[RyanHaunfelder]

I'm having the same issue. The tables never end up loading.

[JasonConway]

Hi @filip_ai - Did you manage to find a workaround for this? I'm facing the same issue, albeit with a Warehouse rather than a Lakehouse.

[FilipAi]

@JasonConway Kind of - I am using Tabular Editor to update the semantic model, which seems to be working fine.

[arpost]

@filip_ai,. do you know how to add a table from the DW using Tabular Editor? I have TE2, which doesn't have the snazzy Import Table feature of TE3.

[FilipAi]

@arpost if you read both JasonConway's posts in this thread, it should give you a clear way of working to achieve it in TE2 (which I am using). HTH

[JasonConway]

Ah, I've given this a try myself and not been sucessful. I'd be interested to hear how you've done it?

I apporached it by duplicating an existing table (to ensure that the Object Type was set to DQ over AS, as this doesn't appear to be something you can change after the table has been created) then chaging the source lineage tag to the relevant table within my warehouse and updating the fields. When I triggered a refresh on the model , rather than taking roughly 30 seconds like I'd be used to, it ran for over 15 minutes. I then cancelled it and concluded that adding the table via Tabular Editor wouldn't work...

[FilipAi]

@JasonConway that's the same steps I've taken. What about the columns? You might want to make sure to only include one column as a start, preferably with the same data type as the original table, to avoid any issues when duplicating the table.

What if you try to refresh the new table in SSMS? Can you see the new table there prior to refresh?

[JasonConway]

Got it now! Looks like I'd missed changing the Entity Name within partition itself, I'd instead just changed the source lineage tag on the table. All is now working.

[v-heq-msft]

Hi @filip_ai ,

According to your description, in power query you can use the architecture view. This feature is designed to optimize the flow when dealing with architecture-level operations by focusing on column information, which may reduce the delays you encounter.
Second, because the semantic model is built on Lakehouse, ensuring that you optimize these tables can significantly impact the speed of architecture extraction. Please consider applying table maintenance practices such as bin-compaction and V-Order to improve performance.
You canperform ad-hoc table maintenance using contextual right-click actions in a delta table within the Lakehouse explorer. Apply bin-compaction, V-Order, and unreferenced old files cleanup.

You can check the following documents for more details
Using Schema view (Preview) - Power Query | Microsoft Learn
Delta Lake table optimization and V-Order - Microsoft Fabric | Microsoft Learn
Use table maintenance feature to manage delta tables in Fabric - Microsoft Fabric | Microsoft Learn
Edit data models in the Power BI service (preview) - Power BI | Microsoft Learn

Best regards,
Albert He

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly

[FilipAi]

@v-heq-msft Thanks for your reply.

In short, none of your suggestions are helpful in practice, at least not at this stage. When I wrote the initial post, I had just added a new table with 4k rows (no other changes to the Lakehouse). For such an addition, to me it seems over the top to perform maintenance on the delta tables, and my expectation is that making the "architectural" change should work smoothly without that. Extrapolating this table size, I am wondering how this will work with larger tables... We cannot perform maintenance on a Lakehouse everytime we want to do schematic changes.

Also, I am not sure what you mean with "According to your description, in power query you can use the architecture view". I don't see a power query (or PQ Schema) view for editing the semantic model. As far as I understand, Power Query comes in to play when working with dataflows in the lakehouse? Anyhow, the operation I tried to do was simply editing the model within in the Power BI service (well, Fabric service).