We are pushing data to a Deltatable on Onelake, managed by an external service using the open source delta-rs library. After creating a shortcut to this table, we are using a Direct Lake connection to visualize the data i PowerBI. This works really well, and we find it to be a major upgrade to our existing pipelines based on DirectQuery mode. However, we recently discovered that this semantic model is loading our capacity quite a lot, in fact about 300,000 CUs the last 14 days. The user interaction with the PowerBI reports has been minimal, averaging to less than one user per day. This is really unexpected, given that no Fabric resources are used to update or maintain the Delta table. From my understanding, all that is needed from Fabric to synch the table would be to load the content of the _delta_log folder. This corresponds to reading about 100 json files and a parquet file. We do not do any data caching. I recently turned off synching of the default model, which reduced the load by 50%. Any suggestions on how I can reduce the load on the capacity? Can we hope for some improvements on the Fabric side in the future?

We have now succesfully reduced the load on the Fabric capacity to a low value, down from about 25 to 50%. The following steps were implemented:Turn off the "Keep your Direct Lake data up to date" option on the default semantic model in the LakehouseTurn off the "Sync the default Power BI semantic model" in the SQL analytics endpointTurn off the "Keep your Direct Lake data up to date" option on the considered semantic modelSynch the model via the API every 5 minutes. Use a Logic app, Azure function or any service with timer function. Every refresh of the semantic model now takes about 20 seconds. The capacity now spends about 20/300 of the time to refresh model. Note that this solution introduce some additional delay between the data source and the report - up to 5 minutes. . Disabling the default model semantic model had a major impact on the capacity load. Exactly what is going on in the background with this model is unclear. For our use case, triggering model refresh only from PowerBI or another downstream load would be the ideal solution.

Shortcut to deltatable on Onelake generates high CU

Forum Discussion

Related content

Shortcut caching

shortcuts - Refresh Method

Reading an existing DeltaTable throws DELTA_MISSING_DELTA_TABLE error

Sharepoint Shortcut

Shortcuts