Forum Discussion
Semantic Model not Updating Values properly
Hello everyone,
I am facing an issue while trying to update my semantic model.
I deployed a semantic model of type DirectLake (Automatic Behavior) with a Warehouse as the source.
After deployment, I did not perform a refresh, yet the reports were functional and updated daily with new data.
Yesterday, I made changes to some previously loaded data in the Warehouse by updating trailing spaces from certain values. For example, I updated "XXXXX " to "XXXXX". (TRIM values)
In the Warehouse, everything looks correct. However, when I opened the report, the values still appeared with the trailing spaces.
The semantic model setting Keep your Direct Lake data up to date is enabled.
I then performed a refresh on the semantic model (although I don’t understand why this should be necessary), and after that, I noticed two values:
The original one with spaces
The new one without spaces
I also tried a full refresh from the XMLA endpoint, but again, both the old and new records appeared.
Additionally, I changed the Query Caching setting from Default to Off, performed another refresh, and still observed the same behavior.
My questions are:
Why do I need to perform a refresh to see these changes? Shouldn’t the model automatically capture them?
How can I refresh the model so that it overwrites the old values instead of keeping both?
Why do the old values still appear even after a full refresh?
Thank you in advance,
Dimitra
yeah, Warehouse manages maintenace for Delta automatically
8 Replies
Hi Anonymous ,
The refresh on the on Direct Lake mode does not actually does the update of the data but it does the rebinding of the folder that are the source of each table in the lakehouse and warehouse.
Since direct lake mode uses the parquet files capacities to make the queries it may need rebinding when some major changes occur to your data. This occurs because the way the parquet files are created when you update your data or create new records is not the same way has in a database where those specific records are simply overwritten.
A good practice is to run the maintenance tools on your table (Optimize, Vaccum and V-order) when major changes are done to your data or to your schemas so that the parquet files get the latest information, and also reduce the number of files created in the warehouse and clean old ones.
This is not a easy topic check some links about those data optimization:
the MFelix 's answer is very good, but keep in mind that Warehouse have auto maintenance therefore you don't have to do it.
in anycase as said the concept of framing is important to manange in DirectLake scenarios: Direct Lake overview - Microsoft Fabric | Microsoft LearnHello MFelix
Thanks for the clarification. In my case, the source is Warehouse, and as per my understanding, the Warehouse is supposed to handle this maintenance automatically, rather than requiring manual execution of Optimize/Vacuum/V-Order.
Could you confirm if that’s correct, or if there are scenarios where we would still need to take care of it manually on the Warehouse side?
yeah, Warehouse manages maintenace for Delta automatically
Hi Anonymous ,
You’re absolutely correct Direct Lake operates differently than traditional databases. In Direct Lake mode, updates don’t overwrite existing rows; instead, each change in the Warehouse or Lakehouse results in new parquet files, with older versions remaining until you explicitly clean them up. That’s why, after trimming trailing spaces, both old and new values appeared in your report.
Refreshing in Direct Lake only rebinds tables to the source files and doesn’t remove outdated versions. To ensure your model displays only the latest data, it’s essential to run table maintenance operations like OPTIMIZE, VACUUM, and V-ORDER. OPTIMIZE and V-ORDER will improve query performance by compacting and ordering files, while VACUUM removes obsolete file versions. Once these steps are completed, Direct Lake queries will reflect only the updated data.
This means your model updates automatically, but older values persist until cleaned up. Running maintenance after major changes guarantees your reports show only the correct, up-to-date results.
Thank you,
Tejaswi.
Hi Anonymous ,
I wanted to follow up and see if you had a chance to review the information shared. If you have any further questions or need additional assistance, feel free to reach out.
Thank you.
Hi @dimitragav10_ ,
I wanted to follow up and see if you had a chance to review the information shared. If you have any further questions or need additional assistance, feel free to reach out.
Thank you.
