Forum Discussion
SQL Endpoint Slow To Reflect Changes In Lakehouse
Microsoft support told me that it is a known issue and is in their internal issues tracker. However, it will not be added to the public Fabric Known Issues page. There is currently no timeline for a fix.
This is a known issue with the metadata sync process which results in stale data when queries through the sql endpoint.
The fix was scheduled for July 31 but has unfortunately been pushed back to September.
The workaround is to create a separate workspace and Lakehouse, shortcut your existing Lakehouse tables into the new Lakehouse and the data will be fresh.
Thanks for the update and also providing a workaround!
Is this issue about the case where we do some changes to the table schema?
(E.g. add columns, remove columns, rename columns, etc.)
Or is it about regular update of the data in the table (with no alterations to columns)?
Or both scenarios?
There are a few reasons why you will get stale data when querying your sql endpoint. If you are loading data into a table via a notebook, the first query you run on the endpoint will freeze the data for all other queries until it completes. If the initial query is long running, you will get stale data presented by all subsequent queries. This is the fix Microsoft are implementing.
We have also observed cases where data lags because the metadata sync job can be blocked by stats generation.
Or you can refer this blog to quickly fix this?
