Forum Discussion

alozovoy's avatar
alozovoy
Icon for Advocate II rankAdvocate II
2 years ago
Solved

SQL Endpoint Slow To Reflect Changes In Lakehouse

I have been using notebooks to create and modify delta tables for several months. I run the notebooks in the browser app and then I query the table using the SQL endpoint to validate the changes. I u...
data engineering
  • alozovoy's avatar
    alozovoy
    2 years ago

    Microsoft support told me that it is a known issue and is in their internal issues tracker. However, it will not be added to the public Fabric Known Issues page. There is currently no timeline for a fix.

frithjof_v's avatar
frithjof_v
Icon for Community Champion rankCommunity Champion
2 years ago

Thanks for the update and also providing a workaround!

 

Is this issue about the case where we do some changes to the table schema?

(E.g. add columns, remove columns, rename columns, etc.)

 

Or is it about regular update of the data in the table (with no alterations to columns)?

 

Or both scenarios?

Anonymous's avatar
Anonymous
Not applicable
2 years ago

There are a few reasons why you will get stale data when querying your sql endpoint. If you are loading data into a table via a notebook, the first query you run on the endpoint will freeze the data for all other queries until it completes. If the initial query is long running, you will get stale data presented by all subsequent queries. This is the fix Microsoft are implementing. 

We have also observed cases where data lags because the metadata sync job can be blocked by stats generation.