Forum Discussion
Semantic model refresh does not refresh data in report
I think I may have a solution... I've added a step to hit the endpoint with a simple query before refreshing my model, looks like it might do the trick.
You may want to follow my thread on the subject here: https://community.fabric.microsoft.com/t5/Service/Semantic-Model-connected-to-Lakehouse-SQL-Analytics-endpoint-not/m-p/4828558/highlight/false#M286453
I think I may have a solution... I've added a step to hit the endpoint with a simple query before refreshing my model, looks like it might do the trick.
You may want to follow my thread on the subject here: https://community.fabric.microsoft.com/t5/Service/Semantic-Model-connected-to-Lakehouse-SQL-Analytics-endpoint-not/m-p/4828558/highlight/false#M286453
Hi DuncanKing,
Your reply is most useful, although I also need time to confirm if it works as expexted.
My gut feeling says that this might be the solution, and your post would also perfectly explain why my 6AM scheduled pipeline run does not update the semantic model, while when I run it later (after I made some queries on the Lakehouse) it does the update.
Now I also added a Lookup activity after the Gold pipeline where I just get the MAX of my TransactionId. In theory this would "warm up" the Lakehouses metadata, and when I do the Semantic model refresh after, it should get the new data.
I only implemented this method this morning though, so I also need a couple of days to confirm the validity of this claim but this also explains why I first got old data displayed when I ran an SQL query directly on my Lakehouse, but when I ran the query again right after it showed the new data; the metadata was not updated because the endpoint was not warm.
Fingers crossed 🤞
P.S.: I wonder if the same applies in case we have the Semantic model refresh activity in the pipeline instead of the API call... 🤔
Well, my refresh looks to have worked last night, so I'm becoming more confident that this is the solution!
So yeah, it looks like just hitting the endpoint with a simple SQL query and then waiting 60 seconds before running the refresh activity is enough for the model to pick up fresh data - you do need that wait though!
DuncanKing , thank you, your suggested solution worked!
I first tried a Lookup activity with a query and without a wait activity, the next day with a wait acitivity, but finally the script activity with the SELECT TOP 1 * and the wait activity combination worked today.
Thank you a lot, I will mark you post as the solution.Great, glad to be able to help!
It really shouldn't be this hard, should it?
Indeed DuncanKing,
Either there should be an option in the refresh activity or scheduled refresh to "warm up" the data source by doing a pre-operation query or similar, or there should be an easy to find and understand documentation about this behaviour with easy to follow solution tutorial.Hopefully you forum post or maybe this one gets picked up by the community at least...