Forum Discussion
shortcuts - Refresh Method
Hi fabricpribeiro ,
Thanks for the detailed clarification. Based on your answers, this appears closer to a SQL Analytics Endpoint metadata/cache synchronization issue over shortcut tables rather than expected shortcut behavior.
The important observation is that even the shortcut tables themselves become empty until refreshed. That suggests the SQL Endpoint is temporarily losing visibility of the shortcut metadata/state.
For the workaround, you can schedule a notebook against the Lakehouse containing the shortcuts/views and periodically refresh the metadata.
You can try the following PySpark code in a Fabric notebook to refresh the shortcut tables metadata, you can start with a simple refresh:spark.sql("REFRESH TABLE Purview_DQRules.table_name")
For multiple tables:
tables = ["table1","table2","table3"]for t in tables:
spark.sql(f"REFRESH TABLE Purview_DQRules.{t}")
print(f"Refreshed {t}")
The REST based MetadataRefreshExternalCommand shared earlier is also valid and refreshes the SQL Analytics Endpoint metadata more directly, which may be more effective in this scenario than only running REFRESH TABLE.Recommended frequency:
Start with every 15 minutes
If the issue persists, try every 5 minutes
If stable, increase gradually to 30 minutes
Best RegardsHi fabricpribeiro I read thru the conversation and I think the solution was to refresh the SQL Analytics Endpoint (SAE) metadata/cache on the source of the data and the community provided a couple of interest programatic solutions via yNotebooks, did it worked for you? Just curious since you have not accepted any solutions. If the solution is the refresh of the SAE and the question is wich alternatives are there to make it happend automatically and via schedule, I guess there's two:
#1. Calling the REST API Programmatically Refresh & Sync SQL Analytics Endp... - Microsoft Fabric Communityvia Notebooks (seems mosth popular) ... fouind this blog post from community member vojtechsima well explain
Programmatically Refresh & Sync SQL Analytics Endp... - Microsoft Fabric Community
#2. Calling new data pipeline activity Refresh SQL Endpoint Activity - Microsoft Fabric | Microsoft Learn , this is currently in preview, but it works very well 😁
I guess it will looks something like this :
I guess there was already a a lot of good information on the current chat, hope you can mark the solution that works for you so the community is aware what worked! If you find this useful, a thumbs up would be nice 😉, proposed also as solution if appropiete.
I am lost as I have here two different opnions. Can someone please help? I am not the only one which saw this issue on shorcuts I spoke as well with other people that say that from time to time they have to refreh their shorcuts. What to do? and with which SQL / phyton command? Thanks a lot
Hi fabricpribeiro ,
In most cases, shortcuts should expose the latest data without requiring manual intervention. However, there are known scenarios where the SQL Analytics Endpoint metadata/cache over shortcut-based views becomes stale, especially after inactivity or upstream changes.
A few clarifications that would help isolate the issue:
Are the views created in the Lakehouse that contains the shortcuts?
Is the semantic model using Direct Lake, DirectQuery, or Import?
When the issue occurs, do the shortcut tables themselves show data, or only the views become blank?
Does manually refreshing the SQL Analytics Endpoint/Lakehouse immediately resolve it?
For production workloads, customers commonly use a scheduled notebook or pipeline as a workaround to refresh metadata periodically.
Hello,
Please find the answer to your questions below:
A few clarifications that would help isolate the issue:
Are the views created in the Lakehouse that contains the shortcuts? Yes
Is the semantic model using Direct Lake, DirectQuery, or Import? Direct SQL not Direct onelake as those are views not tables
When the issue occurs, do the shortcut tables themselves show data, or only the views become blank?No, you have to refresh them as well , only after you see the data in the shorcuts and in the views
Does manually refreshing the SQL Analytics Endpoint/Lakehouse immediately resolve it?So far it did, its the first time today that even refreshing the shorcuts its not solving it
For production workloads, customers commonly use a scheduled notebook or pipeline as a workaround to refresh metadata periodically.
What is the code (python or SQL) to add into the notebook so that it does this refresh? and what is the frequency with which it should be refreshed?
Hi fabricpribeiro ,
Thanks for the detailed clarification. Based on your answers, this appears closer to a SQL Analytics Endpoint metadata/cache synchronization issue over shortcut tables rather than expected shortcut behavior.
The important observation is that even the shortcut tables themselves become empty until refreshed. That suggests the SQL Endpoint is temporarily losing visibility of the shortcut metadata/state.
For the workaround, you can schedule a notebook against the Lakehouse containing the shortcuts/views and periodically refresh the metadata.
You can try the following PySpark code in a Fabric notebook to refresh the shortcut tables metadata, you can start with a simple refresh:spark.sql("REFRESH TABLE Purview_DQRules.table_name")
For multiple tables:
tables = ["table1","table2","table3"]for t in tables:
spark.sql(f"REFRESH TABLE Purview_DQRules.{t}")
print(f"Refreshed {t}")
The REST based MetadataRefreshExternalCommand shared earlier is also valid and refreshes the SQL Analytics Endpoint metadata more directly, which may be more effective in this scenario than only running REFRESH TABLE.Recommended frequency:
Start with every 15 minutes
If the issue persists, try every 5 minutes
If stable, increase gradually to 30 minutes
Best Regards