Forum Discussion
Monitor dataset refresh
Hello sandlox
Pull the data from the Refresh History REST API (or Semantic Link in a Fabric notebook), using a service principal.
If you're on Fabric/Premium, Workspace Monitoring gives deeper detail.
Store the history in a Lakehouse or Warehouse on a schedule, appending each time. The API only keeps recent refreshes, so this is needed for trends.
Model it as a star schema: a refresh fact table plus dataset, workspace, capacity, and date dimensions. Include expected schedules and SLAs so you can catch missed refreshes, not just failed ones.
Track key measures: success rate, data age vs. SLA, duration vs. baseline, consecutive failures, concurrent refreshes, and top errors.
Build the report with an overview health matrix (worst-first), trends, failure drill-through, and a capacity view.
Add alerts through Activator or Power Automate to Teams or email, so you don't depend on people opening the report.
Keep it maintainable: monitor the monitor, apply RLS for workspace owners, and later add dataflows and pipelines for upstream lineage.
Are your datasets on Fabric/Premium capacity or Pro? That decides whether Workspace Monitoring is an option.
If my response helped you, please consider clicking
Accept as Solution and giving it a Like – it helps others in the community too.
Thanks,
Connect with me on:
LinkedIn |
Data With Pankaj - YouTube