Forum Discussion
How to fetch Gen2 Dataflow run statuses and visualize in power BI?
- Anonymous1 year ago
Hi Anonymous ,
Thank you for reaching out to us!I would be happy to help you build a solution to monitor Fabric Gen2 Dataflow runs across multiple workspaces and create a Power BI dashboard with the details you outlined.
Please follow below steps:We can use Fabric’s Monitoring Hub and a custom pipeline to gather refresh data, store it in a centralized table, and visualize it in Power BI. Below are the steps:
1. Gather Dataflow Refresh Details:
The Monitoring Hub in Fabric provides a unified view of Dataflow Gen2 activities across workspaces, including Status (Success/Failure), Start/End Times, Duration, Trigger Type (Manual/Scheduled), and more. For automation, use the Power BI REST API (e.g., GET /dataflows/transactions) to pull this data programmatically. Ensure your account or service principal has access to all relevant workspaces.2. Set Up a Fabric Data Pipeline:
- Fetch refresh data via the Monitoring Hub or API.
- Extract fields like Status, Start/End Time, Duration, Trigger Type, and Last Refreshed timestamp.
- Save the data to a centralized table in a Fabric Lakehouse (e.g., Delta table) or Warehouse.
- Schedule the pipeline to run periodically (e.g., hourly) to keep your data up-to-date.3. Build Your Power BI Dashboard:
- Connect Power BI to your centralized table.
- Create visuals: a color-coded table or card for Status (green for Success, red for Failure), a timeline or table for Start/End Times, a bar chart or KPI for Duration, a pie chart or slicer for Trigger Type, and a “Last Updated” card for Last Refreshed.
- Add filters for Workspace and Dataflow Name to enhance navigation.
Please refer the below links:
https://learn.microsoft.com/en-us/fabric/admin/monitoring-hub
https://learn.microsoft.com/en-us/fabric/data-factory/connector-rest
If the answer meets your requirement,consider accepting it as solution.
Regards,
Pallavi.
Hi Anonymous ,
We haven't heard back from you regarding our last response and wanted to check if your issue has been resolved.
If our response addressed your query, please mark it as Accept Answer and click Yes if you found it helpful.
If you have any further questions, feel free to reach out.
Thank you for being a part of the Microsoft Fabric Community Forum!