Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
1 year ago
Solved

How to fetch Gen2 Dataflow run statuses and visualize in power BI?

Hi Fabric Community  I'm working on a solution where we monitor Fabric Gen2 Dataflow runs across multiple workspaces and log their statuses to a centralized table for reporting in Power BI. ✅ Goal:...
  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi Anonymous ,
    Thank you for reaching out to us!

    I would be happy to help you build a solution to monitor Fabric Gen2 Dataflow runs across multiple workspaces and create a Power BI dashboard with the details you outlined.
    Please follow below steps:

    We can use Fabric’s Monitoring Hub and a custom pipeline to gather refresh data, store it in a centralized table, and visualize it in Power BI. Below are the steps:

    1. Gather Dataflow Refresh Details:
    The Monitoring Hub in Fabric provides a unified view of Dataflow Gen2 activities across workspaces, including Status (Success/Failure), Start/End Times, Duration, Trigger Type (Manual/Scheduled), and more. For automation, use the Power BI REST API (e.g., GET /dataflows/transactions) to pull this data programmatically. Ensure your account or service principal has access to all relevant workspaces.

    2. Set Up a Fabric Data Pipeline:
    - Fetch refresh data via the Monitoring Hub or API.
    - Extract fields like Status, Start/End Time, Duration, Trigger Type, and Last Refreshed timestamp.
    - Save the data to a centralized table in a Fabric Lakehouse (e.g., Delta table) or Warehouse.
    - Schedule the pipeline to run periodically (e.g., hourly) to keep your data up-to-date.

    3. Build Your Power BI Dashboard:
    - Connect Power BI to your centralized table.
    - Create visuals: a color-coded table or card for Status (green for Success, red for Failure), a timeline or table for Start/End Times, a bar chart or KPI for Duration, a pie chart or slicer for Trigger Type, and a “Last Updated” card for Last Refreshed.
    - Add filters for Workspace and Dataflow Name to enhance navigation.
    Please refer the below links:
    https://learn.microsoft.com/en-us/fabric/admin/monitoring-hub
    https://learn.microsoft.com/en-us/fabric/data-factory/connector-rest 



    If the answer meets your requirement,consider accepting it as solution.

    Regards,
    Pallavi.