patrickmaul's avatar
patrickmaul
Frequent Visitor
29 days ago

Fabric Lineage Workbench

Hi everyone, I want to share my custom Microsoft Fabric workload project: Fabric Lineage Workbench.

Overview

 

Detail view

 

AI explanation

 

What it currently can do

This first version focuses on visualizing dependencies inside semantic models and reports. It can also interpret DirectLake and table shortcuts within Lakehouses and Warehouses.

  • See sources of Power BI Report visuals.
  • See upstream and downstream elements of measures, tables and columns
  • See connections between Report, Semantic Model and Lakehouses/Warehouses
  • Get help interpreting Measures and calculated columns

For more information, visit patrickmaul.de

Background and why I developed this

I am a freelance BI consultant working mainly with Fabric and Power BI. While I really like the approach of bringing data engineering and data analytics into one ecosystem, one thing has always frustrated me: understanding and documenting dependencies and lineage across several Fabric artifacts and inside large semantic models.

So, I decided to use the Microsoft Fabric Extensibility Toolkit to create a solution that can help me and my customers understand how things are connected. This first version I want to share focuses on semantic models and Power BI reports. It can visualize where measures, tables, and columns are used and what needs to be checked when something is changed.

I also wanted to minimize documentation effort. This is achieved through metadata extraction directly from the Fabric artifacts.

How does it work

A complete walkthrough can be found on patrickmaul.de/walkthrough.html

  1. Setup all additional elements via Setup Wizard
  2. Extract and process metadata from different workspaces via Notebooks
  3. Store data in Lakehouse
  4. Visualize the data as graph or in detail view.

What is planned for the future

If there is enough interest in my small project, I already have plans for the future. However, I am also always open to suggestions

  • Better extraction performance - metadata extraction takes between 10 and 30 minutes right now.
  • Integration of legacy PBI report format - right now, it only works with reports in PBIR format
  • Integration of Dataflows Gen 2, Notebooks and SQL Queries inside Lakehouses/Warehouses
  • Extension of Requirement management - right now, it is very barebones. My plan is to have a requirements board, so Teams can communicate directly inside Fabric.
  • Better AI integration - right now, you can already use your own Azure AI foundry account to get information about DAX Measures and M queries. I want to extend this to get a better overview over lineage and dependencies.
  • Better filter options - right now, you can filter by names and node types. I want to give the user more information, for example broken visuals, unused measures, etc.

How to get it

The custom workload is free right now; however, I reserve the right to add premium features in the future.

You can find the repo here: https://github.com/pmaul87/fabric-lineage-workbench

How to set up a workload: 

https://learn.microsoft.com/en-us/fabric/extensibility-toolkit/setup-guide

 

If you need any help or have any feedback, it would be greatly appreciated. 

No RepliesBe the first to reply