The Problem
In Microsoft Fabric Data Warehouses, dependencies are often buried in thousands of lines of SQL code. This makes impact analysis manual and risky, as engineers cannot easily see the whole picture when modifying objects.
The Solution
An open-source Fabric Workload that provides interactive object-level lineage. It allows engineers to visualize and trace data flows across Tables, Views, and Stored Procedures directly within the native Fabric UI.
Architecture & Tech Stack
Built with the Fabric Extensibility Toolkit, this solution orchestrates native components to ensure data stays within your tenant:
- Fabric GraphQL API: Efficiently fetches item metadata and object definitions.
- Fabric Notebook: Handles the logic for SQL parsing and dependency extraction.
- Fabric Pipeline: Orchestrates the automated metadata refresh workflows.
- Fabric SQL Database: Stores the processed dependency graph for rapid UI retrieval.
- Frontend: A React-based UI integrated natively as a custom Fabric Workload.
Key Features
- Trace Mode: Right-click any object to find upstream sources or downstream consumers.
- Smart Filtering: Focus on specific schemas, object types, or classifications.
- Code Search: Full-text search across definitions with integrated syntax highlighting.
- Multi-Source: Seamlessly switch between different Fabric Data Warehouses.
- Export: Save your current lineage view as a PNG for documentation.
Scope Note: This tool provides Object-Level Lineage. Column-level lineage is not included.
Project Links
- Video Demo: Watch on YouTube
- Documentation & Project Site: https://chrisdevrepo.github.io/fabric_data_lineage/
