Metadata Driven Framework
Git Hub Code Repo : https://github.com/NandanHegde15/MSFTFabric-Projects/blob/main/Metadata%20Driven%20Framework/README.md Demo : https://www.youtube.com/watch?v=Z7mjHmX8-40 Metadata Driven Framework is a Microsoft Fabric-based end-to-end healthcare analytics platform . This project demonstrates modern data engineering practices including: Multi-layer data architecture (Raw → Silver → Gold layers) FHIR healthcare data modeling with support for patients, claims, coverage, and organizations Automated data pipelines for ingestion, transformation, and aggregation Power BI semantic models for enterprise analytics and reporting Row-level security for data governance and privacy AI-powered insights using Copilot & Data Agent capabilities Authors : Hemanth Kotha, Nandan Hegde https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DZ7mjHmX8-40Medallion Architecture in Microsoft Fabric: Building an AI-Ready Data Pipeline
Video Walkthrough Link------------> (https://raw.githubusercontent.com/trivedisunita/Fabric-Data-Factory-Contest/main/Fabric%20Short%20Video2.mp4) This project presents an end-to-end data pipeline built for the Data Factory Challenge using Microsoft Fabric. The objective was to transform raw job salary data into a structured, analytics-ready format for insights. ------------- Architecture Overview-------- Step1- ##Bronze Layer:-- Raw data is ingested into a Lakehouse from a SharePoint source without transformations, ensuring data is preserved in its original format. Step 2-- ##Silver Layer:-- Data is processed using Data Factory pipelines and stored in a Data Warehouse schema. Transformations include data cleaning, handling null values, and standardizing fields. SQL scripts and Copilot were used to assist in transformations. Step--3 ##Metadata Pipeline:--- Metadata-driven pipeline is implemented to manage and monitor data flow across layers. It enables dynamic execution, parameter handling, and improves pipeline scalability and maintainability within Microsoft Fabric. Step--4 ##Gold Layer:-- Data is modeled using dbt to create fact and dimension tables. A star schema is built for efficient querying, with reusable transformations such as converting coded fields into readable formats. Step--5 ##Semantic Model & Reporting:-- An AI-ready semantic model is created and used in Power BI. The model is optimized by selecting relevant columns and defining structure for better performance and accurate insights. Interactive reports are built on top of this model, enabling natural language queries using Copilot and delivering reliable business insights. https%3A%2F%2Fyoutu.be%2FskgWJyBPMcs
Medallion Architecute with Fabric Meta Data Driven Framework
The Medallion Architecture built on the Microsoft Fabric Metadata-Driven Framework provides a scalable and enterprise-ready approach for processing customer data across Landing Zone, Bronze, Silver, and Gold layers. Customer data from multiple enterprise sources such as CRM systems, APIs, applications, transaction systems, files, and streaming platforms first lands in the Landing Zone within OneLake, ensuring raw data preservation, auditability, and traceability. The Bronze layer standardizes and validates incoming data, while transformations from Bronze to Silver are implemented using Dataflow Gen2 with Copilot-assisted transformations to accelerate development and improve productivity. Both the Silver and Gold layers are built using Microsoft Fabric Warehouse capabilities, enabling high-performance analytical storage, governed modeling, and scalable business-ready datasets. The curated Gold layer powers Semantic Models and Customer 360 Reports built in Power BI, providing a unified view of customer interactions, behavior, transactions, and operational insights. The reporting layer also leverages Copilot-verified answers to enhance business user experience by enabling trusted AI-assisted analytics and natural language-driven insights. Microsoft Fabric services including OneLake, Data Factory, Data Engineering, Dataflow Gen2, Warehouse, Semantic Models, Power BI, and Purview work together seamlessly under a centralized metadata-driven framework with governance, lineage, monitoring, and reusable pipelines. This architecture enables organizations to deliver scalable analytics, accelerate decision-making, and create a modern AI-enabled customer intelligence platform. Medallion Architecute with Fabric Meta Data Driven Framework https://youtu.be/6ffjPLPItLc Copilot assisted transformation in dataflow Gen2 in Fabric https://youtu.be/1MQP9IT3DI0 PowerPoint Presentation: https://github.com/Geekkailash/fabrick-customer360/blob/main/FMD%20Framework.pptx https://github.com/Geekkailash/fabrick-customer360/blob/main/FMD%20Framework.pptx Code Link in Github: https://github.com/Geekkailash/fabrick-customer360 https://github.com/Geekkailash/fabrick-customer360 Author: Kailash Singh Bhandari Github Profile: https://github.com/Geekkailash
