rss.livelink.threads-in-node

Fabric Monday 121: Human in The Loop

DennesTorres — Tue, 22 Sep 2026 15:50:58 GMT

🚨 A Business Event can start a Fabric pipeline automatically. But should every event be allowed to complete the process without a human decision?

Business Events turn important data conditions into governed, reusable signals. The new Approval activity puts a human decision directly inside a Fabric Data Factory pipeline.

Combine them, and you get an event-driven workflow that stays automatic until judgment is actually required.

📡 FROM DATA TO A BUSINESS EVENT

In this video, a Fabric notebook checks incoming sales records for orders with a quantity greater than 15.

When it finds one, it publishes a `HighQuantityOrderDetected` Business Event containing the invoice, item, quantity, and total value.

The event uses a registered schema, so publishers and consumers exchange the same business context in a predictable format.

⚡ FROM BUSINESS EVENT TO PIPELINE

Activator monitors the Business Event in Real-Time Hub.

For each matching event, Activator starts a Fabric pipeline and passes the event fields into pipeline parameters. The pipeline can then enrich the request with additional data before asking for approval.

👤 HUMAN IN THE LOOP

The Approval activity pauses the pipeline and sends the request to a designated reviewer through Outlook or Microsoft Teams.

The reviewer sees the business context, opens the Review experience in Monitoring Hub, and chooses whether to approve or reject the request.

👉 Approved — the pipeline continues through the success path
👉 Rejected — the pipeline continues through the failure path
👉 Timeout — the activity fails and follows the rejection path

After the decision, the pipeline resumes automatically.

💾 KEEPING THE DECISION

Both branches call a notebook that records the result in the Lakehouse.

The saved record includes the event values, approval result, pipeline run ID, and decision timestamp. The original signal, the human decision, and the downstream action remain connected and traceable.

🎯 WHY THIS COMBINATION MATTERS

Business Events remove the need for every producer and consumer to build a private integration contract.

The Approval activity adds a controlled decision point without moving the process outside Fabric.

Together they enable workflows that are:

👉 Event-driven — the process starts when a meaningful business condition occurs
👉 Context-aware — the event payload travels into the approval request
👉 Governed — only the step that needs judgment is paused
👉 Auditable — the pipeline records who decided, the outcome, and what happened nextreti

This is where real-time detection becomes an operational business process: automatic detection, human judgment, and an automated response in one Fabric workflow.

⚠️ The Business Events alert integration and the Approval activity shown in this video are currently in Preview.

Fabric Monday 117: SCD Type 2 Uing Copy Job

DennesTorres — Mon, 07 Sep 2026 13:23:28 GMT

🔄 SCD Type 2 has been one of the most needed — and most painful — patterns in data warehousing for decades.

Every time a dimension record changes, you need to preserve the old version and insert a new one. Valid_From, Valid_To, Is_Current. Simple concept. Traditionally brutal to implement.

Copy Job in Microsoft Fabric now does this with a single toggle.

📋 WHAT COPY JOB DOES FOR YOU

When a record changes in the source:

👉 The existing current row gets Valid_To set + Is_Current = false
👉 A new row is inserted with the updated values, Valid_From, and Is_Current = true

When a record is deleted at the source:
👉 Soft delete — the destination row is closed (Valid_To + Is_Current = false). History is never lost.

No MERGE logic. No custom code per table. No framework to maintain.

⚙️ ONE STEP

1. Create a Copy Job
2. Select a CDC-enabled source
3. Select a supported destination
4. Set the write method to SCD Type 2

That's it.

🔗 LINKED TO INCREMENTAL LOAD

SCD Type 2 in Copy Job is built on top of Change Data Capture (CDC). The incremental load detects what changed — Copy Job applies the SCD2 history pattern automatically.

watch?v=tAktgDFTYHw

Medallion Architecute with Fabric Meta Data Driven Framework

geekkailash — Fri, 08 May 2026 18:13:40 GMT

The Medallion Architecture built on the Microsoft Fabric Metadata-Driven Framework provides a scalable and enterprise-ready approach for processing customer data across Landing Zone, Bronze, Silver, and Gold layers. Customer data from multiple enterprise sources such as CRM systems, APIs, applications, transaction systems, files, and streaming platforms first lands in the Landing Zone within OneLake, ensuring raw data preservation, auditability, and traceability. The Bronze layer standardizes and validates incoming data, while transformations from Bronze to Silver are implemented using Dataflow Gen2 with Copilot-assisted transformations to accelerate development and improve productivity. Both the Silver and Gold layers are built using Microsoft Fabric Warehouse capabilities, enabling high-performance analytical storage, governed modeling, and scalable business-ready datasets.

The curated Gold layer powers Semantic Models and Customer 360 Reports built in Power BI, providing a unified view of customer interactions, behavior, transactions, and operational insights. The reporting layer also leverages Copilot-verified answers to enhance business user experience by enabling trusted AI-assisted analytics and natural language-driven insights. Microsoft Fabric services including OneLake, Data Factory, Data Engineering, Dataflow Gen2, Warehouse, Semantic Models, Power BI, and Purview work together seamlessly under a centralized metadata-driven framework with governance, lineage, monitoring, and reusable pipelines. This architecture enables organizations to deliver scalable analytics, accelerate decision-making, and create a modern AI-enabled customer intelligence platform.

Medallion Architecute with Fabric Meta Data Driven Framework
https://youtu.be/6ffjPLPItLc

Copilot assisted transformation in dataflow Gen2 in Fabric
https://youtu.be/1MQP9IT3DI0

PowerPoint Presentation: https://github.com/Geekkailash/fabrick-customer360/blob/main/FMD%20Framework.pptx

https://github.com/Geekkailash/fabrick-customer360/blob/main/FMD%20Framework.pptx

Code Link in Github: https://github.com/Geekkailash/fabrick-customer360

https://github.com/Geekkailash/fabrick-customer360

Author: Kailash Singh Bhandari

Github Profile: https://github.com/Geekkailash

Metadata Driven Framework

NandanHegde — Fri, 08 May 2026 16:39:07 GMT

Git Hub Code Repo : https://github.com/NandanHegde15/MSFTFabric-Projects/blob/main/Metadata%20Driven%20Framework/README.md

Demo : https://www.youtube.com/watch?v=Z7mjHmX8-40

Metadata Driven Framework is a Microsoft Fabric-based end-to-end healthcare analytics platform . This project demonstrates modern data engineering practices including: Multi-layer data architecture (Raw → Silver → Gold layers) FHIR healthcare data modeling with support for patients, claims, coverage, and organizations Automated data pipelines for ingestion, transformation, and aggregation Power BI semantic models for enterprise analytics and reporting Row-level security for data governance and privacy AI-powered insights using Copilot & Data Agent capabilities

Authors : Hemanth Kotha, Nandan Hegde

https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DZ7mjHmX8-40

Medallion Architecture in Microsoft Fabric: Building an AI-Ready Data Pipeline

trivedisunita — Sun, 03 May 2026 13:06:58 GMT

Video Walkthrough Link------------>
(https://raw.githubusercontent.com/trivedisunita/Fabric-Data-Factory-Contest/main/Fabric%20Short%20Video2.mp4)

This project presents an end-to-end data pipeline built for the Data Factory Challenge using Microsoft Fabric. The objective was to transform raw job salary data into a structured, analytics-ready format for insights.

------------- Architecture Overview--------

Step1-

##Bronze Layer:--

Raw data is ingested into a Lakehouse from a SharePoint source without transformations, ensuring data is preserved in its original format.

Step 2--

##Silver Layer:--

Data is processed using Data Factory pipelines and stored in a Data Warehouse schema. Transformations include data cleaning, handling null values, and standardizing fields. SQL scripts and Copilot were used to assist in transformations.

Step--3

##Metadata Pipeline:---

Metadata-driven pipeline is implemented to manage and monitor data flow across layers. It enables dynamic execution, parameter handling, and improves pipeline scalability and maintainability within Microsoft Fabric.

Step--4

##Gold Layer:--

Data is modeled using dbt to create fact and dimension tables. A star schema is built for efficient querying, with reusable transformations such as converting coded fields into readable formats.

Step--5

##Semantic Model & Reporting:--

An AI-ready semantic model is created and used in Power BI. The model is optimized by selecting relevant columns and defining structure for better performance and accurate insights.

Interactive reports are built on top of this model, enabling natural language queries using Copilot and delivering reliable business insights.

https%3A%2F%2Fyoutu.be%2FskgWJyBPMcs

Microsoft Fabric Copy Job Tutorial for Beginners

suparnababu8 — Fri, 27 Feb 2026 11:22:06 GMT

In this beginner-friendly tutorial, I explain how to create and configure a Copy Job in Microsoft Fabric step by step — from source connection to destination loading.

This session is perfect for:
✔️ Beginners in Microsoft Fabric
✔️ Aspiring Data Engineers
✔️ ETL / Data Integration learners
✔️ Power BI & Azure professionals moving to Fabric

By the end of this video, you will confidently create your own Copy Job in Fabric.

#MicrosoftFabric #FabricCopyJob #MicrosoftFabricTutorial #DataEngineering #FabricForBeginners #DataIngestion #CopyActivity #AzureData #FabricLakehouse #ETL #NDAC #NextGenData #DataPipeline #CloudData #LearnFabric

Follow our WhatsApp channel : https://whatsapp.com/channel/0029VbB2EH33AzNVJEyA6L0r
Visit us at : https://ndac.community/
Join us at Meetup: https://www.meetup.com/nextgen-data-aspirants-community/

watch?v=mmZSvks32bg

Fabric Monday 100: Dataflows Destinations and Schema

DennesTorres — Sun, 11 Jan 2026 21:20:49 GMT

If you’re working with Microsoft Fabric Dataflows Gen2, Default Data Destinations can be a huge productivity boost… if you know how to use them properly.

In this video, I show:
⏱️ How the Default Data Destination saves serious time by removing the need to configure a destination query by query
🧠 Why this is especially powerful when you’re building dataflows with many entities
🧩 How schema selection actually works (yes, it is supported — but only if Fabric is set up the right way)
👀 The small, easy-to-miss details that decide whether schemas are available or silently ignored

If you’ve ever:
🚫 clicked through destinations for every single query
😤 wondered why schemas sometimes don’t appear
⚙️ wanted faster, cleaner Dataflows without hidden pitfalls

…this video will save you time and frustration.

watch?v=5KB8sjqXnDs

Introduction To Microsft Fabric Lakehouse

joeydj — Mon, 08 Dec 2025 10:02:54 GMT

This is a level 100 tutorials. It provides overview of Microsoft Fabric LAKEHOUSE

1. Concept, definition and principle of Lakehouse

2. LAkehouse Key Feature

3. Lakehouse Core functioanlity

4. How to create Fabric Lakehouse

5. Demo to navigate into the Lakehouse

watch?v=57gVydAQhUI

Fabric Monday 71: Variable Libraries, now and the future

DennesTorres — Sun, 20 Apr 2025 22:37:44 GMT

Discover what are variable libraries in Microsoft Fabric. What are their purposes and benefits and how to work with them. It's also important to understand what could we expect for the future of this feature

watch?v=W-G4JDcRRrI

Fabric Monday 67: Fabric and Azure Data Factory

DennesTorres — Tue, 18 Mar 2025 22:27:36 GMT

Discover the two existing methods to integrate Fabric and Azure Data Factory and what's the best scenario to use each one of them

watch?v=dMYaGqNudaY&t=3s

Microsoft Fabric Community Conference 2025

pleblanc — Tue, 17 Dec 2024 23:42:19 GMT

Hear from Arun Ulag, CVP of Azure Data on his favorite part of FabCon Europe, what's new at the second FabCon in Las Vegas, and how we're making sure it's bigger and better than ever before across a variety of topics! Join us March 31-April 2 in Las Vegas to kick off our second year of the Microsoft Fabric Community Conference. Register Now at https://fabricconf.com/.

More questions? Try asking the Power BI forums @ https://community.fabric.microsoft.com/t5/Power-BI-forums/ct-p/powerbi

#Microsoft #PowerBI #FabCon

watch?v=IqvDWEhozEI

Fabric Update November 2024

GuyInACube — Tue, 17 Dec 2024 23:43:01 GMT

Welcome to the November 2024 update. Be sure to check out our Monthly Update blog for a deep dive into everything new and all the updates for the month of November.

Blog: https://blog.fabric.microsoft.com/en-us/blog/fabric-november-2024-feature-summary?ft=All

Sections

POWER BI

Copilot and AI - 1:37

Reporting - 6:41

Modeling - 15:30

Developers - 28:25

Visualizations - 29:10

Other - 29:29

FABRIC

Platform - 30:01

OneLake - 31:43

Databases - 32:14

Data Engineering - 33:15

Data Science - 42:58

Data Warehouse - 44:55

Real-Time Intelligence - 46:05

Data Factory - 1:22:09

DEMOS

Copilot and AI - 2:40

Reporting - 8:36, 13:37

Modeling - 18:24, 21:25, 25:25

Data Engineering - 36:15, 37:48, 39:22

Data Science - 43:22

Real-Time Intelligence - 56:56, 1:02:36, 1:06:46, 1:11:15, 1:14:07, 1:18:00

Data Factory - 1:25:41, 1:26:36

More questions? Try asking the Power BI forums @https://community.fabric.microsoft.com/t5/Power-BI-forums/ct-p/powerbi

#Microsoft #MicrosoftFabric

watch?v=eyjVj-k8m1M?si=0FlRLuF-sm7GIZr0

Fabric Update October 2024

pleblanc — Wed, 20 Nov 2024 23:04:54 GMT

Welcome to the October 2024 update. Be sure to check out our Monthly Update blog for a deep dive into everything new and all the updates for the month of October!

Blog: https://blog.fabric.microsoft.com/en-us/blog/fabric-october-2024-monthly-update?ft=All

Sections

POWER BI

Copilot and AI - 0:56

Reporting - 2:30

Modeling - 9:46

Mobile - 15:40

Visualizations - 16:38

FABRIC

Data Warehouse - 17:15

Data Engineering - 19:31

Real-Time Intelligence 23:11

Demos

Reporting - 4:15, 6:18

Modeling - 11:45

Data Engineering - 22:11

More questions? Try asking the Power BI forums @ https://community.fabric.microsoft.com/t5/Power-BI-forums/ct-p/powerbi

watch?v=kn23stylRQM?si=IPpzwyeUWlBfN4y8

Fabric Monday 55: Pipelines High Concurrency to Save Your Time and Money

DennesTorres — Wed, 09 Oct 2024 00:38:05 GMT

Discover the High Concurrency for pipelines executing notebooks. This feature can save you a lot of execution time and money

watch?v=oc2b1Xcu1ts

Fabric Update September 2024

Anonymous — Thu, 31 Oct 2024 18:48:43 GMT

Welcome to the September 2024 update. Be sure to check out our Monthly Update blog for a deep dive into everything new and all the updates for the month of September!

Blog: https://blog.fabric.microsoft.com/en-us/blog/fabric-september-2024-monthly-update?ft=All

Aruns blog: https://www.microsoft.com/en-us/microsoft-fabric/blog/2024/09/25/european-fabric-community-conference-2024-building-an-ai-powered-data-platform/

Power BI YouTube Channel: https://www.youtube.com/channel/UCy--PYvwBwAeuYaR8JLmrfg

Sections

Power BI Highlights - 1:22

Core - 2:48

OneLake - 10:12

Data Warehouse - 12:18

Data Engineering - 17:32

Data Science - 28:37

Real-Time Intelligence - 34:42

Data Factory - 1:00:36

Data pipeline - 1:05:13

Mirroring - 1:18:11

Demos

Core- 5:45

Data Warehouse - 15:05

Data Engineering - 25:00, 25:49, 27:31

Data Science - 30:02, 33:12

Real-Time Intelligence - 41:20, 45:17, 47:49, 52:12, 56:18, 59:15

Data Factory - 1:02:03

Data pipeline - 1:07:36, 1:10:25, 1:14:19

Mirroring - 1:19:29, 1:20:31

More questions? Try asking the Power BI forums @ https://community.fabric.microsoft.com/t5/Power-BI-forums/ct-p/powerbi

watch?v=sWQMlt9ikE0?si=G9SD20lCmOW5Pe2T

Fabric Update August 2024

Anonymous — Wed, 25 Sep 2024 21:06:26 GMT

Welcome to the August 2024 update. Be sure to check out our Monthly Update blog for a deep dive into everything new and all the updates for the month of August!

Blog: https://blog.fabric.microsoft.com/en-us/blog/microsoft-fabric-august-2024-update?ft=All

Sections

Copilot and AI – 00:52

Reporting – 1:24

Modeling – 16:07

Embedded Analytics – 19:22

Visualizations – 21:24

Core – 21:41

OneLake – 22:02

Data Warehouse – 23:36

Data Engineering – 35:16

Data Science – 37:17

Real-Time Intelligence – 48:43

Data Factory – 49:17

Demos

Reporting – 4:34, 10:21, 13:18, 15:25

Modeling – 16:28

Embedded Analytics – 19:47

OneLake – 22:50

Data Warehouse – 27:14, 30:5, 31:02, 32:24

Data Science – 41:07, 45:05

More questions? Try asking the Power BI forums @https://community.fabric.microsoft.com/t5/Power-BI-forums/ct-p/powerbi

watch?v=36U44YK68G0

Fabric Update July 2024

Anonymous — Wed, 28 Aug 2024 21:58:49 GMT

Welcome to the July 2024 update. Be sure to check out our Monthly Update blog for a deep dive into everything new and all the updates for the month of July!

Blog: https://blog.fabric.microsoft.com/en-us/blog/microsoft-fabric-july-2024-update?ft=All

Sections

Reporting - 1:21

Modeling - 5:27

Service - 13:07

Developers - 13:33

Visualizations - 14:00

Paginated Reports - 14:33

Report Server - 20:02

Core - 20:24

Data Warehouse - 29:44

Data Engineering - 34:30

Real-Time Intelligence - 39:28

Data Factory (Dataflow Gen2) - 40:39

Data Pipeline - 42:25

Demos

Reporting - 1:34

Modeling - 6:07, 9:30

Service - 13:10

Paginated Reports - 15:07

Report Server - 20:08

Core - 23:21, 26:42, 28:13

Data Warehouse - 31:46

Data Engineering - 36:06

More questions? Try asking the Power BI forums @ https://community.fabric.microsoft.com/t5/Power-BI-forums/ct-p/powerbi

watch?v=WDpWMsOuqCM?si=QsJE_jDGVp9bZpWd

Fabric Update June 2024

Anonymous — Fri, 26 Jul 2024 01:10:18 GMT

Welcome to the June 2024 update. Be sure to check out our Monthly Update blog for a deep dive into everything new and all the updates for the month of June!

Blog: https://blog.fabric.microsoft.com/en-us/blog/microsoft-fabric-june-2024-update?ft=Monthly-update:category

Sections

Power BI

Reporting - 0:28

Modeling - 1:49

Mobile - 2:55

Developers - 3:37

Visualizations - 9:50

Paginated Reports - 10:15

Fabric

Service - 10:33

OneLake - 10:55

Data Warehouse - 11:19

Data Engineering - 13:43

Real-Time Intelligence - 16:10

Data Factory Dataflow Gen2 - 17:28

Data Pipeline - 17:51

Demos

Developers (Power BI enhanced report format) - 4:14

Data Warehouse (Mirroring Azure SQL Database) - 12:40

*Note* - Lakehouse Schemas - A New Way to Organize Your Tables PREVIEW will not ship until the July update.

More questions? Try asking in the Power BI forums @ https://community.fabric.microsoft.com/t5/Power-BI-forums/ct-p/powerbi

#microsoft #microsoftfabric

watch?v=sdnPeFR6jmQ?si=E3EIDQn8iai4jrs6

Fabric Monday 41: Mirroring Azure SQL to Fabric

DennesTorres — Tue, 18 Jun 2024 02:25:44 GMT

Discover how the mirroring feature works, when you should use it and when you shouldn't

watch?v=ic9GtH5uPak&list=PLNbt9tnNIlQ5TB-itSbSdYd55-2F1iuMK&index=1

The Unified Data Platform for the Era Of AI | Microsoft Fabric Community Conference Day 1 Keynote

Natalie_iTalent — Fri, 31 May 2024 20:30:29 GMT

Microsoft Fabric is a unified analytics platform that brings together all the data and analytics tools that you need. Fabric unifies your data estate covering everything from data warehouse to data movement to data science, real-time analytics, and business intelligence. Get ready for a keynote session jam-packed with announcements, sneak peeks, and new capabilities across Microsoft Fabric.

Learn more: https://www.microsoft.com/en-us/micro...

#Microsoft #MicrosoftFabric

watch?v=BYob0cGW0Nk