I Let Fabric's New Data Engineering Agent Run My ETL: Here's What Happened
Last week I handed Microsoft Fabric's new Data Engineering Agent a messy ETL job that took me 11 hours to build by hand last quarter. It had a working, validated pipeline in front of me in under an hour. I still changed 3 of its 9 steps before I'd let it near anything important. That is the short version. The long version is more interesting, because the places where it went wrong tell you exactly how to use it well. Some context first. At FabCon Europe 2026 in Barcelona (September 28 to October 1), Microsoft announced the Data Engineering Agent in preview. It can plan and run migrations, ETL and performance work under guardrails that engineers define. It builds on technology from Osmos, the agentic data engineering startup Microsoft acquired in January 2026, and was demoed on stage as "Project Osmos". Copilot has been helping us write code for a while now. This is a different idea. You describe the outcome, and the agent does long-running, multi-step work on its own while you supervise. I wanted to know whether that holds up on a real, slightly ugly dataset rather than a keynote demo. So I tried it. What the Data Engineering Agent actually is The agent is an autonomous worker inside your Fabric workspace, not a chat assistant that suggests code. You give it a goal, and it does the multi-step work itself. Its roots are in Osmos, a Seattle startup whose "AI Data Engineer" already generated production-grade PySpark notebooks inside Fabric. Microsoft bought the company in January 2026 and folded the team into Fabric engineering. Based on Microsoft's FabCon session on Project Osmos, a typical run looks like this: Read the request. You describe the outcome, even loosely ("land the orders CSVs in a clean silver table"). Inspect the workspace. It looks at your existing code, Lakehouses and data layout before writing anything. Plan. It breaks the job into steps and can hand parts to sub-agents for long-running work. Build. It creates and updates notebooks and other Fabric items. Validate. It checks its own output so you can trust what was built. Hand back to you. A human reviews and approves. Microsoft is explicit that supervision stays part of the process. The headline use cases are migrations, ETL and performance tuning. I chose ETL, because that is where most of us spend our week. The test: a real pipeline, not a demo I picked a job I had already built by hand, so I could judge the agent's work line by line against my own. The data is sales orders from a mid-sized retail client . Every day their order system drops a CSV export into a Lakehouse folder. The business wants a clean orders table for analysts and a daily sales summary for a Power BI report. Item My setup Source data 14 daily CSV exports, 18.4 million rows, about 2.1 GB, in Files/raw/orders/ Target silver_orders (cleaned Delta table) and gold_daily_sales (aggregate by day, store and category) Known traps Duplicate order IDs from system retries, two date formats, a currency column with nulls, cancelled orders flagged as negative quantities Capacity F64 trial capacity, sandbox workspace Hand-built baseline About 11 hours across two days, including testing The exact prompt I gave it: Ingest the daily order CSVs in Files/raw/orders into a cleaned silver Delta table called silver_orders. Remove duplicates, standardise dates and currency, and exclude cancelled orders. Then build gold_daily_sales with total revenue, order count and average basket size by day, store and category. The pipeline must run incrementally each day. I deliberately kept the prompt a little vague. I didn't say which date formats exist, how cancellations are encoded, or which column identifies a duplicate. In real life, requirements rarely arrive as a perfect spec. I wanted to see whether the agent would investigate, ask, or just guess. What happened, step by step 1. It read the workspace before writing anything The first thing I noticed: it didn't start coding. For the first four minutes it profiled the data. It sampled each CSV, inferred the schema and looked at my existing notebooks in the workspace. It came back with a short summary of what it found. Two findings impressed me: It spotted both date formats (2026-09-14 and 14/09/2026) and flagged that files from 9 September onwards switched format. It noticed my existing notebooks used a nb_<layer>_<entity> naming pattern and said it would follow it. It missed one thing, which matters later: it reported quantity as "integer, some negative values" without asking what negative quantities meant. 2. The plan it proposed Next it produced a 9-step plan and stopped for my approval: Create a staging notebook to read all CSVs with an explicit schema Normalise both date formats to a single order_date column Fill null currency values from the store's default currency Deduplicate on order_id, keeping the latest record Remove cancelled orders Write silver_orders as a Delta table, partitioned by order_date Build gold_daily_sales with the three requested measures Add incremental logic based on the source file date Run validation checks and produce a summary report This is a sensible plan. It is roughly what I wrote on a whiteboard last quarter. But reading it closely, I changed three things before approving: Step 4: it planned to overwrite silver_orders on each run. I changed it to a MERGE on order_id, because late-arriving corrections are common in this data. Step 5: it planned to remove cancellations using a status column. That column exists, but this client never fills it. Cancellations show up as negative quantities. I told it so. Step 8: it wanted to detect new data by file modified time. I asked it to use the date in the file name instead, which is more reliable when files are re-uploaded. It accepted all three changes and rewrote the plan in a few seconds. 3. The build Once approved, it worked for 38 minutes without needing me. It created three notebooks (nb_stg_orders, nb_silver_orders, nb_gold_daily_sales) and a small pipeline to run them in order. The code was clean and readable. Here is its date handling, which is close to what I would write myself: from pyspark.sql import functions as F df = df.withColumn( "order_date", F.coalesce( F.to_date("order_date_raw", "yyyy-MM-dd"), F.to_date("order_date_raw", "dd/MM/yyyy") ) ) And the merge into silver, after my change in the plan: from delta.tables import DeltaTable silver = DeltaTable.forName(spark, "silver_orders") (silver.alias("t") .merge(updates.alias("s"), "t.order_id = s.order_id") .whenMatchedUpdateAll() .whenNotMatchedInsertAll() .execute()) It asked me one question mid-way: whether average basket size should be revenue per order or items per order. That was exactly the right question, and one I had to ask the client myself last time. 4. Self-validation When the build finished, it ran its own checks and produced a short report: Row counts reconciled from raw to staging to silver, with each dropped row explained (duplicates, cancellations) No nulls left in order_date, currency or store_id No duplicate order_id values in silver_orders Daily totals in gold_daily_sales matched a direct aggregate of silver_orders One check failed on the first pass. 41,200 rows had a null order_date after parsing. It traced the cause itself: a third date format (14-Sep-2026) appeared in two of the older files, which its initial sampling had missed. It added a third pattern to the coalesce, re-ran, and the check passed. That moment sold me on the validation step. A human could easily miss 0.2% of rows quietly disappearing. 5. My review against my own version I compared its output with the pipeline I built by hand last quarter, running both on the same 14 files. Measure Agent Me, by hand Time to working pipeline 52 minutes (incl. my review) About 11 hours Plan steps I changed 3 of 9 Not applicable Data issues found 4 of 4, after one self-correction 4 of 4 Full run time on 18.4M rows 6 min 40 s 5 min 55 s Silver row count 17,962,118 17,962,118 The row counts matched exactly. My hand-built version ran about 45 seconds faster, mostly because I had tuned partitioning. More on that below. Would I merge its code as-is? After my three plan changes, yes, with one small performance fix. Guardrails: the part you shouldn't skip An agent that can create and change items in your workspace needs limits before it needs a prompt. Microsoft designed this agent around guardrails that engineers define, and setting them up was the best 15 minutes of the whole test. What I set up before the run: A sandbox workspace. It's a preview feature, so it ran on a copy of the data, nowhere near production. Scoped data. It could read Files/raw/orders/ and write only to new tables. My existing tables were off limits. Approval points. It had to stop for my approval after the plan, and again before the first write to any Delta table. Git in the loop. The workspace was connected to a Git repo, so every notebook it created or changed landed as a diff I could review line by line. In practice, it never tried to step outside these limits. The approval after the plan turned out to be the most valuable one: that is where I caught all three of my changes. If you set only one guardrail, make it that one. Where it struggled No agent review is credible without the failures, so here are the three that mattered. It assumed instead of asking about business rules. It saw negative quantities and planned to use an empty status column for cancellations. Had I approved the plan without reading it, about 310,000 cancelled orders would have flowed into the gold table and inflated revenue. It asked a good question about basket size, but not this one. Its sampling missed a rare date format. The third format appeared in only two files. Its validation caught the problem, but only after the first run. On a much larger dataset, that first run costs real capacity. Correct, but not tuned. It partitioned silver_orders by order_date, which produced many small files for this data volume. Switching to monthly partitions and running OPTIMIZE closed most of the 45-second gap with my version. When I asked it to "make the silver write faster", it suggested the same fix, but it didn't do it unprompted. Three caveats worth knowing before you try it: It's a preview. Features, behaviour and pricing can change before general availability. Don't build production processes on it yet. Watch your capacity. Long-running agent work consumes capacity units on top of the Spark jobs themselves. Check the Capacity Metrics app after your first run so you know what a typical job costs. It doesn't know your business. It can read your schema, not your meeting notes. Every rule I wrote into the prompt was handled correctly. Every rule I left out was a guess. The verdict The Data Engineering Agent is like a fast, tireless junior engineer: excellent at the mechanics, careful about checking its own work, and still in need of a senior reviewing the plan. What I'd tell my team after this test: Spend your time on the prompt and the plan, not the code. Every problem I found was visible in the plan before a single line ran. Write business rules down explicitly. Cancellations, duplicates, currency defaults. If it's tribal knowledge, the agent will guess. Trust the validation step, but read its report. It caught a data loss bug I might have missed myself. Tune afterwards. Treat its first version as correct, then ask it for performance improvements. Try it now if you have a sandbox workspace, a backlog of repetitive ingestion or migration jobs, and someone who can review PySpark. Wait for GA if you need it in production, your pipelines are full of undocumented business logic, or your capacity budget is tight. The bigger shift is in our role. Going from 11 hours to under an hour didn't remove me from the work. It moved my effort from typing code to writing clear requirements, setting guardrails and reviewing output. That is a skill worth building now, preview or not. Have you tried the Data Engineering Agent yet? Tell me in the comments what you threw at it, and where it surprised you. Sources FabCon Europe 2026 announcements (Kanerika) From Prompt to Finished Project: Introducing Project Osmos (FabCon Europe 2026 session) Microsoft Acquires Osmos (Redmond Magazine) Fabric January 2026 Feature Summary (Microsoft Fabric Blog)FabCon & SQLCon 2026: The Ultimate Roundup of Announcements
While I wasn’t able to catch the action live in Barcelona last week for FabCon and SQLCon 2026, keeping tabs on the announcements has been a wild ride. Microsoft Fabric has officially cleared the 40,000-customer milestone and the event heavily emphasized Frontier Transformation – moving past standard AI productivity hacks and embedding autonomous agents directly into corporate workflows. Yet, among all the flashy Copilot integrations and agentic apps, one specific announcement completely stole the show for me: the introduction of the F0 capacity tier with on-demand billing. For anyone who has ever wrestled with budget approvals just to test out a new data platform, this is a game-changer. Let’s dive into why F0 is my favorite takeaway, alongside a full roundup of everything else that dropped in Barcelona. 1. My #1 Favorite: The F0 Capacity SKU & Pay-As-You-Go Billing Historically, dipping your toes into Microsoft Fabric meant committing upfront to a reserved compute tier (like an F2 or higher). If you just wanted to spin up a quick test environment, build a proof of concept, or play with OneLake shortcuts, you still had to navigate a monthly capacity bill. What’s new: Microsoft introduced F0, a zero-provisioned capacity SKU featuring true on-demand billing. Why it’s a massive deal: It acts as the “serverless” equivalent for Fabric. You can now evaluate features, run lightweight development tasks and test out workloads without locking into an upfront monthly reservation. Compute costs finally align strictly to what actually runs, shattering the barrier to entry for smaller teams and developer sandboxes. 2. Grounding Microsoft Copilot with Fabric IQ AI tools often hallucinate because they lack proper context. Microsoft is tackling this by wiring business context directly into Microsoft Copilot. What’s new: Fabric IQ is generally available as a shared intelligence bridge, connecting OneLake data, Power BI semantic models and operational metrics directly into Copilot Chat and Cowork – with zero additional AI token costs. Why it matters: Instead of guessing, your organization’s Copilot can answer business questions using the exact semantic data definitions your teams already trust and govern. Read more: Check out Arun Ulag’s keynote overview on the official Microsoft Azure Blog. 3. Power BI’s Evolution: Agentic App Creation Power BI is stepping past standard dashboards into operational application building. What’s new: You can use natural language inside Power BI Desktop to build, preview and publish purpose-built data applications straight from a trusted semantic model. These apps can handle inputs, write-back data and power workflows. Why it matters: Power BI Pro and Premium Per User (PPU) users will get access to Fabric Apps and Fabric Database capabilities (up to 1 GB per app) at no additional cost. Read more: Dive into Mohammad Ali’s deep dive on Power BI’s next chapter: The evolution of business intelligence. 4. Fast-Tracking Apps to Production with Fabric Apps & Rayfin Prototyping an app with AI is easy, but getting it to a secure production environment used to mean heavy lifting. Fabric Apps solves this. What’s new: Using the open-source Rayfin SDK and CLI, developers can build app backends and deploy them directly to Fabric with built-in TypeScript functions, a Secret Store, PostgreSQL support and private-by-default security. Why it matters: Apps plug straight into existing Lakehouses and Warehouses without data duplication. Read more: Catch Sachin Patney’s post, From prompt to production: What’s new in Fabric Apps. 5. Expanding the OneLake Ecosystem & IQ Sharing What’s new: Microsoft rolled out IQ sharing (in preview), allowing organizations to securely share governed data, markdown agent instructions and ontologies across different teams, partners and external ecosystems. Read more: Read Dipti Borkar’s breakdown on What’s new in Microsoft OneLake and its rapidly growing ecosystem. 6. Autonomous Data Engineering Agents What’s new: Driven by technology from the Osmos acquisition, Microsoft introduced a preview of the Fabric data engineering agent. Rather than just auto-completing code snippets, this agent handles complex, long-running engineering tasks like migrations, ETL pipeline creation and modernizations based on human guardrails. Read more: Check out Bogdan Crivat’s blog on Bringing governed analytics into the flow of work: Fabric Analytics at FabCon Europe 2026. 7. SQLCon: Scale, Intelligence, and Serverless Pauses Running parallel to FabCon, SQLCon delivered major updates for database administrators balancing traditional relational databases with modern demands. What’s new: A new Database Hub (Public Preview) serves as a single pane of glass to monitor health and security across SQL Server, Azure SQL and PostgreSQL. Plus, DiskANN vector indexes are now generally available to supercharge native semantic search using T-SQL. Read more: Read Shireesh Thota’s update on SQLCon Barcelona 2026: Advancing SQL with greater control, scale and intelligence. Final Thoughts: Looking Ahead Reflecting on everything announced in Barcelona, it’s clear that Microsoft is moving faster than ever to bridge the gap between heavy-duty data engineering and practical, AI-driven applications. While agentic workflows and Copilot enhancements grab the headlines, structural updates like the F0 tier are what truly make this ecosystem accessible to everyone from lone developers to enterprise architects. Honestly, I am still trying to digest all these announcements made and also can’t wait to try them soon! We are officially stepping into an era where building, scaling and operationalizing data isn’t just about managing tables – it’s about powering the intelligence layer that runs the business. Are you as thrilled about the F0 capacity tier as I am? What feature from FabCon are you testing first? Let me know in the comments below!Data Days | Data Days Your Way
Return to Data Days Homepage Data Days Your Way Find all sessions on demand!. The Power BI Dataviz World Champs is happening now | start your journey to the finals in Barcelona. Start with what fits you: Fabric | Power BI | SQL | AI | Beginner / New You are looking for: Certification Prep | Data Engineering Deep Dives | Data Visualization | Community Browse by Language: Spanish | Português | French | Japanese / 日本語 | Hindi Fabric User DP 700- Microsoft Fabric Training | Episode 1: Fabric Overview, Domains, Workspaces & OneLake Date: June 14, 6:30 PM Host: Amit Chandak Managing secure access, trusted discovery & data sharing with OneLake (3 sessions) Date: June 16, 9:00 AM Host: Josh Ndemenge Tenant management with Sempy Date: June 16, 11:00 AM Host: Taylor Amy, Teemu Multanen Get Certified: (DP-700) Fabric Data Engineer Essentials (APAC) Date: June 16, 3pm Host: Mike Fortman, Martin Catherall Get Certified: (DP-700) Fabric Data Engineer Essentials (US/EMEA) Date: June 16, 3pm Host: Aleksi Partanen, Phillip Burton Get Certified: (DP-600) Fabric Analytics Engineer Essentials (APAC) Date: June 17, 8am Host: Heidi Hasting, Martin Catherall Get Certified: (DP-600) Fabric Analytics Engineer Essentials (US/EMEA) Date: June 17, 3pm Host: Ásgeir Gunnarsson, Rajendra Ongole DP 700- Microsoft Fabric Training | Episode 4: Dataflow Gen2 End-to-End Date: June 17, 6:30 PM Host: Amit Chandak Orchestrating Fabric Spark and Best Practices for Production-Ready Workload Date: June 18, 8am Host: Santhosh Kumar Ravindran; Ashit Gosalia Security and Governance in Fabric Date: June 20, 09:30 AM Host: Amit Kumar Mahato (Cloud Guru Amit) Fabric Data Pipelines Full Course For Beginners (Data Days Edition 2026) Date: June 21, 05:30 AM Host: Ansh Lamba Data Ingestion and Discovery in Fabric Date: June 21, 09:30 AM Host: Amit Kumar Mahato (Cloud Guru Amit) DP 700- Microsoft Fabric Training | Episode 6: Data Pipelines, Scheduling & OneLake Shortcuts Date: June 21, 6:30 PM Host: Amit Chandak Prepare for the Microsoft Data Days Date: June 22, 5 am Host: Aman Jindal DP 700- Microsoft Fabric Training | Episode 7: Real-Time Analytics, Eventstream, Eventhouse & KQL Date: June 22, 6:30 PM Host: Amit Chandak Location Intelligence with Maps and GeoAnalytics in Microsoft Fabric Date: June 23, 07:30 AM Host: Philippa Burgess Microsoft Cloud & AI Frontier Week: Unify Your Data with OneLake for Analytics, AI and Agents Date: June 24, 01:00 AM Host: Sevgi Guzzella Microsoft Cloud & AI Frontier Week: Turn Data into Intelligent Action with Microsoft Fabric Date: June 24, 02:00 AM Host: Simon Lidberg The Future of AI in Microsoft Fabric: Data Agents and Beyond Date: June 24, 08:00 AM Host: Brian Bønk, Philippa Burgess DP 700- Microsoft Fabric Training | Episode 9: Mirroring, Databases, Composite Models & Data Agents Date: June 24, 6:30 PM Host: Amit Chandak Global Fabric Day 2026 Date: June 27 Host: Kim Manis Global Fabric Day 2026: Security, Location & Intelligence in Microsoft Fabric Date: June 27, 09:30 AM Host: Philippa Burgess Transforming and Modeling Data in Fabric Date: June 27, 09:30 AM Host: Amit Kumar Mahato (Cloud Guru Amit) Modeling Real LMS Data in Power BI: Star Schema from a Messy MySQL Source Date: June 28, 06:30 AM Host: Parul Rani Sagar Configuring Workspace Settings in Fabric Date: June 28, 09:30 AM Host: Amit Kumar Mahato (Cloud Guru Amit) Learn KQL in 10 minutes Date: June 29 Host: Phillip Burton Get to Know Esri: From Living Atlas to Spatial Analysis for Fabric Users Date: June 30, 07:30 AM Host: Philippa Burgess Orchestrating Pipelines, and Notebooks in Fabric Date: July 4, 09:30 AM Host: Amit Kumar Mahato (Cloud Guru Amit) Designing Data Load Strategies in Fabric Date: July 5, 09:30 AM Host: Amit Kumar Mahato (Cloud Guru Amit) Data and AI Security and Governance in Microsoft Fabric Date: July 7, 07:30 AM Host: Philippa Burgess Inside Fabric Runtime 2.0: Spark 4 and Delta 4 in Action Date: July 15, 4:00 PM Host: Arshad Ali and Miles Cole Fabric IQ for Data Professionals Date: July 21, 07:30 AM Host: Philippa Burgess Building Scalable Bronze Layer in Microsoft Fabric Date: July 23, 09:00 AM Host: Aleksi Partanen, Teemu Multanen Win in the Seams 🧵 Stitching Together Data and Security with Microsoft Fabric, KQL, and Data Logs Date: July 25, 09:30 AM Host: Philippa Burgess KQL for Data and Security Professionals Date: July 28, 07:30 AM Host: Philippa Burgess Fabric Analytics Engineer Certification Training (DP-600) Discover resources to prepare for this exam. Fabric Data Engineer Certification Training (DP-700) Discover resources to prepare for this exam. Get started with Microsoft Fabric Self-paced training DP 700- Microsoft Fabric Training | Episode 1: Fabric Overview, Domains, Workspaces & OneLake Date: June 14, 6:30 PM Host: Amit Chandak Managing secure access, trusted discovery & data sharing with OneLake (3 sessions) Date: June 16, 9:00 AM Host: Josh Ndemenge DP 700- Microsoft Fabric Training | Episode 4: Dataflow Gen2 End-to-End Date: June 17, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 6: Data Pipelines, Scheduling & OneLake Shortcuts Date: June 21, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 7: Real-Time Analytics, Eventstream, Eventhouse & KQL Date: June 22, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 9: Mirroring, Databases, Composite Models & Data Agents Date: June 24, 6:30 PM Host: Amit Chandak Inside Fabric Runtime 2.0: Spark 4 and Delta 4 in Action Date: July 15, 4:00 PM Host: Arshad Ali and Miles Cole Back to top Power BI Using Slicers and What-If Parameters in Power BI Date: June 15, 6:00 AM Host: Ilgar Zarbaliyev Power BI Dataviz World Championships: Start Your Journey to Barcelona Date: June 16, 8am Host: Valerie Junk, Lakshmi Ponnurasan Power BI Dataviz World Championships: Comece sua jornada para Barcelona Date: June 16, 2pm Host: Samyr Moises, Dirceu Moraes Resende Power BI Dataviz World Championships: Comienza tu camino a Barcelona Date: June 16, 4pm Host: Walter Calcagno, Lucrecia Krause Dynamic Currency Conversion in Power BI Date: June 22, 6:00 AM Host: Ilgar Zarbaliyev Get Certified: (PL-300) Power BI Data Analyst (US/EMEA) Date: June 22, 8am Host: Ilgar Zarbaliyev, Doher Drizzle Pablo Get Certified: (PL-300) Power BI Data Analyst (APAC) Date: June 23, 3pm Host: Anupama Natarajan, Chris Hyde DP 700- Microsoft Fabric Training | Episode 8: Direct Lake Semantic Models & Power BI Performance Date: June 23, 6:30 PM Host: Amit Chandak Modeling Real LMS Data in Power BI: Star Schema from a Messy MySQL Source Date: June 28, 06:30 AM Host: Parul Rani Sagar Implementing Row-Level Security (RLS) Date: June 29, 6:00 AM Host: Ilgar Zarbaliyev Building Interactive Dashboards and Data Alerts Date: July 6, 6:00 AM Host: Ilgar Zarbaliyev Exploring Data with AI and Natural Language Features Date: July 13, 6:00 AM Host: Ilgar Zarbaliyev Microsoft Certified: Power BI Data Analyst Associate Date: July 18, 5:30 AM Host: Inturi Suparna Babu, Ajay Babu Inturi, Upputuri Gopikrishna Power BI Essentials — Data Days with Data Analytic Group Date: July 18, 09:00 PM Host: Rajendra Ongole,Lanka, Shashi Performing Analytics in Power BI using DAX Date: July 20, 6:00 AM Host: Ilgar Zarbaliyev Get started with Microsoft data analytics Self-paced training Power BI Data Analyst Certification Training (PL-300) Discover resources to prepare for this exam. Power BI Dataviz World Championships Do you have what it takes? DP 700- Microsoft Fabric Training | Episode 8: Direct Lake Semantic Models & Power BI Performance Date: June 23, 6:30 PM Host: Amit Chandak Back to top SQL DP 700- Microsoft Fabric Training | Episode 2: Lakehouse, Warehouse & T-SQL Date: June 15, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 3: Lakehouse with Spark SQL Date: June 16, 6:30 PM Host: Amit Chandak Microsoft Cloud & AI Frontier Week: Modernize SQL for AI-Ready Databases Date: June 24, 03:00 AM Host: Diaa Radwan Microsoft Cloud & AI Frontier Week: Power Intelligent Apps and Agents with Azure Databases Date: June 24, 04:00 AM Host: Diaa Radwan Build with SQL + AI: From Prompt to Intelligent Apps Date: June 25, 01:00 PM Host: Matt Gordon, Alpa Buddhabhatti Modeling Real LMS Data in Power BI: Star Schema from a Messy MySQL Source Date: June 28, 06:30 AM Host: Parul Rani Sagar Starting with Data API Builder in 10 minutes Date: June 30, TBD Host: Phillip Burton Designing and Implementing Database Objects in Azure SQL Database Date: July 11, 9:30 AM Host: Amit Kumar Mahato (Cloud Guru Amit) Advanced Query Techniques in Azure SQL Date: July 12, 9:30 AM Host: Amit Kumar Mahato (Cloud Guru Amit) Get Certified SQL+AI (DP-800): Design and Develop SQL Solutions Like a Pro (EMEA / US) Date: July 15, 8:00 AM Host: Javier Villegas; Hamish Watson Get Certified SQL+AI (DP-800): Design and Develop SQL Solutions Like a Pro (APAC) Date: July 16, 3:00 PM Host: Martin Catherall; Greg Low Implementing Programmability Objects in Azure SQL Database Date: July 18, 9:30 AM Host: Amit Kumar Mahato (Cloud Guru Amit) Securing Data Access in Azure SQL Database Date: July 19, 9:30 AM Host: Amit Kumar Mahato (Cloud Guru Amit) Get Certified DP-800: Secure, Optimize, & Ship SQL+AI Solutions (APAC) Date: July 20, 4:00 PM Host: Mike Fortman; Mayte Castillo Get Certified DP-800: Secure, Optimize, & Ship SQL+AI Solutions (EMEA/US) Date: July 21, 8:00 AM Host: Jeff Taylor; Matt Gordon Get Certified SQL+AI (DP-800): Bring AI to SQL with Embeddings, Search, and RAG (EMEA / US) Date: July 23, 8:00 AM Host: Gaston Cruz; Armando Lacerda Optimizing Performance and integrity in Azure SQL Database Date: July 25, 9:30 AM Host: Amit Kumar Mahato (Cloud Guru Amit) Win in the Seams 🧵 Stitching Together Data and Security with Microsoft Fabric, KQL, and Data Logs Date: July 25, 09:30 AM Host: Philippa Burgess Get Certified SQL+AI (DP-800): Bring AI to SQL with Embeddings, Search, and RAG (APAC) Date: July 29, 3:00 PM Host: Greg Low; Anupama Natarajan Query and modify data with Transact-SQL Self-paced training SQL AI Engineer Certification Training (DP-800) Discover resources to prepare for this exam. DP 700- Microsoft Fabric Training | Episode 2: Lakehouse, Warehouse & T-SQL Date: June 15, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 3: Lakehouse with Spark SQL Date: June 16, 6:30 PM Host: Amit Chandak Get Certified SQL+AI (DP-800): Design and Develop SQL Solutions Like a Pro (EMEA / US) Date: July 15, 8:00 AM Host: Javier Villegas; Hamish Watson Get Certified SQL+AI (DP-800): Design and Develop SQL Solutions Like a Pro (APAC) Date: July 16, 3:00 PM Host: Martin Catherall; Greg Low Get Certified DP-800: Secure, Optimize, & Ship SQL+AI Solutions (APAC) Date: July 20, 4:00 PM Host: Mike Fortman; Mayte Castillo Get Certified DP-800: Secure, Optimize, & Ship SQL+AI Solutions (EMEA/US) Date: July 21, 8:00 AM Host: Jeff Taylor; Matt Gordon Get Certified SQL+AI (DP-800): Bring AI to SQL with Embeddings, Search, and RAG (EMEA / US) Date: July 23, 8:00 AM Host: Gaston Cruz; Armando Lacerda Get Certified SQL+AI (DP-800): Bring AI to SQL with Embeddings, Search, and RAG (APAC) Date: July 29, 3:00 PM Host: Greg Low; Anupama Natarajan Back to top Using AI Microsoft Cloud & AI Frontier Week: Transform Data Silos into AI Fuel – Build an End-to-End Data Foundation Date: June 22, 04:00 AM Host: Mark Torr, Chris Webb, Sarina Stevens Microsoft Cloud & AI Frontier Week: Unify Your Data with OneLake for Analytics, AI and Agents Date: June 24, 01:00 AM Host: Sevgi Guzzella Microsoft Cloud & AI Frontier Week: Turn Data into Intelligent Action with Microsoft Fabric Date: June 24, 02:00 AM Host: Simon Lidberg Microsoft Cloud & AI Frontier Week: Modernize SQL for AI-Ready Databases Date: June 24, 03:00 AM Host: Diaa Radwan Microsoft Cloud & AI Frontier Week: Power Intelligent Apps and Agents with Azure Databases Date: June 24, 04:00 AM Host: Diaa Radwan Microsoft Cloud & AI Frontier Week: Transform Fragmented Data into Trusted AI at Scale – A Roadmap for CDOs Date: June 24, 05:00 AM Host: Seda Teber & Marcel Franke The Future of AI in Microsoft Fabric: Data Agents and Beyond Date: June 24, 08:00 AM Host: Brian Bønk, Philippa Burgess DP 700- Microsoft Fabric Training | Episode 9: Mirroring, Databases, Composite Models & Data Agents Date: June 24, 6:30 PM Host: Amit Chandak Build with SQL + AI: From Prompt to Intelligent Apps Date: June 25, 01:00 PM Host: Matt Gordon, Alpa Buddhabhatti Data and AI Security and Governance in Microsoft Fabric Date: July 7, 07:30 AM Host: Philippa Burgess Exploring Data with AI and Natural Language Features Date: July 13, 6:00 AM Host: Ilgar Zarbaliyev Get Certified SQL+AI (DP-800): Bring AI to SQL with Embeddings, Search, and RAG (EMEA / US) Date: July 23, 8:00 AM Host: Gaston Cruz; Armando Lacerda Get Certified SQL+AI (DP-800): Bring AI to SQL with Embeddings, Search, and RAG (APAC) Date: July 29, 3:00 PM Host: Greg Low; Anupama Natarajan DP 700- Microsoft Fabric Training | Episode 9: Mirroring, Databases, Composite Models & Data Agents Date: June 24, 6:30 PM Host: Amit Chandak Get Certified SQL+AI (DP-800): Bring AI to SQL with Embeddings, Search, and RAG (EMEA / US) Date: July 23, 8:00 AM Host: Gaston Cruz; Armando Lacerda Get Certified SQL+AI (DP-800): Bring AI to SQL with Embeddings, Search, and RAG (APAC) Date: July 29, 3:00 PM Host: Greg Low; Anupama Natarajan Back to top New Fabric / Power BI / SQL Users DP 700- Microsoft Fabric Training | Episode 1: Fabric Overview, Domains, Workspaces & OneLake Date: June 14, 6:30 PM Host: Amit Chandak Fabric Data Pipelines Full Course For Beginners (Data Days Edition 2026) Date: June 21, 05:30 AM Host: Ansh Lamba Get Certified: Which Data Exam Fits You Best? Date: June 23, 12pm Host: Dean Jurecic, Taylor Amy Learn KQL in 10 minutes Date: June 29 Host: Phillip Burton Starting with Data API Builder in 10 minutes Date: June 30, TBD Host: Phillip Burton From “I’m Just Getting Started” to “I Made This” Date: July 28, 8:00 AM Host: Philippa Burgess; Taylor Amy Get started with Microsoft data analytics Self-paced training Get started with Microsoft Fabric Self-paced training Introduction to Microsoft Azure Data core data concepts Self-paced training Query and modify data with Transact-SQL Self-paced training DP 700- Microsoft Fabric Training | Episode 1: Fabric Overview, Domains, Workspaces & OneLake Date: June 14, 6:30 PM Host: Amit Chandak From “I’m Just Getting Started” to “I Made This” Date: July 28, 8:00 AM Host: Philippa Burgess; Taylor Amy Back to top Certification Prep Certification Resources Which Data Exam Fits You Best? Date: June 23, 12pm Host: Dean Jurecic, Taylor Amy What to Expect and How to Pass Date: June 25, 8am Host: Heini Ilmarinen, Teemu Multanen Get Certified: (Exam Day) What to Expect and How to Pass (US/EMEA) Date: August 6, 8:00 AM Host: Brian Bønk; Charley Hanania Find a Study Group DP-600, DP-700, DP-800, and PL-300 Free Certification Exam Voucher DP-600, DP-700 or DP-800 Get Certified: (Exam Day) What to Expect and How to Pass (US/EMEA) Date: August 6, 8:00 AM Host: Brian Bønk; Charley Hanania DP-600 - Fabric Analytics Engineer Get Certified: (DP-600) Fabric Analytics Engineer Essentials (APAC) Date: June 17, 3pm Host: Heidi Hasting, Martin Catherall Certifícate: (DP-600) Fabric Analytics Engineer Conceptos Clave Date: June 17, 4pm Host: Renzo Roca, Javier Villegas Get Certified: (DP-600) Fabric Analytics Engineer Essentials (US/EMEA) Date: June 18, 8am Host: Ásgeir Gunnarsson, Rajendra Ongole Certifique-se: (DP-600) Fundamentos de Analytics no Fabric Date: June 18, 12pm Host: Ladislau Andre, Roberto Fonseca DP-600 to Real Project: What the Certification Taught Me (and What It Didn't) Date: July 11, 09:30 PM Host: Parul Rani Sagar Microsoft Certified: Fabric Analytics Engineer Associate(DP 600) Date: July 17, 08:30 PM Host: Inturi Suparna Babu, Ajay Babu Inturi, Upputuri Gopikrishna DP-600 Exam Prep — Fabric Analytics Engineer with Data Analytic Group Date: July 25, 09:00 PM Host: Rajendra Ongole,Lanka, Shashi Prepare for Exam DP-600 Prep resources DP-600 In Depth On-demand recorded sessions Free Certification Exam Voucher DP-600, DP-700 or DP-800 Find a Study Group DP-600, DP-700, DP-800, and PL-300 DP-700 - Fabric Data Engineer DP 700- Microsoft Fabric Training | Episode 1: Fabric Overview, Domains, Workspaces & OneLake Date: June 14, 6:30 PM Host: Amit Chandak Certifique-se: (DP-700) Fundamentos de Dados no Fabric Date: June 15, 12pm Host: Luiz Santana, Percy Machado Certifícate: (DP-700) Fabric Data Engineer Conceptos Clave Date: June 15, 4pm Host: Gonzalo bissio, Keyla Dolores Mendez DP 700- Microsoft Fabric Training | Episode 1: Fabric Overview, Domains, Workspaces & OneLake Date: June 15, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 2: Lakehouse, Warehouse & T-SQL Date: June 15, 6:30 PM Host: Amit Chandak Get Certified: (DP-700) Fabric Data Engineer Essentials (APAC) Date: June 16, 3:00 PM Host: Mike Fortman, Martin Catherall DP 700- Microsoft Fabric Training | Episode 2: Lakehouse, Warehouse & T-SQL Date: June 16, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 3: Lakehouse with Spark SQL Date: June 16, 6:30 PM Host: Amit Chandak Get Certified: (DP-700) Fabric Data Engineer Essentials (US/EMEA) Date: June 17, 8:00 AM Host: Aleksi Partanen, Phillip Burton DP 700- Microsoft Fabric Training | Episode 3: Lakehouse with Spark SQL Date: June 17, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 4: Dataflow Gen2 End-to-End Date: June 17, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 4: Dataflow Gen2 End-to-End Date: June 18, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 5: PySpark Notebooks for Data Engineering Date: June 18, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 5: PySpark Notebooks for Data Engineering Date: June 19, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 6: Data Pipelines, Scheduling & OneLake Shortcuts Date: June 21, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 6: Data Pipelines, Scheduling & OneLake Shortcuts Date: June 22, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 7: Real-Time Analytics, Eventstream, Eventhouse & KQL Date: June 22, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 7: Real-Time Analytics, Eventstream, Eventhouse & KQL Date: June 23, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 8: Direct Lake Semantic Models & Power BI Performance Date: June 23, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 8: Direct Lake Semantic Models & Power BI Performance Date: June 24, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 9: Mirroring, Databases, Composite Models & Data Agents Date: June 24, 6:30 PM Host: Amit Chandak Como passar na Certificação DP-700: Guia Definitivo! Date: June 25, 3:00 PM Host: Sidney Cirqueira DP 700- Microsoft Fabric Training | Episode 10: End-to-End Fabric Project & DP-700 Exam Preparation Date: June 25, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 9: Mirroring, Databases, Composite Models & Data Agents Date: June 25, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 10: End-to-End Fabric Project & DP-700 Exam Preparation Date: June 26, 6:30 PM Host: Amit Chandak Getting Started with PySpark for DP-700 Date: July 16, 9:00 AM Host: Teemu Multanen DP-700 In Depth On-demand recorded sessions Find a Study Group DP-600, DP-700, DP-800, and PL-300 Free Certification Exam Voucher DP-600, DP-700 or DP-800 Prepare for Exam DP-700 Prep resources DP 700- Microsoft Fabric Training | Episode 1: Fabric Overview, Domains, Workspaces & OneLake Date: June 14, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 2: Lakehouse, Warehouse & T-SQL Date: June 15, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 3: Lakehouse with Spark SQL Date: June 16, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 4: Dataflow Gen2 End-to-End Date: June 17, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 5: PySpark Notebooks for Data Engineering Date: June 18, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 6: Data Pipelines, Scheduling & OneLake Shortcuts Date: June 21, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 7: Real-Time Analytics, Eventstream, Eventhouse & KQL Date: June 22, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 8: Direct Lake Semantic Models & Power BI Performance Date: June 23, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 9: Mirroring, Databases, Composite Models & Data Agents Date: June 24, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 10: End-to-End Fabric Project & DP-700 Exam Preparation Date: June 25, 6:30 PM Host: Amit Chandak DP-800 - SQL AI Engineer Get Certified SQL+AI (DP-800): Design and Develop SQL Solutions Like a Pro (EMEA / US) Date: July 15, 8:00 AM Host: Javier Villegas; Hamish Watson Get Certified SQL+AI (DP-800): Design and Develop SQL Solutions Like a Pro (APAC) Date: July 16, 3:00 PM Host: Martin Catherall; Greg Low Get Certified DP-800: Secure, Optimize, & Ship SQL+AI Solutions (APAC) Date: July 20, 4:00 PM Host: Mike Fortman; Mayte Castillo Get Certified DP-800: Secure, Optimize, & Ship SQL+AI Solutions (EMEA/US) Date: July 21, 8:00 AM Host: Jeff Taylor; Matt Gordon Get Certified SQL+AI (DP-800): Bring AI to SQL with Embeddings, Search, and RAG (EMEA / US) Date: July 23, 8:00 AM Host: Gaston Cruz; Armando Lacerda Get Certified SQL+AI (DP-800): Bring AI to SQL with Embeddings, Search, and RAG (APAC) Date: July 29, 3:00 PM Host: Greg Low; Anupama Natarajan DP-800 In Depth On-demand recorded sessions Find a Study Group DP-600, DP-700, DP-800, and PL-300 Free Certification Exam Voucher DP-600, DP-700 or DP-800 Prepare for Exam DP-800 Prep resources Get Certified SQL+AI (DP-800): Design and Develop SQL Solutions Like a Pro (EMEA / US) Date: July 15, 8:00 AM Host: Javier Villegas; Hamish Watson Get Certified SQL+AI (DP-800): Design and Develop SQL Solutions Like a Pro (APAC) Date: July 16, 3:00 PM Host: Martin Catherall; Greg Low Get Certified DP-800: Secure, Optimize, & Ship SQL+AI Solutions (APAC) Date: July 20, 4:00 PM Host: Mike Fortman; Mayte Castillo Get Certified DP-800: Secure, Optimize, & Ship SQL+AI Solutions (EMEA/US) Date: July 21, 8:00 AM Host: Jeff Taylor; Matt Gordon Get Certified SQL+AI (DP-800): Bring AI to SQL with Embeddings, Search, and RAG (EMEA / US) Date: July 23, 8:00 AM Host: Gaston Cruz; Armando Lacerda Get Certified SQL+AI (DP-800): Bring AI to SQL with Embeddings, Search, and RAG (APAC) Date: July 29, 3:00 PM Host: Greg Low; Anupama Natarajan PL-300 - Power BI Data Analyst Using Slicers and What-If Parameters in Power BI Date: June 15, 6:00 AM Host: Ilgar Zarbaliyev Dynamic Currency Conversion in Power BI Date: June 22, 6:00 AM Host: Ilgar Zarbaliyev Get Certified: (PL-300) Power BI Data Analyst (US/EMEA) Date: June 22, 8am Host: Ilgar Zarbaliyev, Doher Drizzle Pablo Certifique-se: (PL-300) Fundamentos de Análise de Dados com Power BI Date: June 22, 12pm Host: Brendell Silva Gomes, Miguel Felix Get Certified: (PL-300) Power BI Data Analyst (APAC) Date: June 23, 3pm Host: Anupama Natarajan, Chris Hyde Certifícate: (PL-300) Power BI Data Analyst Conceptos Clave Date: June 23, 4pm Host: Adrian Fernandez Zenteno, Ricardo Rincón Implementing Row-Level Security (RLS) Date: June 29, 6:00 AM Host: Ilgar Zarbaliyev Building Interactive Dashboards and Data Alerts Date: July 6, 6:00 AM Host: Ilgar Zarbaliyev Microsoft Certified: Power BI Data Analyst Associate Date: July 18, 5:30 AM Host: Inturi Suparna Babu, Ajay Babu Inturi, Upputuri Gopikrishna Performing Analytics in Power BI using DAX Date: July 20, 6:00 AM Host: Ilgar Zarbaliyev Prepare for Exam PL-300 Prep resources PL-300 In Depth On-demand recorded sessions Free Certification Exam Voucher DP-600, DP-700 or DP-800 Find a Study Group DP-600, DP-700, DP-800, and PL-300 Back to top Data Engineering Deep Dives Managing secure access, trusted discovery & data sharing with OneLake (3 sessions) Date: June 16, 9:00 AM Host: Josh Ndemenge Tenant management with Sempy Date: June 16, 11:00 AM Host: Taylor Amy, Teemu Multanen DP 700- Microsoft Fabric Training | Episode 3: Lakehouse with Spark SQL Date: June 16, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 4: Dataflow Gen2 End-to-End Date: June 17, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 5: PySpark Notebooks for Data Engineering Date: June 18, 6:30 PM Host: Amit Chandak Orchestrating Fabric Spark and Best Practices for Production-Ready Workload Date: June 19, 8am Host: Santhosh Kumar Ravindran; Ashit Gosalia Data Ingestion and Discovery in Fabric Date: June 21, 09:30 AM Host: Amit Kumar Mahato (Cloud Guru Amit) DP 700- Microsoft Fabric Training | Episode 6: Data Pipelines, Scheduling & OneLake Shortcuts Date: June 21, 6:30 PM Host: Amit Chandak Modeling Real LMS Data in Power BI: Star Schema from a Messy MySQL Source Date: June 28, 06:30 AM Host: Parul Rani Sagar Orchestrating Pipelines, and Notebooks in Fabric Date: July 4, 09:30 AM Host: Amit Kumar Mahato (Cloud Guru Amit) Designing Data Load Strategies in Fabric Date: July 5, 09:30 AM Host: Amit Kumar Mahato (Cloud Guru Amit) Inside Fabric Runtime 2.0: Spark 4 and Delta 4 in Action Date: July 15, 4:00 PM Host: Arshad Ali and Miles Cole Building Scalable Bronze Layer in Microsoft Fabric Date: July 23, 09:00 AM Host: Aleksi Partanen, Teemu Multanen Managing secure access, trusted discovery & data sharing with OneLake (3 sessions) Date: June 16, 9:00 AM Host: Josh Ndemenge DP 700- Microsoft Fabric Training | Episode 3: Lakehouse with Spark SQL Date: June 16, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 4: Dataflow Gen2 End-to-End Date: June 17, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 5: PySpark Notebooks for Data Engineering Date: June 18, 6:30 PM Host: Amit Chandak DP 700- Microsoft Fabric Training | Episode 6: Data Pipelines, Scheduling & OneLake Shortcuts Date: June 21, 6:30 PM Host: Amit Chandak Inside Fabric Runtime 2.0: Spark 4 and Delta 4 in Action Date: July 15, 4:00 PM Host: Arshad Ali and Miles Cole Back to top Dataviz Using Slicers and What-If Parameters in Power BI Date: June 15, 6:00 AM Host: Ilgar Zarbaliyev Power BI Dataviz World Championships: Start Your Journey to Barcelona Date: June 16, 8am Host: Valerie Junk, Lakshmi Ponnurasan Power BI Dataviz World Championships: Comece sua jornada para Barcelona Date: June 16, 2pm Host: Samyr Moises, Dirceu Moraes Resende Power BI Dataviz World Championships: Comienza tu camino a Barcelona Date: June 16, 4pm Host: Walter Calcagno, Lucrecia Krause Dynamic Currency Conversion in Power BI Date: June 22, 6:00 AM Host: Ilgar Zarbaliyev Building Interactive Dashboards and Data Alerts Date: July 6, 6:00 AM Host: Ilgar Zarbaliyev Inside the Mind of a Dataviz World Champion Date: July 14, 8:00 AM Host: Valerie Junk; Santhana Lakshmi Ponnurasan Performing Analytics in Power BI using DAX Date: July 20, 6:00 AM Host: Ilgar Zarbaliyev From “I’m Just Getting Started” to “I Made This” Date: July 28, 8:00 AM Host: Philippa Burgess; Taylor Amy Power BI Dataviz World Championships Do you have what it takes? Inside the Mind of a Dataviz World Champion Date: July 14, 8:00 AM Host: Valerie Junk; Santhana Lakshmi Ponnurasan From “I’m Just Getting Started” to “I Made This” Date: July 28, 8:00 AM Host: Philippa Burgess; Taylor Amy Back to top Community connections Global Fabric Day 2026 Date: June 27 Host: Kim Manis Data & IA sous contrôle - Meetup MTG:Bordeaux Juillet 2026 - Data Days Edition Date: July 2, 9:30 AM Host: Youva Gharout, Iuliia Mazur, Pierre Chaumont, Xavier Noya, Christian Bonnaud, Alexandre Nédélec Build Connections. Grow Your Career. Start Here for Your Community in India. Date: July 13, 11:00 PM Host: Amit Chandak; Vinodh Kumar Stop Lurking, Start Connecting: You Belong In the Microsoft Data & AI Communities Date: July 16, 9:00 AM Host: Teemu Multanen; Santhana Lakshmi Ponnurasan; Mike Fortman Data Days Hotline: No Slides, Just Answers Date: July 22, 8:00 AM Host: Johan Ludvig Brattås; Markus Ehrenmueller-Jensen Data Days Lightning Talks: Short Talks. Big Ideas. Zero Fluff. Date: July 30, 8:00 AM Host: Jennifer Ratten; Stephanie Bruno Data Days Universidad ICESI - Cali, Colombia Date: August 5, 12:00 PM Host: Álvaro Rodríguez, Cristhian Cabra, Angely Andrade, Andrés Gallego Connect on Reddit Fabric Fina a local Azure Group On the Meetup Find a local User Group On Meetup Find a Study Group DP-600, DP-700, DP-800, and PL-300 More Ways to Connect Find your community Start a New User Group Learn more about User Groups Build Connections. Grow Your Career. Start Here for Your Community in India. Date: July 13, 11:00 PM Host: Amit Chandak; Vinodh Kumar Stop Lurking, Start Connecting: You Belong In the Microsoft Data & AI Communities Date: July 16, 9:00 AM Host: Teemu Multanen; Santhana Lakshmi Ponnurasan; Mike Fortman Data Days Hotline: No Slides, Just Answers Date: July 22, 8:00 AM Host: Johan Ludvig Brattås; Markus Ehrenmueller-Jensen Data Days Lightning Talks: Short Talks. Big Ideas. Zero Fluff. Date: July 30, 8:00 AM Host: Jennifer Ratten; Stephanie Bruno Back to top Spanish Certifícate: (DP-700) Fabric Data Engineer Conceptos Clave Date: June 15, 4pm Host: Gonzalo bissio, Keyla Dolores Mendez Power BI Dataviz World Championships: Comienza tu camino a Barcelona Date: June 16, 4pm Host: Walter Calcagno, Lucrecia Krause Certifícate: (DP-600) Fabric Analytics Engineer Conceptos Clave Date: June 17, 4pm Host: Renzo Roca, Javier Villegas Certifícate: (PL-300) Power BI Data Analyst Conceptos Clave Date: June 23, 4pm Host: Adrian Fernandez Zenteno, Ricardo Rincón Study Group: From Power BI to Microsoft Fabric: The DP-600 Analytics Engineer Vision Date: June 25, 2026 Host: Comunidad Power BI, Fabric & AI en Español Track: DP-600 | Location: Virtual Certifícate: Preparación para el examen – Qué esperar y cómo aprobar Date: June 25, 4pm Host: Gaston Cruz, Keyla Dolores Mendez Study Group: Operational Architecture in Fabric: Workspaces, Capacity, Governance, and Permissions Date: July 1, 2026 Host: Comunidad Power BI, Fabric & AI en Español Track: DP-600 | Location: Virtual Cierre de la temporada 3 - Data Days Edition Date: July 1, 10:00 AM Host: Ana María Bisbé, Diana Aguilera Reyna, Nelson López Centeno Study Group: Martes 7 de julio - Diseño e implementación de soluciones analíticas en Microsoft Fabric Date: July 7, 2026 Host: AP Data & IA Track: DP-700 | Location: Virtual Study Group: OneLake and Lakehouse: Data Strategy, Ingestion, and Unified Access Date: July 8, 2026 Host: Comunidad Power BI, Fabric & AI en Español Track: DP-600 | Location: Virtual Study Group: Jueves 9 de julio - Ingesta, transformación y procesamiento de datos Date: July 9, 2026 Host: AP Data & IA Track: DP-700 | Location: Virtual Data Days Edition by BIExpert - Fabric Data Engineering Date: July 10, 4:00 PM Host: Nicolas Nakasone, Natali Lujan Power BI + MCP: la nueva forma de conectar la IA con tus datos (6 sesiones, en español) — 13 de julio – 17 de agosto de 2026 Date: July 13, 07:00 AM Host: Vicente Antonio Juan Magallanes Study Group: Martes 14 de julio - Monitoreo, rendimiento y optimización de cargas de trabajo Date: July 14, 2026 Host: AP Data & IA Track: DP-700 | Location: Virtual Study Group: Data Warehouse in Fabric: Dimensional Modeling and Analytical Design Date: July 15, 2026 Host: Comunidad Power BI, Fabric & AI en Español Track: DP-600 | Location: Virtual Study Group: Jueves 16 de julio - Preparación para el examen DP-700 y sesión abierta de preguntas y respuestas Date: July 16, 2026 Host: AP Data & IA Track: DP-700 | Location: Virtual Study Group: Data Days - Grupo de Estudio DP-600 / DP-800 Date: July 18, 2026 Host: Cloud Experts Community Track: DP-600 | Location: In-Person | University Norbert Wiener Study Group: Data Preparation in Fabric: Quality, Transformation, SQL, and KQL Date: July 22, 2026 Host: Comunidad Power BI, Fabric & AI en Español Track: DP-600 | Location: Virtual Study Group: Sesión 1: Diseño y desarrollo de soluciones SQL Date: July 28, 2026 Host: AP Data & IA Track: DP-800 | Location: Virtual Study Group: Enterprise Semantic Models: DAX, Direct Lake, and Power BI Performance Date: July 29, 2026 Host: Comunidad Power BI, Fabric & AI en Español Track: DP-600 | Location: Virtual Study Group: Sesión 2: Seguridad, optimización y administración de soluciones SQL con confianza Date: July 30, 2026 Host: AP Data & IA Track: DP-800 | Location: Virtual Agentes inteligentes con Microsoft Fabric: MCP, LLM y datos empresariales (6 sesiones, en español) — 31 de julio – 4 de septiembre de 2026 Date: July 31, 07:00 AM Host: Vicente Antonio Juan Magallanes Study Group: Sesión 3: Incorpo Date: August 4, 2026 Host: AP Data & IA Track: DP-800 | Location: Virtual Study Group: End-to-End Fabric Solution: Security, Governance, Lifecycle, and DP-600 Preparation Date: August 5, 2026 Host: Comunidad Power BI, Fabric & AI en Español Track: DP-600 | Location: Virtual Data Days Universidad ICESI - Cali, Colombia Date: August 5, 12:00 PM Host: Álvaro Rodríguez, Cristhian Cabra, Angely Andrade, Andrés Gallego Data Days Edition by BIExpert - Fabric Data Engineering Date: August 7, 04:00 PM Host: Nicolas Nakasone, Natali Lujan Data Days Edition by BIExpert - Fabric Data Engineering Date: July 10, 4:00 PM Host: Nicolas Nakasone, Natali Lujan Back to top Português Certifique-se: (DP-700) Fundamentos de Dados no Fabric Date: June 15, 12pm Host: Luiz Santana, Percy Machado Power BI Dataviz World Championships: Comece sua jornada para Barcelona Date: June 16, 2pm Host: Samyr Moises, Dirceu Moraes Resende Certifique-se: (DP-600) Fundamentos de Analytics no Fabric Date: June 18, 12pm Host: Ladislau Andre, Roberto Fonseca Study Group: Serie 1 - Preparação para o Exame PL-300 (Power BI Data Analyst) Date: June 21, 2026 Host: Fabric Lusofono Track: PL-300 | Location: Virtual Building a Medallion Architecture in Microsoft Fabric Date: June 22 Host: Ladislau Andre Certifique-se: (PL-300) Fundamentos de Análise de Dados com Power BI Date: June 22, 12pm Host: Brendell Silva Gomes, Miguel Felix Designing Modern Data Architectures with Microsoft Fabric Date: June 23, 11:00 AM Host: To be announced Certifique-se: Dia do Exame — O que esperar e como passar Date: June 25, 12pm Host: Luiz Santana, Miguel Felix Como passar na Certificação DP-700: Guia Definitivo! Date: June 25, 3:00 PM Host: Sidney Cirqueira Medallion Architecture + Data Mesh Architecture Date: June 26, TBD Host: Ladislau Andre Introdução à Análise de Dados da Microsoft Date: June 27, 08:00 PM Host: Shalom André, Filomena Adão Semantic Models in Power BI and Fabric Date: June 28, TBD Host: Ladislau Andre Study Group: Serie 2 - Preparação para o Exame DP-600 (Fabric Analytics Engineer) Date: June 28, 2026 Host: Fabric Lusofono Track: DP-600 | Location: Virtual Semantic Models Schedule Refresh with Pipelines Date: June 30, TBD Host: Ladislau Andre Semantic Models on Fabric notebooks Date: July 2, TBD Host: Ladislau Andre Semantic Models In Enterprise BI Date: July 4, 11:00 AM Host: Ladislau Andre Preparar Dados para Análise com o Power BI Date: July 4, 08:00 PM Host: Shalom André, Evaristo Quiosa Study Group: Serie 3 - Preparação para o Exame DP-700 (Fabric Data Engineer) Date: July 5, 2026 Host: Fabric Lusofono Track: DP-700 | Location: Virtual Modelar Dados com o Power BI Date: July 11, 08:00 PM Host: Shalom André Study Group: Serie 4 - Preparação para o Exame DP-800 (SQL Developer & AI Solutions) Date: July 12, 2026 Host: Fabric Lusofono Track: DP-800 | Location: Virtual Criar Relatórios Eficazes no Power BI Date: July 18, 08:00 PM Host: Shalom André Pare de scrolar e comece a se conectar: você pertence às comunidades de Dados e IA da Microsoft. Date: July 21, 12:00 PM Host: Alison Pezzott; Sidney Oliveira Cirqueira Dentro do Cérebro de campeões mundiais de Dataviz Date: July 21, 2:00 PM Host: Percy Machado; Samyr Moises; Paulo Grijó Get Certified SQL+AI (DP-800): Projete e Desenvolva Soluções SQL como um Profissional Date: July 23, 12:00 PM Host: Armando Lacerda; Ladislau André Gerenciar e Proteger o Power BI Date: July 25, 08:00 PM Host: Shalom André, Milton Dunda Preparação para o Exame e Sessão Q&A Date: July 26, 08:00 PM Host: Shalom André De “tô só começando” para “caramba, eu fiz isso” Date: July 28, 12:00 PM Host: Hugo Venturini; Dirceu Moraes Resende Get Certified SQL+AI (DP-800): Proteja, Otimize e Entregue Soluções SQL com Confiança Date: July 30, 12:00 PM Host: Brendell Silva Gomes; Eda Oliviera Get Certified SQL+AI (DP-800): Leve IA para o SQL com Embeddings, Busca e RAG Date: August 4, 12:00 PM Host: Luis Gustavo Nascimento Serra; Thiago Zavaschi Pare de scrolar e comece a se conectar: você pertence às comunidades de Dados e IA da Microsoft. Date: July 21, 12:00 PM Host: Alison Pezzott; Sidney Oliveira Cirqueira Dentro do Cérebro de campeões mundiais de Dataviz Date: July 21, 2:00 PM Host: Percy Machado; Samyr Moises; Paulo Grijó Get Certified SQL+AI (DP-800): Projete e Desenvolva Soluções SQL como um Profissional Date: July 23, 12:00 PM Host: Armando Lacerda; Ladislau André De “tô só começando” para “caramba, eu fiz isso” Date: July 28, 12:00 PM Host: Hugo Venturini; Dirceu Moraes Resende Get Certified SQL+AI (DP-800): Proteja, Otimize e Entregue Soluções SQL com Confiança Date: July 30, 12:00 PM Host: Brendell Silva Gomes; Eda Oliviera Get Certified SQL+AI (DP-800): Leve IA para o SQL com Embeddings, Busca e RAG Date: August 4, 12:00 PM Host: Luis Gustavo Nascimento Serra; Thiago Zavaschi Back to top French Study Group: Building the Microsoft Fabric Analytics Foundation (French | Virtual) Date: July 18, 2026 Host: Data & AI France Study Group Track: DP-600 | Location: Virtual Study Group: Prepare data with Power BI desktop and Power Query (French | Virtual) Date: July 25, 2026 Host: Data & AI France Study Group Track: PL-300 | Location: Virtual Study Group: Designing and Optimizing Semantic Models in Microsoft Fabric (French | Virtual) Date: July 25, 2026 Host: Data & AI France Study Group Track: DP-600 | Location: Virtual Study Group: Designing Modern SQL Database Solutions (French | Virtual) Date: July 30, 2026 Host: Data & AI France Study Group Track: DP-800 | Location: Virtual Study Group: Model data with Power BI Desktop (French | Virtual) Date: August 1, 2026 Host: Data & AI France Study Group Track: PL-300 | Location: Virtual Study Group: Building Enterprise Reports, Governance and End-to-End Analytics (French | Virtual) Date: August 1, 2026 Host: Data & AI France Study Group Track: DP-600 | Location: Virtual Study Group: Ingesting and Managing Data with Microsoft Fabric (French | Virtual) Date: August 1, 2026 Host: Data & AI France Study Group Track: DP-700 | Location: Virtual Study Group: Visualize, secure and deploy data on the Power BI service (French | Virtual) Date: August 2, 2026 Host: Data & AI France Study Group Track: PL-300 | Location: Virtual Study Group: Transforming and Engineering Data at Scale (French | Virtual) Date: August 8, 2026 Host: Data & AI France Study Group Track: DP-700 | Location: Virtual Study Group: Developing and Querying SQL Databases (French | Virtual) Date: August 8, 2026 Host: Data & AI France Study Group Track: DP-800 | Location: Virtual Data & IA sous contrôle - Meetup MTG:Bordeaux Juillet 2026 - Data Days Edition Date: July 2, 9:30 AM Host: Youva Gharout, Iuliia Mazur, Pierre Chaumont, Xavier Noya, Christian Bonnaud, Alexandre Nédélec From rows to reasoning: Designing databases for AI apps and agents - Morocco Data Days Edition, Date: July 30, 11:00 AM Host: ANAS BELABBES Back to top Japanese / 日本語 Study Group: DP 600 Session 1 Date: July 8, 2026 Host: Japan Microsoft Data Platform User Group Track: DP-600 | Location: Virtual Study Group: DP 700 Session 1 Date: July 9, 2026 Host: Japan Microsoft Data Platform User Group Track: DP-700 | Location: Virtual Study Group: DP 700 Session 2 Date: July 15, 2026 Host: Japan Microsoft Data Platform User Group Track: DP-700 | Location: Virtual Study Group: DP 600 Session 2 Date: July 16, 2026 Host: Japan Microsoft Data Platform User Group Track: DP-600 | Location: Virtual Study Group: DP 700 Session 3 Date: July 18, 2026 Host: Japan Microsoft Data Platform User Group Track: DP-700 | Location: Virtual Study Group: DP 700 Session 4 Date: July 20, 2026 Host: Japan Microsoft Data Platform User Group Track: DP-700 | Location: Virtual Study Group: DP 600 Session 3 Date: July 22, 2026 Host: Japan Microsoft Data Platform User Group Track: DP-600 | Location: Virtual Study Group: DP 600 Session 4 Date: July 29, 2026 Host: Japan Microsoft Data Platform User Group Track: DP-600 | Location: Virtual Back to top Hindi Road to Microsoft Data Days Date: June 15, 05:30 AM Host: Aman Jindal Prepare for the Microsoft Data Days Date: June 22, 5 am Host: Aman Jindal DP 700 Hindi- Microsoft Fabric Training | Episode 1: Fabric Overview, Domains, Workspaces & OneLake Date: July 19, 06:30 PM Host: Amit Chandak DP 700 Hindi- Microsoft Fabric Training | Episode 2: Lakehouse, Warehouse & T-SQL Date: July 20, 06:30 PM Host: Amit Chandak DP 700 Hindi- Microsoft Fabric Training | Episode 3: Lakehouse with Spark SQL Date: July 21, 06:30 PM Host: Amit Chandak DP 700 Hindi- Microsoft Fabric Training | Episode 4: Dataflow Gen2 End-to-End Date: July 22, 06:30 PM Host: Amit Chandak DP 700 Hindi- Microsoft Fabric Training | Episode 5: PySpark Notebooks for Data Engineering Date: July 23, 06:30 PM Host: Amit Chandak DP 700 Hindi- Microsoft Fabric Training | Episode 6: Data Pipelines, Scheduling & OneLake Shortcuts Date: July 26, 06:30 PM Host: Amit Chandak DP 700 Hindi- Microsoft Fabric Training | Episode 7: Real-Time Analytics, Eventstream, Eventhouse & KQL Date: July 27, 06:30 PM Host: Amit Chandak DP 700 Hindi- Microsoft Fabric Training | Episode 8: Direct Lake Semantic Models & Power BI Performance Date: July 28, 06:30 PM Host: Amit Chandak DP 700 Hindi- Microsoft Fabric Training | Episode 9: Mirroring, Databases, Composite Models & Data Agents Date: July 29, 06:30 PM Host: Amit Chandak DP 700 Hindi- Microsoft Fabric Training | Episode 10: End-to-End Fabric Project & DP-700 Exam Preparation Date: July 30, 06:30 PM Host: Amit Chandak Back to top
Microsoft Fabric Database Hub: A Unified Control Plane for the Modern Database Estate
Introduction Organizations today manage an increasingly complex database landscape. Mission-critical data lives across Azure SQL Database, SQL Server, Azure Database for PostgreSQL, Azure Cosmos DB, Fabric databases, and various on-premises environments. While each platform serves important business needs, managing them often requires separate monitoring tools, governance frameworks, operational processes, and administrative experiences. As enterprises accelerate their AI and data transformation initiatives, fragmented database management becomes a significant challenge. Database administrators, platform teams, and data leaders need a unified way to understand the health, security, performance, and operational posture of their entire database estate. Microsoft Fabric Database Hub addresses this challenge by introducing a centralized database management experience inside Microsoft Fabric. Rather than managing databases through multiple portals and disconnected tools, organizations gain a single operational experience to discover, observe, govern, and optimize databases across cloud, on-premises, and Fabric environments. What is Microsoft Fabric Database Hub? Database Hub is a unified operational experience within Microsoft Fabric that provides centralized visibility into an organization's database estate. The Database Hub enables customers to: Discover databases across multiple platforms Monitor performance and health Identify operational risks Improve governance and compliance Receive AI-assisted recommendations Investigate issues across environments Prioritize remediation activities Instead of jumping between Azure Portal, SQL Management tools, monitoring platforms, and Fabric workspaces, administrators gain a single pane of glass for database operations. The vision is simple: One place to see the estate, understand what matters, and take the next best action. Why Database Hub Matters The Challenge of Database Fragmentation Most enterprises operate hundreds or even thousands of databases across multiple technologies: Azure SQL Databases Azure SQL Managed Instances SQL Server Azure Database for PostgreSQL Azure Cosmos DB SQL Database in Fabric Hybrid and Arc-enabled deployments Each platform typically has: Separate monitoring tools Separate security dashboards Different governance processes Different performance views Different troubleshooting experiences As AI initiatives expand, organizations require broader visibility across operational and analytical systems. Database Hub brings these experiences together. A Strategic Shift Historically, organizations focused on managing individual databases. Database Hub introduces a new operational model: Manage the entire database fleet rather than managing databases one at a time. This estate-first approach enables platform teams to: Identify systemic issues Prioritize risks Establish governance standards Improve operational consistency Scale DBA operations efficiently Supported Database Platforms Database Hub provides a unified view across Microsoft database technologies including: Azure SQL Database Azure SQL Elastic Pools Azure SQL Managed Instance SQL Database in Fabric Azure Cosmos DB Azure Database for PostgreSQL SQL Server enabled by Azure Arc SQL Server running on Azure Virtual Machines This broad support allows customers to modernize at their own pace while maintaining visibility into legacy and modern environments. Key Capabilities 1. Estate-Level Visibility One of the most powerful capabilities of Database Hub is estate-wide visibility. Database administrators can: View all databases in a single inventory Search and filter resources Group databases by platform Understand deployment distribution Assess operational health Instead of reviewing systems individually, administrators gain immediate visibility across their entire estate. 2. Centralized Monitoring Database Hub introduces a consolidated monitoring experience. Teams can analyze: Database health Availability indicators Capacity trends Utilization metrics Resource consumption Performance patterns The platform helps identify emerging issues before they become business-impacting incidents. 3. Unified Performance Insights Performance data is often scattered across multiple monitoring tools. Database Hub provides: Estate-wide performance visibility Cross-database trend analysis Real-time monitoring Historical performance analysis Administrators can quickly understand: Which databases require attention Performance degradation trends Resource bottlenecks Optimization opportunities 4. Issues and Recommendations Database Hub automatically surfaces: Issues Issues represent conditions requiring immediate attention: Security concerns Configuration problems Performance degradation Availability risks Operational anomalies Suggestions Suggestions identify opportunities for improvement: Performance tuning Cost optimization Security enhancements Utilization improvements Best-practice recommendations This helps teams move from reactive operations to proactive optimization. 5. AI-Assisted Database Operations One of the most exciting innovations is the integration of AI-driven assistance. Database Hub leverages intelligent database agents and Copilot-powered experiences to help users: Understand operational changes Investigate risks Diagnose issues Recommend corrective actions Identify optimization opportunities Rather than simply displaying metrics, the platform helps explain: What changed Why it matters What action should be taken This dramatically reduces time-to-resolution and accelerates troubleshooting. 6. Governance and Compliance Visibility Governance remains a top concern for regulated organizations. Database Hub supports: Centralized governance visibility Security posture monitoring Policy reporting Compliance tracking Risk identification Importantly, organizations can maintain their existing governance and operational models while benefiting from centralized observability. This is particularly valuable for industries such as: Oil & Gas Energy Financial Services Healthcare Government Manufacturing 7. Hybrid and Multicloud Awareness Many organizations continue operating hybrid environments. Database Hub acknowledges this reality by providing visibility across: Cloud databases On-premises databases Arc-enabled environments Fabric-native databases Customers gain a consistent operational experience without requiring workload migration. How Database Hub Fits into the Fabric Vision Microsoft Fabric is evolving into a unified data platform that combines: Data Engineering Data Factory Data Science Real-Time Intelligence Power BI OneLake Fabric Databases Fabric IQ Database Hub extends this strategy by integrating operational databases into the broader Fabric ecosystem. The result is a platform that spans: Operational Data Transaction processing Business applications Line-of-business systems Analytical Data Warehouses Lakehouses Power BI models AI Workloads Intelligent applications Retrieval-Augmented Generation (RAG) Agentic AI solutions Copilot experiences Database Hub serves as the operational bridge connecting these worlds. Benefits for Customers For Database Administrators Single management experience Faster troubleshooting Estate-wide visibility Reduced tool sprawl For Platform Teams Centralized governance Consistent operational standards Improved fleet management For Executives Better operational risk visibility Improved compliance posture Greater operational efficiency AI-ready database strategy For Data and AI Teams Better connection between operational and analytical systems Easier integration with OneLake Faster access to enterprise data Improved AI readiness Real-World Customer Scenario Consider a global energy company running: SQL Server on-premises Azure SQL Managed Instance Azure Database for PostgreSQL Azure Cosmos DB Fabric Databases Traditionally, each environment requires separate management processes. With Database Hub, the organization can: Discover all databases from one location. Monitor estate-wide health and performance. Identify security and compliance gaps. Receive AI-generated recommendations. Investigate issues before they impact production. Prioritize remediation activities across the entire portfolio. This shifts operations from reactive administration to intelligent estate management. The Future of Database Operations The database landscape is becoming more distributed and complex. At the same time, organizations expect: Greater reliability Lower operational costs Stronger governance Faster innovation AI-driven insights Database Hub represents Microsoft's vision for the future of database operations: A unified, AI-powered control plane capable of managing the complete database estate regardless of where workloads run. As organizations continue their modernization journey, solutions like Database Hub will become increasingly important for maintaining visibility, governance, and operational excellence. Final Thoughts Microsoft Fabric Database Hub is more than another management portal. It is a strategic evolution in how organizations operate, govern, and optimize their database estates. By bringing Azure, Fabric, SQL Server, PostgreSQL, Cosmos DB, and hybrid environments into a single operational experience, Database Hub empowers teams to move from fragmented database administration to intelligent estate-wide operations. For organizations pursuing data modernization and AI transformation, Database Hub provides a critical foundation: visibility, governance, observability, and actionability across the entire database landscape. In the era of AI, the winners will not simply be the organizations with the most data. They will be the organizations that can understand, govern, and operationalize their data estate most effectively. Database Hub in Microsoft Fabric is designed to help them do exactly that.From Chaos to Clarity: How the Manufacturing Dashboard Helps Everyone Understand the Business
Let's walk through what each page actually shows, and why it matters. Page 1: Customer Insights — "Who are we selling to, and are they happy?" Every business lives or dies by its customers, but it's surprisingly hard to keep track of hundreds of relationships in your head. This page acts like a customer relationship "scoreboard." It shows things like: * How many active customers the business currently has * Which customers are considered healthy (buying regularly, paying on time) versus at risk (going quiet, showing warning signs) * How revenue is spread across different customers and regions — are we relying too heavily on just a few big accounts? Think of it like a doctor's checkup, but for relationships instead of health. If ten customers suddenly go from "healthy" to "at risk," that's a signal to act before they walk away for good — not after the revenue has already dropped and everyone's asking why. Page 2: Production & Supply — "Are we making enough, and can we deliver it?" This is the page for anyone who cares about what's actually happening on the factory floor and getting product out the door. It's less about money and more about operations. Key questions it answers: * How much did we produce this month, and is that on target? * Are shipments going out on time, or are we falling behind on delivery promises? * Is one particular plant underperforming compared to the others? If you're a plant manager, this is likely the first thing you check every morning — before coffee, even. It tells you at a glance whether today is a "business as usual" day or a "we need to fix something now" day. Page 3: Overview — "How's the whole business doing, in 30 seconds?" This is the page built for someone with almost no time to spare — a CEO, an investor, or anyone who just needs the headline numbers without digging through details. At the top sit eight simple cards, each showing one important number and whether it went up or down compared to last month: * Total Revenue — how much money came in * Total Production — how much product was made * Capacity Utilization — how much of the factories' potential is actually being used * Gross Profit — how much money is left after production costs * Supply Fulfillment — how reliably orders are being delivered * Inventory Value — how much stock is sitting in the warehouse * Active Customers — how many customers are currently buying * Overall Operational Efficiency — a single score summarizing how smoothly everything is running Below that, charts break revenue and production down by month, by plant, and by region — so if a number drops, you don't just see that it dropped, you can immediately see where. Was it one plant having a bad month, or a slowdown across an entire region? There's also a simple "Alerts & Insights" section that puts the numbers into plain words — things like "Supply on track: fulfillment is 94% and improving" — so nobody has to guess what a chart is trying to tell them. Page 4: Inventory & Working Capital — "Do we have too much stock, or too little?" This page tackles a balancing act every manufacturing business faces. Keep too much inventory sitting around, and you're tying up cash and warehouse space that could be used elsewhere. Keep too little, and you risk running out of product right when a customer needs it — losing sales and trust. This page shows: * Total Inventory Value — how much money is currently tied up in stock * Inventory Turnover — how quickly that stock is being sold and replaced (a higher number generally means things are moving efficiently) * Days Inventory Outstanding — roughly how many days' worth of stock is sitting around unused * Raw Material vs. Finished Goods Stock — how much is still waiting to be turned into product versus ready to ship * Slow-Moving Inventory — stock that isn't selling and may need attention * Stockout Risk — a warning flag for items at risk of running out * Inventory Accuracy — how well the recorded stock counts match what's physically in the warehouse There's even a detailed table listing specific materials by name, plant, and status (like "Slow-Moving" or "Healthy"), so instead of a vague warning, someone gets a precise to-do list of exactly what needs a closer look. Why This Approach Works for Everyone The real magic of this dashboard isn't any single chart — it's the consistency. Every page follows the same basic structure: 1. Filters at the top (date range, region, plant, product category) so anyone can narrow the view to exactly what matters to them 2. A handful of key numbers, shown as simple cards with an up or down arrow — no complicated formulas to interpret 3. A plain-language "Alerts & Insights" section that explains, in a sentence or two, what changed and why it matters That means a machine operator, a plant manager, a customer success rep, and a CEO can all open the same dashboard and immediately find what's relevant to them — no translation needed, no waiting for someone else to "run the numbers." In a business with as many moving parts as manufacturing, that kind of shared clarity isn't a luxury. It's what keeps everyone — from the shop floor to the boardroom — pointed in the same direction.Global Aircraft✈️ Live Tracking with Microsoft Fabric Real-Time Intelligence
In this blog, I’ll walk you through, build a real-time global flight tracking system. We will ingest live flight data from the public OpenSky Network API using a Python polling script, stream it through Microsoft Fabric Eventstream into an Eventhouse (KQL Database), transform the dense raw arrays using KQL update policies and visualize the results on a Real-Time Dashboard complete with maps, KPIs and analytical charts. Prerequisites: Valid Fabric Capacity / Trail License Knowledge on Python Knowledge on KQL Step1: Setup Workspace & Eventhouse (KQL Database) Created a workspace “FlightTracking-[WS]” Created an eventhouse “FLightTracking-EH” Create a raw ingestion table “RawFlightbatc” in KQL Databse Step 2: Setup a Fabric Eventstream Created a Eventstream “GlobalFlightStream” and select ‘Use custom endpoint’ Click ‘Add’ Click on ‘Publish’ Copy the ‘Event hub name’ and ‘connection string-primary key’ into notepad Step 3: Notebook Creation and Setup Python script Created a notebook. Make sure select Python Install azure eventhub package Let’s go back to eventstream and add destination by selecting ‘Eventhouse’ Configure all details and click on save Comeback to Notebook and insert the Python script which is having all the connection strings / passwords etc. Run the notebook Now notebook started running and sending the data to eventstream Data is loaded into eventstream and Click on Publish Now eventstream is ‘Live’ Data is loading into KQL Database Step 4: Regularizing Data with KQL & Update Policies Create a cleaned table “FlightStates” Create the parsing function and update policy Alter table with updated policy Step 5: Building Real-Time Dashboard Click on Realtime dashboard and give a name “FlightOperationsDashboard” Now click on edit Run the below code to get total active flights count Change the chart to Stat and rename, click on Apply KPI added and click on Add visual and take new Stat visual Insert the code and run to get India Origin Flights and Format it and click Apply In the same way, I built other KPIs Select Map Chart Run the below code and Fill all details and Click on Apply. Here we’re calculating Flights trend We can see chart added to Dashboard Select a Bar chart Run the below code, fill all details and click on apply to add Bar chart to Dashboard. Here we’re getting the top 10 countries by aircrafts Run the below code, fill all details and click on apply to add Column chart to Dashboard. Here we’re categorizing the baro-altitude which is critical for airport delay predection Run the below code, fill all details and click on apply to add Pie chart to Dashboard. It splits the aircraft parked versus those actively flying Together, these visuals transform raw flight telemetry into an operational monitoring experience. Key takeaways: Fabric Eventstream provides a streamlined way to ingest external real-time feeds. Eventhouse/KQL Database provides a real-time analytical environment for flight telemetry. KQL mv-expand simplifies the processing of nested flight-state arrays. Update Policies automate transformation from raw streaming data into structured analytical data. KQL geospatial functions enable location-based flight analysis. Real-Time Dashboards transform streaming telemetry into actionable operational insights. Conclusion: This project demonstrates how Microsoft Fabric Real-Time Intelligence can be used to build an end-to-end real-time aircraft tracking solution. we can transform continuously arriving aircraft telemetry into meaningful real-time insights. Do you want to replicate? Get Code file from my GitHub link You can find all the KQL queries, update policy functions, and the complete Python polling script in the official GitHub repository below: [Download from here] Acknowledgements I would like to express my sincere gratitude to @SuryaTejaJosyul , @minniwalia and @rajendraongole1 for their continuous guidance and support throughout this Real-Time Intelligence (RTI) implementation. Their insights and encouragement played a key role in helping me complete this solution successfull Happy learning! — Inturi Suparna Babu [LinkedIn]Sequence Modeling for Data Science in Microsoft Fabric
In this edition, we're exploring how sequence modeling changes the way one thinks about data, why treating rows as independent often leads to shallow insights, and how RNNs and GRUs step in to model memory. By the end, you should have a clear intuition for when simple recurrence is enough, when gating really matters, and how to start applying this mindset in Microsoft Fabric (without overengineering anything!)Forecasting with Autoregression, ARIMA & AIC / BIC for Data Science in Microsoft Fabric
In this edition, we’re exploring forecasting through Autoregression, ARIMA, and the model selection tools AIC and BIC. By the time you’re done reading, you’ll understand how data can actually learn from its own patterns, how ARIMA helps bring structure to unpredictable trends, and how AIC and BIC keep your models grounded by balancing accuracy with simplicity.Model Explainability That Goes Beyond the Notebook for Data Science in Microsoft Fabric
A machine learning model can be remarkably accurate and still leave people completely unconvinced. That is one of the strange realities of working in data science. You can spend weeks cleaning data, testing algorithms, tuning parameters, and validating performance, only to reach the moment when somebody looks at a prediction and asks a very reasonable question: “Why?” Suddenly, accuracy alone does not feel like enough. A probability score might tell you what the model believes, but it does not necessarily help the person receiving that prediction understand how the model arrived there. That gap between prediction and understanding is exactly where model explainability becomes important. What you will learn: In this edition, we’re exploring model explainability in production and how SHAP, FastAPI, and Plotly can work together to make predictions easier to understand. By the time you’re done with this, you’ll have a clear view of how SHAP can explain individual model decisions, how those explanations can be served dynamically through an API, and how interactive visualizations can turn technical attribution values into something far more approachable. Source: Sahir Maharaj (https://sahirmaharaj.com) It is tempting to think of explainability as a chart you generate after training a model, but production explainability is really an ongoing capability. Every new observation can produce a different prediction, and every prediction may have a different explanation. A customer predicted to leave a service might receive a high risk score because of declining engagement, while another customer receives the same score because of repeated service issues. The prediction may look identical at the surface, but the reasoning behind it can be completely different. That difference matters when somebody needs to decide what to do next. This is where SHAP becomes particularly valuable because it gives you a structured way to describe feature contributions. Imagine a model predicting whether a machine is likely to fail. The prediction itself might tell you that failure risk is 82 percent. Useful, certainly, but incomplete. SHAP can help reveal that rising operating temperature, vibration intensity, and unusually long operating hours are pushing the prediction upward, while another factor such as recent maintenance is reducing the predicted risk. Suddenly the prediction becomes much easier to reason about. You still have a probabilistic model, but you also have something resembling an explanation of its behavior. As a data scientist, I find that this distinction between global and local understanding is especially important. During model development, I often want to understand the model globally. Which variables matter most overall? Is the model relying heavily on variables I expected it to use? Are there surprising relationships hiding in the data? In production, however, people often care about something much more specific. They want to know why this customer, this transaction, this machine, or this observation received the prediction it did. That is a local explanation, and SHAP is particularly useful for helping bridge that gap. Source: Sahir Maharaj (https://sahirmaharaj.com) You can think about the difference through a credit-risk example. Globally, a model might rely heavily on payment history, debt ratios, and income stability. That tells you something useful about the overall model. But suppose one applicant receives a higher risk prediction than expected. A general feature importance chart does not really answer the person's question. You need to understand what happened for that particular prediction. Perhaps the debt ratio increased the risk score significantly, while long-term employment reduced it. That level of explanation provides much more context than simply showing which variables matter across thousands of predictions. Explainability can also become an important debugging tool for you as the person building the model. Suppose a model performs well according to the metrics you are monitoring, but its explanations repeatedly show that one unexpected variable dominates many predictions. That should make you curious. import numpy as np import pandas as pd import shap import matplotlib.pyplot as plt from sklearn.ensemble import RandomForestClassifier from sklearn.model_selection import train_test_split np.random.seed(42) n = 800 X = pd.DataFrame({ "usage_drop": np.random.normal(20, 10, n), "support_calls": np.random.poisson(3, n), "login_days": np.random.randint(1, 30, n), "tenure_months": np.random.randint(1, 72, n), "monthly_cost": np.random.normal(80, 20, n) }) score = ( 0.08 * X["usage_drop"] + 0.45 * X["support_calls"] - 0.05 * X["login_days"] - 0.02 * X["tenure_months"] + 0.015 * X["monthly_cost"] ) y = (score + np.random.normal(0, 1, n) > np.median(score)).astype(int) X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, random_state=42 ) model = RandomForestClassifier(n_estimators=150, random_state=42) model.fit(X_train, y_train) explainer = shap.TreeExplainer(model) values = explainer(X_test) class_values = values[:, :, 1] if values.values.ndim == 3 else values shap.plots.beeswarm(class_values, max_display=5) plt.show() shap.plots.waterfall(class_values[0], max_display=5) plt.show() Imagine that somebody submits new information to a prediction service. The model receives those features, processes them, and generates a result. If explainability is part of the design, the same interaction can also produce SHAP information describing the important factors behind that specific prediction. Instead of storing explanations for every imaginable case beforehand, you generate them when they are needed. FastAPI fits naturally into this kind of workflow because it gives the prediction and explanation process a consistent interface. Rather than another application needing to understand your model directly, it can send the required information to an API. The API can coordinate the prediction process, calculate the explanation, and return the relevant output. The application consuming the result does not need to know the mathematical details of SHAP or how the model itself works internally. It simply receives information in an agreed structure. That separation can make a production system much easier to manage. From what I observe when building data science workflows, this separation becomes valuable surprisingly quickly. During early experimentation, everything often exists together. Data preparation, prediction, explanation, and visualization may all happen inside one environment. That is perfectly reasonable when you are exploring an idea. Production introduces more responsibilities. The application requesting a prediction should not need access to the entire modeling environment, and your model should not depend on somebody manually generating explanations. Creating a clear service boundary helps each part of the system do its job without becoming unnecessarily tangled with everything else. Dynamic explanations also give you flexibility over how much information you return. A technical monitoring application may want detailed SHAP values for every available feature. A decision-support interface may only need the five strongest contributors. Another application may need a short summary explaining the strongest positive and negative influences. The explanation engine can be the same while the presentation changes according to the audience. That matters because explainability is not only a modeling problem. It is also a communication problem. Source: Sahir Maharaj (https://sahirmaharaj.com) Performance deserves careful attention here as well. Calculating an explanation adds work beyond producing the prediction itself, and depending on the model, the amount of additional work may not be trivial. That does not mean explanations should be avoided. It means you should think about when they are required. Some systems may need an explanation for every single prediction. Others may generate explanations only when a user requests additional detail. You might also decide that certain high-impact decisions always receive explanations while low-impact automated predictions do not. Production design is often about finding that balance between completeness and responsiveness. There is also an important consistency issue. An explanation is only useful if it corresponds to the exact model and transformation process that generated the prediction. Imagine updating your model but accidentally continuing to generate explanations using an older version. The results could look perfectly reasonable while describing behavior that no longer matches the prediction being shown. import numpy as np import pandas as pd import shap import plotly.graph_objects as go from sklearn.ensemble import GradientBoostingClassifier np.random.seed(7) n = 700 X = pd.DataFrame({ "temperature": np.random.normal(70, 12, n), "vibration": np.random.normal(4, 1.5, n), "operating_hours": np.random.normal(900, 250, n), "maintenance_days": np.random.randint(1, 120, n) }) risk = ( 0.06 * X["temperature"] + 0.7 * X["vibration"] + 0.003 * X["operating_hours"] - 0.025 * X["maintenance_days"] ) y = (risk + np.random.normal(0, 1, n) > np.median(risk)).astype(int) model = GradientBoostingClassifier(random_state=7) model.fit(X, y) explainer = shap.Explainer(model, X) row = pd.DataFrame([{ "temperature": 92, "vibration": 6.8, "operating_hours": 1250, "maintenance_days": 75 }]) prediction = model.predict_proba(row)[0, 1] explanation = explainer(row) names = row.columns.tolist() impacts = explanation.values[0] waterfall = go.Figure(go.Waterfall( orientation="h", y=names, x=impacts, measure=["relative"] * len(names), text=[f"{x:+.2f}" for x in impacts], textposition="outside" )) waterfall.update_layout( title="SHAP Explanation: What Drove This Prediction?", xaxis_title="SHAP Impact on Model Output", yaxis_title="Feature", height=450 ) waterfall.show() gauge = go.Figure(go.Indicator( mode="gauge+number+delta", value=prediction * 100, number={"suffix": "%"}, delta={"reference": 50}, title={"text": "Predicted Machine Failure Risk"}, gauge={ "axis": {"range": [0, 100]}, "steps": [ {"range": [0, 40]}, {"range": [40, 70]}, {"range": [70, 100]} ], "threshold": { "line": {"width": 4}, "thickness": 0.8, "value": prediction * 100 } } )) gauge.update_layout(height=400) gauge.show() pd.DataFrame({ "Feature": names, "Value": row.iloc[0].values, "SHAP Impact": impacts }).sort_values("SHAP Impact", key=abs, ascending=False) The third objective is turning those explanation values into something people can interpret without needing to understand every detail of SHAP. This is where visualization becomes more than decoration. Raw values can tell you exactly how features influenced a prediction, but they require effort to interpret. A Plotly visualization can organise those contributions so the strongest influences immediately stand out. Features pushing a prediction in one direction can be separated visually from those pushing it in the opposite direction, allowing somebody to understand the basic story of the prediction within a few moments. Imagine a customer churn model that predicts an unusually high probability of cancellation. A dynamic explanation might show that a recent drop in product usage is the strongest contributor, followed by repeated support contacts and a reduction in login frequency. At the same time, a long customer relationship might slightly reduce the churn prediction. That is much more meaningful than presenting a table containing feature names and decimal values. The visualization helps the reader understand not only which features mattered but also how they worked together. Source: Sahir Maharaj (https://sahirmaharaj.com) When I look at explanations intended for other people, I try not to assume that more information automatically creates more transparency. It is very easy to generate a detailed visualization containing dozens of variables simply because the values are available. The result may technically be comprehensive while being practically overwhelming. If somebody has to study a chart for several minutes before understanding the main message, the explanation probably needs refinement. In many situations, highlighting the strongest contributors and allowing additional detail to be explored when needed creates a much better experience. Interactivity can make this even more useful. Plotly allows explanations to become something people can investigate instead of merely observe. A user might hover over a feature to inspect the contribution more closely or compare several predictions to understand why similar observations received different results. Imagine comparing two loan applications that received different risk scores. An interactive explanation can make it easier to see that one prediction was strongly influenced by debt levels while another was affected primarily by inconsistent payment history. That comparison helps reveal how the model responds to different combinations of information. At the same time, visual polish should never become a substitute for clarity. A sophisticated interactive chart can create an impression of authority, and that means you have a responsibility to provide enough context around what the visualization represents. The user should understand that the chart explains the model's behavior, not necessarily the underlying truth of the real-world situation. If a feature makes a large contribution, that means the model relied heavily on that feature for the prediction. It does not automatically prove that the feature caused the outcome. Keeping that distinction clear is a major part of responsible model communication. Source: Sahir Maharaj (https://sahirmaharaj.com) So give it a try. Do not wait until you have a huge production system or a complicated machine learning platform before thinking about transparency. Start with one prediction, one explanation, and one clear visualization. Build from there as your understanding grows. The more comfortable you become explaining what your model is doing, the easier it becomes to recognise when its behaviour deserves a closer look. And that ability is valuable whether you are just entering data science or have been building models for years. If you ever need anything while exploring it, just reach out to me. I’ll be more than happy to help. Thanks for taking the time to read my post! I’d love to hear what you think and connect with you! 🙂 LinkedIn Kaggle Topmate (Free Power BI / Data Science Sessions and Resources) Website The Tech Journal (Blog) About the author Sahir Maharaj is a Lead Data Scientist who leads the design and deployment of end-to-end AI solutions that drive strategic decisions at scale. As a Microsoft MVP, he has been featured internationally, including in The Indian Express and on New York's Times Square billboards, and is a prolific content creator on LinkedIn.
