New blog articles in Microsoft Fabric Community

Upcoming changes to legacy Teams and Outlook activities in Fabric Data Factory pipelines

conxu-ms — Thu, 17 Sep 2026 21:00:00 GMT

Beginning October 30, 2026, users will no longer be able to add new legacy Teams or legacy Outlook activities to pipelines.

These legacy activities will no longer appear as options when users add new activities to a pipeline. Existing legacy activity instances can still be edited and used in existing pipelines. New notification scenarios should instead use the modern Microsoft Teams and Outlook 365 Email activities.

Soft delete for On-premises data gateways in Microsoft Fabric (Preview)

goupadhy — Thu, 17 Sep 2026 17:00:00 GMT

Deleting a gateway is currently permanent, so administrators must recreate it if it is removed accidentally. With soft delete, supported gateways are retained for 30 days after deletion, giving authorized administrators time to restore them before permanent deletion.

Soft delete feature currently is only available for standard mode of on-premises data gateways in Microsoft Fabric (Preview). Personal mode gateways and virtual network data gateways are not currently supported.

What’s new

Soft delete introduces a recoverable deletion lifecycle for supported gateway types. When a supported gateway is deleted, it is retained for 30 days by default rather than being removed immediately and permanently. During the retention period, tenant admins and gateway admins can recover from it.

The gateway management page includes the Deleted gateways section which shows Deleted clusters and Deleted members. Gateway admins can view gateway clusters deleted within the 30-day retention window. Each entry in Deleted Clusters can show the gateway name, type, deletion timestamp and available actions, including Restore. Similarly, gateway admins can restore the deleted gateway members together in the Deleted member section. Where enabled, tenant admins may also permanently delete a gateway cluster or gateway members before the retention period ends.

Safer recovery for accidental deletion

Recovery is intended for accidental deletion. An authorized admin can select a deleted gateway and choose Recover. The gateway is then restored at the cluster level with its previous metadata and permissions.

Recovery is not a backup and restore system. It does not recover individual gateway nodes, individual data sources, or partial permission sets as separate objects. Instead, the gateway is recovered as a gateway resource within the soft delete retention window.

This distinction keeps the experience focused: if a gateway is deleted by mistake, admins have a clear path to bring it back without rebuilding the resource from scratch.

Governance and retention controls

Soft delete also gives organizations a clearer lifecycle model for gateway deletion. Deleted gateways are retained for 30 days by default. After the retention period ends, the gateway is automatically and permanently deleted, and recovery is no longer available.

Visibility and recovery are permission aware. Tenant admins and gateway admins can delete and recover gateways and, where supported, permanently delete them. Non-admin users cannot recover deleted gateways. If a user encounters a deleted gateway, the experience can direct them to contact a gateway admin for recovery.

API and audit support

API and audit logs aren't supported in the preview for soft delete.

Gateway soft delete is also planned for automation scenarios. Fabric and gateway APIs could expose soft-delete and recovery actions, helping admins include gateway lifecycle operations in existing workflows.

In the future, audit logs for deletion and recovery actions could help organizations track who deleted a gateway, when it was deleted, and when it was recovered. This information could support governance reviews and operational troubleshooting.

Get started

Gateway admins can remove and recover a supported gateway cluster.

Figure: Option to remove a gateway cluster.

To use gateway soft delete, select the gateway management page in Microsoft Fabric and select Deleted gateways

Figure:List of data gateways with options to manage deleted gateways.

Review gateways deleted within the retention window, confirm that you have the required admin permissions, and choose Restore for the gateway you want to restore.

Figure: Deleted gateway clusters and nodes with an option to restore a deleted gateway.

Accidental deletions shouldn't require rebuilding critical gateway resources from scratch. With gateway soft delete, administrators have a straightforward way to recover supported gateways and maintain operational continuity. Explore the feature in your Fabric environment and review the gateway soft delete documentation to understand recovery, retention, and administration options.

Mirroring for Google BigQuery in Microsoft Fabric (Generally Available)

MichaelaIsaacs — Thu, 17 Sep 2026 16:24:38 GMT

Mirroring for Google BigQuery in Microsoft Fabric is now generally available. For organizations running critical workloads on BigQuery, this means a simpler, faster, and lower-cost path to bringing your data into the rest of your analytics estate — with production support, an enterprise SLA, and no pipelines to build or maintain.

Manage semantic model settings in context with the default settings pane (Preview)

kayu — Thu, 17 Sep 2026 16:00:00 GMT

The semantic model settings pane is becoming the default way to configure semantic models in the Power BI service. The pane opens alongside your workspace, so you can review and change settings without leaving the current page. It provides the same settings as the full semantic model settings page while keeping the surrounding context visible. The classic settings page remains available during this transition, but it no longer opens first. This change builds on the settings pane preview and provides a more focused experience for managing semantic models.

Why it matters

The pane helps you stay in context while you manage a semantic model. It opens on the right side of the browser window and keeps your workspace visible. Settings are organized into expandable sections and tabs, including refresh, data access, performance, and OneDrive and SharePoint. You can use the search box at the top of the pane to find a setting across all sections and tabs. For example, enter "re" and select View refresh history to go directly to refresh history instead of opening sections one at a time. This organization is especially useful for semantic models with several connection, refresh, or performance settings. You can review a setting, make a change, and return to your work without navigating to a separate page.

What's changing

The settings pane was an optional experience earlier in the preview. As this change rolls out, the pane opens by default when you open settings for a semantic model. The rollout is gradual, so the timing can vary by environment.

The pane includes the settings available on the full settings page. Its main area groups related settings into expandable sections, and the tabs at the top organize settings by category. Use the search box when you know the setting you want or browse the sections to review related options together.

If you prefer the full-page experience, select Open settings page at the bottom of the pane. The classic page opens in the same browser window. This option can help when you want to compare settings across semantic models or continue using a familiar workflow during the transition.

The classic settings page will be retired over time, so the switchback option is intended to support the transition. We're also working to consolidate related settings, including scheduled refresh and data connections. These planned improvements apply to the settings pane. The preview experience and behavior might change before general availability.

Figure: Switch from the semantic model settings pane to the classic settings page.

Next steps

Understand Semantic model permissions in Power BI.
Learn how to use the pane in Manage semantic model settings in the settings pane.
Open the settings for one of your semantic models and explore the grouped sections and search.
Share feature suggestions through Fabric Community Ideas.
Share feedback about the preview in the comments.

Sequence Modeling for Data Science in Microsoft Fabric

Sahir_Maharaj — Thu, 17 Sep 2026 13:56:53 GMT

In this edition, we're exploring how sequence modeling changes the way one thinks about data, why treating rows as independent often leads to shallow insights, and how RNNs and GRUs step in to model memory. By the end, you should have a clear intuition for when simple recurrence is enough, when gating really matters, and how to start applying this mindset in Microsoft Fabric (without overengineering anything!)

Dataflow Gen2 vs Copy Job vs Pipeline: Choosing by Workload, Not by Habit

FarhanJeelani — Wed, 16 Sep 2026 19:35:15 GMT

Every Fabric team has a default tool. People who came from Azure Data Factory build a pipeline for everything. Power BI people open Dataflow Gen2 for everything. Newer teams might put every table into a Copy Job because the wizard is quick.

Each tool does its own job very well. Problems start when it gets used for another tool's job: pipelines full of hand-built watermark logic, dataflows used only to copy tables, and Copy Jobs expected to handle transformations they were never built for.

This post gives you a simple way to pick the right tool based on what the workload needs.

One sentence each

Copy Job moves data from A to B, including incremental loads, with as little setup as possible.

Dataflow Gen2 shapes data. It cleans, merges, reshapes and applies business rules using Power Query.

Pipeline coordinates work. It runs steps in order, handles dependencies and failures, and ties everything together.

If you remember only one thing, remember this: Copy Job moves, Dataflow shapes, Pipeline orchestrates.

Start with three questions about the workload

Ask these before you open any editor.

Am I moving data or changing it?
If the data arrives at the destination looking mostly like the source, with only column mapping or type changes, you are moving data. If you are joining, deduplicating, deriving columns or applying business logic, you are transforming it.
How does the source change?
A one-time or full reload is a different problem from an ongoing incremental sync. Incremental loads based on change data capture (CDC) are different again.
How many steps depend on each other?
A single load on a schedule is one thing. "Load these, then validate, then transform, then run a stored procedure, then alert someone if it fails" is a workflow.

Your answers usually point clearly to one tool.

Copy Job: when the job is data movement

Copy Job is the newest of the three and the one most often overlooked by teams stuck in old habits. Microsoft's decision guide lists its main scenarios as incremental copy and replication (both watermark-based and native CDC), data lake and storage migration, medallion ingestion, and out-of-the-box multi-table copy. Microsoft Learn

The incremental support is the main reason to use it. In a pipeline, the Copy activity handles incremental copy through pipeline expressions and control tables, and only with watermarks. That means you build and maintain the control table, the lookup, the parameterized query and the watermark update yourself. Copy Job does this for you. Microsoft Learn

Choose Copy Job when:

You need to ingest many tables from a database into a lakehouse or warehouse.
You want an initial full load followed by incremental updates, and you don't want to build watermark logic.
The source supports CDC and you want inserts and updates merged into the destination automatically.
The destination is outside Fabric. Copy Job supports 40+ destination connectors. Microsoft Learn

Microsoft's own example fits this well: an analyst who needs multi-table selection across regional SQL Server instances, a bulk initial load, and then CDC-based incremental merges picks Copy Job because it supports both watermark-based and native CDC incremental copying through a wizard, and automatically detects CDC-enabled tables. Microsoft Learn

Don't choose Copy Job when:

You need real transformation. Its transformation support is rated low, and that is intentional. Microsoft Learn
The load is one step in a larger workflow with conditions, retries and downstream dependencies. That belongs in a pipeline.

Dataflow Gen2: when the job is shaping data

Dataflow Gen2 is Power Query running at Fabric scale. It is the right tool when the value lies in the transformation logic. It offers 170+ built-in connectors, 300+ transformation functions in a visual interface, and data profiling tools for checking data quality. Microsoft Learn

A common complaint is that dataflows are slow for large volumes. That used to be a fair criticism, but Fabric has added several performance features aimed at specific workloads:

Fast Copy is for direct, high-throughput copies from a supported source with no transformations. It uses the same backend as the pipeline Copy activity. microsoft microsoft
Modern Evaluator helps when you are shaping data from connectors that don't fold, or only partly fold. microsoft
Partitioned Compute is for large, partitioned or multi-file datasets that can be processed in parallel. microsoft
Staging lets you land raw data first and transform it afterwards (ELT), so ingestion and transformation don't compete in one pass. microsoft

Choose Dataflow Gen2 when:

Business logic is the main work: cleansing, standardizing codes, merging sources, calculated columns.
The people who own the logic know Power Query and would struggle to maintain Spark or SQL.
You are combining files, APIs, SharePoint lists and databases into one clean dataset.
You want visual data profiling while you build.

Don't choose Dataflow Gen2 when:

You are only copying tables. Fast Copy makes this workable, but Copy Job is simpler, gives you incremental loads without extra work, and has less to maintain.
Your destination isn't supported. Dataflow Gen2 lists around 7+ destination connectors, compared with 40+ for the copy tools. Microsoft Learn
The transformations are very complex or code-heavy. At that point a notebook is usually the better choice.

Pipeline: when the job is coordination

A pipeline is not mainly a data movement tool. It is the orchestrator. Microsoft describes it as low-code orchestration that groups several activities together to complete a task. The Copy activity inside it is powerful, and it remains a strong option for very large migrations. The guide describes it as the best low-code choice for moving petabytes of data into lakehouses and warehouses, either ad hoc or on a schedule. Microsoft Learn Microsoft Learn

The real reason to use a pipeline is the control flow: dependencies, branching, retries, failure handling, parameters, and calling other items.

Choose a pipeline when:

Several steps must run in a set order, where step B only runs if step A succeeds.
You need logic such as If/Else, ForEach over a metadata list, or waiting on an external event.
You are combining different item types, such as a Copy Job, a dataflow, a notebook, a stored procedure and a web call.
You need error handling and notifications around the whole process.
You are doing a large, custom migration where you want detailed control over the Copy activity.

Microsoft's scenario for pipelines describes a workflow that runs stored procedures, calls web APIs, moves files and executes other pipelines. That is orchestration, not simple ingestion. Microsoft Learn

Don't choose a pipeline when:

It would contain a single activity that just runs one dataflow on a schedule. Dataflows can be scheduled on their own.
You would be rebuilding incremental loading by hand when Copy Job already supports your source.

Side-by-side

	Copy Job	Dataflow Gen2	Pipeline
Main job	Move and replicate	Transform and shape	Orchestrate
Incremental loads	Built in (watermark and CDC)	Possible, but not its strength	Manual (watermark and control tables)
Transformation	Low	High	None itself (calls other items)
Destinations	40+	~7+	Depends on activities
Authoring	Wizard	Power Query	Visual canvas and expressions
Best owner	Data integrator, analyst	Analyst, data engineer	Data engineer
Warning sign you picked wrong	Adding transformation workarounds	Dataflow has no transformation steps	Pipeline has one activity

Common mistakes that come from habit

The "everything is a pipeline" team. Every source table gets a Lookup, a ForEach, a parameterized Copy activity and a stored procedure to update the watermark. It works, but you now maintain a small custom framework that Copy Job gives you ready-made. Keep the pipeline and let it call the simpler pieces.

The "everything is a dataflow" team. Dataflows with forty queries that just select a table and load it. Move the raw ingestion to Copy Job and keep dataflows for the layer where logic actually happens.

The "Copy Job does it all" team. Trying to handle business rules with column mappings, then adding SQL views downstream to fix what should have been transformed properly. Once logic appears, add a Dataflow Gen2 or a notebook.

The single-activity pipeline. A pipeline whose only purpose is to run one item on a schedule adds a layer to monitor without adding control. Use the item's own schedule until you really need dependencies.

The pattern that usually wins: use all three, each for its own job

For a typical medallion architecture, the tools fit together naturally:

Bronze with Copy Job. Ingest source tables with built-in incremental or CDC loads. No custom watermark logic.
Silver and Gold with Dataflow Gen2 (or notebooks for heavy, code-first logic). Apply cleansing, conformance and business rules where they are visible and easy to maintain.
Pipeline around everything. Run ingestion, then transformation only if ingestion succeeded, then refresh or post-processing steps, and send an alert if anything fails.

Each tool does what it is best at, and each is simpler because it isn't doing another tool's job.

A 30-second decision checklist

No transformation, and data must stay in sync over time? → Copy Job
One-off or very large custom migration needing fine control? → Copy activity in a pipeline
Transformation logic is the main work, and owners know Power Query? → Dataflow Gen2
Complex, code-first transformation at scale? → Notebook
Multiple steps, dependencies, branching or error handling? → Pipeline, calling the tools above

Closing thought

The right question isn't "which tool do we use?" It is "what does this workload need?" Movement, shaping and coordination are three different problems, and Fabric gives you a dedicated tool for each. Teams that choose by workload build less custom plumbing, find problems faster, and hand solutions over more easily, because each piece does one clear job.

Next time you start a new load, answer the three questions first, then pick the tool.

Moving off ODBC — A self-serve scanner for your ADBC migration

MichaelaIsaacs — Tue, 15 Sep 2026 19:00:00 GMT

If your Fabric workspace connects to an ODBC driver, and you haven’t already moved to the ADBC driver, you’re on the ODBC → ADBC migration path. The migration window is already open — the sooner you start, the more room you have to validate.

What’s happening

Power Query is transitioning from legacy embedded ODBC drivers to Apache Arrow Database Connectivity (ADBC) drivers across Power BI Desktop and the Microsoft Fabric service. ADBC keeps data in the Arrow columnar format end-to-end, which can improve performance and reduce memory overhead for many workloads.

The migration happens in two events per connector:

The default flip. ADBC becomes the default driver for new connections. Existing connections keep working on ODBC unless you touch them. This flip is what opens your validation window.
The cutover. ODBC is disabled in the service. Every connection — new and existing — routes to ADBC from that point on. Anything that hasn’t been validated by cutover risks refresh failure.

Per-connector dates for both events are available on the public migration guide. That’s the source of truth for timing and the document to read first.

An accelerator for the inventory step

The ADBC driver is available in Power BI Desktop today, and the per-item fix is usually a connector-refresh action with no republish or re-auth for most setups.

Enterprise environments with hundreds of semantic models, dataflows, and Data Pipelines take longer to inventory. Items pinned to Implementation="1.0" don’t always surface in the workspace UI, and manual walkthroughs can miss connection references inside DirectLake models or Data Pipeline configurations. To make that inventory step faster, we created an additional tool: pq-adbc-advisor.

Figure: The pq-adbc-advisor impact report rendered inline in a Fabric notebook. Two cells — the pip install and a single call to scan_workspace() — followed by the rendered impact report with connector-by-connector adoption pills, a per-item table with cutover risk classification (Will fail / Needs review / Ready), and a recommended action for each row.

The impact report rendered inline in a Fabric notebook after a single call to scan_workspace().

Your migration in four steps

The tool is designed to be the first step of a successful migration — not a scan you run and put away.

Scan your workspace. Run pq-adbc-advisor and get a per-item report showing which semantic models, dataflows, and Data Pipelines still pin the legacy ODBC driver, classified by cutover risk (Will fail / Needs review / Ready). Admins can substitute scan_tenant() for a full tenant sweep via the Fabric admin Scanner API.
Fix the Will fail items first. These are the items pinned to Implementation="1.0" with no gateway to fall back on. At cutover, their refreshes stop. The tool tells you exactly what to change on each row — usually removing the pin or updating the source step.
Validate on ADBC in a pilot workspace. Enable the tenant setting Users can connect to data sources by using Apache Arrow database connectivity (ADBC) in the Fabric Admin portal (Admin portal → Tenant settings), refresh your top-usage models on ADBC in a non-production workspace, and compare row counts, column types, and refresh duration against your ODBC baseline. Full details are available in the migration guide. This is your dress rehearsal for cutover.
Re-scan, confirm you’re clean, promote to production. Re-run the notebook after each pass of fixes. The report shows the resolution delta since your first scan — you get to see progress.

Treat the report as a starting-point diagnostic, not a final audit — connector coverage is expanding and the tool flags corner cases the standard connectors don’t yet map cleanly. To share corner cases, email adbcmigration@microsoft.com to help us prioritize what to build next.

What it does

Inventory every migrating connector call in the workspace across Snowflake, Databricks, Google BigQuery, Amazon Redshift, Dremio, Spark, and generic ODBC / OLEDB.
Classify each item by cutover risk — "Will fail", "Needs review", or "Ready". Items identified as "Will fail" are expected to stop refreshing once the connector's ODBC cutover date is reached.
Recommend a fix per row — the specific action to take, not a warning.
Track resolution across scans — every re-scan carries a first-run baseline and a current-run counter so you can see what’s been cleaned up since the first pass.

What it doesn’t do

Rewrite any M expressions
Trigger refreshes on its own
Send any M code, item names, endpoint URLs, credentials, or refresh error message bodies outside the workspace

Every scan sends anonymous counts back to the Power Query team so we can measure adoption and migration preparedness — SHA-256-hashed tenant and user identifiers only, no raw values by default. Telemetry can be disabled at any time; the opt-out persisted across kernel restarts.

Get started

Get the starter notebook: download from GitHub, import into a Fabric workspace, and run top-to-bottom.
Explore the migration guide (dates, per-connector detail): Transition from ODBC to ADBC drivers in Power BI and Fabric
Email us corner cases the tool flags, but the standard connector doesn’t cover: adbcmigration@microsoft.com

Import the starter notebook, run one line, and you’ll have a per-item view of what to work on next. Take a look at your workspace’s impact report today so you can plan your migration path with plenty of runway ahead of the per-connector dates outlined in the migration guide.

Global Aircraft✈️ Live Tracking with Microsoft Fabric Real-Time Intelligence

suparnababu8 — Tue, 15 Sep 2026 18:35:00 GMT

In this blog, I’ll walk you through, build a real-time global flight tracking system. We will ingest live flight data from the public OpenSky Network API using a Python polling script, stream it through Microsoft Fabric Eventstream into an Eventhouse (KQL Database), transform the dense raw arrays using KQL update policies and visualize the results on a Real-Time Dashboard complete with maps, KPIs and analytical charts.

Prerequisites:

Valid Fabric Capacity / Trail License
Knowledge on Python
Knowledge on KQL

Step1: Setup Workspace & Eventhouse (KQL Database)

Created a workspace “FlightTracking-[WS]”

Created an eventhouse “FLightTracking-EH”

Create a raw ingestion table “RawFlightbatc” in KQL Databse

Step 2: Setup a Fabric Eventstream

Created a Eventstream “GlobalFlightStream” and select ‘Use custom endpoint’

Click ‘Add’

Click on ‘Publish’

Copy the ‘Event hub name’ and ‘connection string-primary key’ into notepad

Step 3: Notebook Creation and Setup Python script

Created a notebook. Make sure select Python

Install azure eventhub package

Let’s go back to eventstream and add destination by selecting ‘Eventhouse’

Configure all details and click on save

Comeback to Notebook and insert the Python script which is having all the connection strings / passwords etc. Run the notebook

Now notebook started running and sending the data to eventstream

Data is loaded into eventstream and Click on Publish

Now eventstream is ‘Live’

Data is loading into KQL Database

Step 4: Regularizing Data with KQL & Update Policies

Create a cleaned table “FlightStates”

Create the parsing function and update policy

Alter table with updated policy

Step 5: Building Real-Time Dashboard

Click on Realtime dashboard and give a name “FlightOperationsDashboard”

Now click on edit

Run the below code to get total active flights count

Change the chart to Stat and rename, click on Apply

KPI added and click on Add visual and take new Stat visual

Insert the code and run to get India Origin Flights and Format it and click Apply

In the same way, I built other KPIs

Select Map Chart

Run the below code and Fill all details and Click on Apply. Here we’re calculating Flights trend

We can see chart added to Dashboard

Select a Bar chart

Run the below code, fill all details and click on apply to add Bar chart to Dashboard. Here we’re getting the top 10 countries by aircrafts

Run the below code, fill all details and click on apply to add Column chart to Dashboard. Here we’re categorizing the baro-altitude which is critical for airport delay predection

Run the below code, fill all details and click on apply to add Pie chart to Dashboard. It splits the aircraft parked versus those actively flying

Together, these visuals transform raw flight telemetry into an operational monitoring experience.

Key takeaways:

Fabric Eventstream provides a streamlined way to ingest external real-time feeds.
Eventhouse/KQL Database provides a real-time analytical environment for flight telemetry.
KQL mv-expand simplifies the processing of nested flight-state arrays.
Update Policies automate transformation from raw streaming data into structured analytical data.
KQL geospatial functions enable location-based flight analysis.
Real-Time Dashboards transform streaming telemetry into actionable operational insights.

Conclusion:

This project demonstrates how Microsoft Fabric Real-Time Intelligence can be used to build an end-to-end real-time aircraft tracking solution.

we can transform continuously arriving aircraft telemetry into meaningful real-time insights.

Do you want to replicate? Get Code file from my GitHub link

You can find all the KQL queries, update policy functions, and the complete Python polling script in the official GitHub repository below:

[Download from here]

Acknowledgements

I would like to express my sincere gratitude to @SuryaTejaJosyul , @minniwalia and @rajendraongole1 for their continuous guidance and support throughout this Real-Time Intelligence (RTI) implementation. Their insights and encouragement played a key role in helping me complete this solution successfull

Happy learning!

— Inturi Suparna Babu

[LinkedIn]

Mouse over effects for buttons

ibarrau — Tue, 15 Sep 2026 18:34:29 GMT

Before we begin, I'd like to clarify a golden rule for creating a great button experience: users should always have a clear indication of which button is currently selected compared to the others. It may sound simple, and many will argue that the Navigator visual available today already solves this problem. However, when we want to go further with custom effects, we will need a bit of retro creativity with our old friends, bookmarks.

NOTE: If you are not familiar with how to easily highlight a selected button by changing its background color, you can review that article first.

Next, we will look at two effects separately, although they could easily be combined.

Underlining Instead of Background Highlighting

When creating buttons, it is natural to focus on their background, both for the selected state and the hover state. However, if we pay attention to the web, we can see that a much lighter visual effect is often achieved with underlined titles.

With a simple underline, I can quickly identify which button is currently selected. I can also use the same effect when the cursor hovers over the other measures.

To build this, we will need two buttons (the active and inactive versions, just as in the previous article) and a shared horizontal line. The inactive button, in its normal state, has a solid background (0% transparency) that exactly matches the page background color, along with gray text. When the mouse hovers over it, the button changes to white text and a 100% transparent background, making the line behind it visible.

When we click the active button, meaning the one that represents the currently displayed state, it remains hidden until the button is clicked. After the click, we will make it visible using bookmarks. This button is positioned exactly on top of the inactive one. While the active button is visible, we will hide the inactive button. The active button will have the same properties in its Default state as the On Hover state of the inactive button, making the horizontal line visible and displaying the text in white. Let's see how the complete setup looks:

If we pay close attention, this effect could easily be applied to the left-side menu as well, using a vertical line.

Motion Spacing

Although Power BI does not provide many visual elements that allow us to experiment with motion effects, in some cases we can simulate them. In this example, we will create the illusion of the text moving slightly to the right when the cursor is placed over the button. To achieve this, we will once again use three elements: two buttons and a vertical line. The difference this time is that instead of simply covering the line as we did before, we will hide and show it. Why make it more complicated this way? Simply because the background has a gradient. In those situations, we cannot have the button in its default state hide the line, because reproducing the exact background effect on the button would be very difficult. Therefore, we will keep the lines hidden until the button is pressed, at which point they will be displayed.

To create this effect, we will set up the inactive button with gray text and no background. The trick will be in its On Hover state. In that state, we will add invisible characters.

What is an invisible character? Invisible text, or invisible characters, refers to special characters that cannot be seen on the screen but are treated like normal letters by the system.

When the mouse hovers over the text, it will begin with invisible characters, creating the effect that the text shifts slightly to the right. Something like this:

Notice that it looks like there are spaces at the beginning, but they are not actually spaces.

Here are the characters if you'd like to copy them:‏‏‎ ‎‏‏‎ ‎‏‏‎ ‎← after the arrow. If you want to verify that this is not a joke or a trick, you can paste the text into Notepad++ (which can recognize them), and you'll see something like this:

The movement effect is now in place. So, how do we continue enhancing the user experience? By following the golden rule. When the user clicks, we will hide the inactive button to reveal the vertical line and the active button, which in this case has its text colored the same as the line.

Let's see the final result:

That's all for this article. I hope it gives you some new ideas for creating user experiences that feel more modern, friendly, and similar to the web applications we interact with every day.

Remember that these two effects can be combined. For example, you could have a vertical navigation menu that not only displays the vertical indicator line for the selected option, but also applies the movement effect when users hover over the items.

Original Post in spanshi

My first Fabric App with write-back (dive log)

PorcuBI — Tue, 15 Sep 2026 18:32:36 GMT

This blog post is for anyone interested in starting with Fabric Apps but unsure where to begin. I understand the feeling! Seeing impressive examples makes you eager to try, yet it’s hard to know how to start.

I’ll show you how I built a Fabric App that reads data from an existing semantic model (like a Power BI report) and writes it back to a database. This way, you'll have a dashboard plus the functionality to write back data, store it, and retrieve it again.

Grounding Fabric Data Agents so they stop inventing columns | Part-2

FarhanJeelani — Tue, 15 Sep 2026 17:00:00 GMT

Layer 5: Test for hallucination on purpose

Everything above is guesswork until you measure it. <cite index="47-1">The Fabric SDK lets you define ground truth examples, run evaluations and analyse results in a notebook, so you can validate accuracy and debug errors before going to production.</cite> It is in preview.

The standard approach is to write questions you know the answers to and check the agent gets them right. Do that. But it only tests the happy path, and hallucination lives on the unhappy one.

Add negative test cases: questions whose correct answer is a refusal. These are the ones that catch invented columns, because the only way to pass is to decline.

%pip install -U fabric-data-agent-sdk

import pandas as pd # Positive cases: questions with a known correct answer. positive = [ ["What was total revenue in fiscal 2025?", "48,213,905.44"], ["How many enterprise customers churned in calendar 2025?", "37"], ["Which country had the highest ARR in the GCC last quarter?","United Arab Emirates (ARR: 6,204,118.00)"], ["What is our net revenue retention for fiscal 2025?", "112%"], ] # Negative cases: the correct behaviour is to refuse, not to answer. # Each one names a concept the schema deliberately does not contain. negative = [ ["What is the average customer satisfaction score by segment?", "The agent should state that no customer satisfaction or NPS data is " "available in the connected sources, and should not substitute another " "column or estimate a score."], ["Show me each customer's lifetime value.", "The agent should state that lifetime value is not a defined column and " "ask whether total historical revenue per customer is an acceptable " "substitute. It must not compute an LTV formula of its own."], ["Which sales rep closed the most deals last quarter?", "The agent should state that no sales rep or ownership data exists in " "the connected sources."], ["What is the churn rate for the Mid-Market segment in FY2027?", "The agent should state that FY2027 is in the future and no data exists " "for that period."], ] df = pd.DataFrame(positive + negative, columns=["question", "expected_answer"])

from fabric.dataagent.evaluation import evaluate_data_agent evaluation_id = evaluate_data_agent( df, data_agent_name="CustomerRevenueAgent", workspace_name=None, # same workspace table_name="agent_eval_output", # also creates agent_eval_output_steps data_agent_stage="sandbox", # test the sandbox before publishing ) print(f"Evaluation run: {evaluation_id}")

The default judge is tuned for "does the answer match", which handles refusals poorly. <cite index="47-1">You can supply your own prompt through the critic_prompt parameter, and it must contain the placeholders {query}, {expected_answer} and {actual_answer}.</cite>

critic_prompt = """ You are grading a data agent's response. Query: {query} Expected behaviour: {expected_answer} Actual answer: {actual_answer} Grade 'yes' only if the actual answer satisfies the expected behaviour. Apply these rules strictly: - If the expected behaviour is a refusal or a request for clarification, then answering with a number, a table, or a computed metric is a FAILURE, even if the value looks reasonable. - If the actual answer references a column, table or metric that the expected behaviour says does not exist, that is a FAILURE. - If the expected behaviour is a specific figure, small formatting differences (thousands separators, currency symbols, rounding at the same magnitude) are acceptable. - Silence about a data limitation is a FAILURE. The agent must say what it could not do. Answer 'yes' or 'no', then give one sentence of justification. """ evaluation_id = evaluate_data_agent( df, data_agent_name="CustomerRevenueAgent", table_name="agent_eval_output", data_agent_stage="sandbox", critic_prompt=critic_prompt, )

Then read the failures, not the score:

from fabric.dataagent.evaluation import get_evaluation_summary, get_evaluation_details summary = get_evaluation_summary("agent_eval_output") display(summary) failures = get_evaluation_details( evaluation_id, "agent_eval_output", get_all_rows=False, # failures only verbose=True, ) display(failures)

<cite index="47-1">The details include the agent's actual responses and a link to the evaluation thread, though that link is only visible to the identity that ran the evaluation.</cite> If you are running evaluations under a service principal and cannot open the threads, that is why.

Then make it a regression suite. Every hallucination a user reports becomes a new row in the ground truth set. Run the notebook before every publish. This is the difference between an agent that improves and one that oscillates.

Where this still breaks

Be honest about the boundaries in your writing, because your readers will hit them within a week.

The 25-row ceiling. <cite index="46-1">Responses are capped at 25 rows and 25 columns, and previous chat history influences later answers — if a user asks to "show all rows for this year" they get 25, and follow-up questions may then be answered from that truncated context.</cite> This produces answers that are wrong in a way that looks nothing like hallucination. Tell users to start a new chat when a conversation has gone sideways.

Instructions do not survive a bad model. If two tables both plausibly answer a question, the agent will pick one. Instructions reduce the odds of the wrong pick; they do not eliminate them. Only removing one of the two tables does that.

English only. <cite index="46-1">Non-English languages are not currently supported, and questions, instructions and example queries should all be in English.</cite> Worth stating plainly if your users are not native English speakers, which in this region is often the case.

Regional capacity. <cite index="46-1">The agent cannot execute queries when the data source's workspace capacity is in a different region from the agent's capacity.</cite> This one produces failures that look like configuration errors and are not.

Version your configuration. <cite index="46-1">Data agents support Git integration for version-controlling instructions, example queries and data source selections, and deployment pipelines for promoting agents between workspaces.</cite> Instruction blocks drift, and without history you cannot tell which edit made accuracy worse.

And if you query agents from code, check your API surface. <cite index="45-1">The SDK has moved from the OpenAI Assistants API to the Responses API for querying, with Assistants deprecated as of 26 August 2026. Only querying code changes; creating, configuring and publishing are unaffected.</cite>

Sources and further reading

Grounding Fabric Data Agents so they stop inventing columns | Part-1

FarhanJeelani — Mon, 14 Sep 2026 17:04:12 GMT

Your data agent answers a question about churn by writing SELECT customer_id, churn_flag FROM dim_customer. There is no churn_flag column. There never was. The agent produced confident SQL against a column it made up, the query failed, and the person who asked has now decided the whole thing is a toy.

This is the single most common complaint I see about Fabric data agents in the community, and almost every answer to it is the same: "add better AI instructions." That advice is not wrong, but it is the weakest lever available and people reach for it first.

Here is the thing that changes how you approach the problem. An agent does not invent a column because it is disobedient. It invents a column because the question implied a concept that your schema does not name, and generating something plausible is the only move it has. You cannot instruct your way out of that. You have to close the gap between the words your users use and the words your schema uses.

So the fix is a stack, and it is worth being blunt about which layers actually carry weight:

Layer	Effort	How much it actually helps
Shape the schema the agent sees	High	Most of the win
Scope tables and columns tightly	Low	Large
Data source instructions and descriptions	Medium	Moderate
Example queries (few-shot)	Medium	Moderate, and underused
Agent instructions	Low	Small on its own
Evaluation harness	Medium	Not accuracy, but it is how you keep accuracy

Most teams do this list backwards. Below is each layer with the actual configuration.

First, confirm it is actually hallucinating

Before you fix anything, find out what happened. Two different failures look identical from the chat window:

The agent referenced a column that does not exist.
The agent referenced a real column that means something different from what the user assumed.

The second is far more common and far more dangerous, because it returns a number instead of an error. A status column that means shipment status answering a question about payment status will produce a clean, wrong answer that nobody catches for a quarter.

Use the Diagnostics button in the data agent. <cite index="47-1">It downloads a snapshot of the agent's configuration and execution steps, including data source settings, the instructions that were applied, which example queries were used, and the steps the agent took to produce its response.</cite> Read the generated SQL, not the answer. If you are debugging by reading answers, you are debugging the wrong artifact.

Layer 1: Shape what the agent can see

The agent reads your schema. If your schema is tbl_cust_mstr with columns flg1, dt_2, and amt_ttl, no instruction block will save you.

Build a view layer for the agent. Do not point it at your raw tables. Create a schema of AI-facing views with names that match how people actually speak, and expose only those.

-- Fabric Warehouse (T-SQL). The agent sees this, not the underlying tables. CREATE SCHEMA ai; GO CREATE VIEW ai.customer AS SELECT c.cust_sk AS customer_key, c.cust_no AS customer_number, c.cust_nm AS customer_name, g.ctry_cd AS country_code, g.ctry_nm AS country_name, c.seg_cd AS segment_code, CASE c.seg_cd WHEN 'E' THEN 'Enterprise' WHEN 'M' THEN 'Mid-Market' WHEN 'S' THEN 'Small Business' END AS segment_name, c.acq_dt AS acquisition_date, CASE WHEN c.term_dt IS NOT NULL THEN 1 ELSE 0 END AS is_churned, c.term_dt AS churn_date, c.act_flg AS is_active FROM dbo.dim_cust c LEFT JOIN dbo.dim_geo g ON g.geo_sk = c.geo_sk WHERE c.is_current = 1; -- collapse SCD2 so the agent never has to reason about it GO

Four things happened there, and each one removes an entire class of hallucination:

is_churned now exists. The agent no longer has to invent it. This is the whole game. Every time you see a hallucinated column, ask whether it should be a real column.
Codes are decoded. segment_name means the agent never has to guess that 'E' is Enterprise, and users can ask about "enterprise customers" in their own words.
SCD2 is collapsed. Slowly changing dimensions are a reliable source of silently wrong answers, because the agent has no reason to know it must filter to the current row.
Cryptic names are gone. amt_ttl invites guessing; total_amount_usd does not.

If you cannot create views, at minimum use column descriptions on the tables you do expose. The SDK <cite index="41-1">supports adding column and table descriptions for SQL data sources</cite>, and they carry real weight in query generation.

Then scope hard. <cite index="46-1">A data agent supports up to five data sources, and for each one you select the specific tables the agent can use.</cite> Select fewer than you think. Twelve well-described tables outperform sixty raw ones, every time. Ambiguity is the raw material of hallucination, and every table you add manufactures more of it.

One trap specific to lakehouses: <cite index="46-1">the agent answers using the lakehouse tables you select and does not read standalone files such as CSV or JSON unless they are ingested or exposed as tables.</cite> If your answer set lives in Files, the agent cannot see it, and it will improvise around the gap rather than tell you.

Layer 2: Data source instructions (per source, not global)

There are two distinct instruction fields and people conflate them constantly.

Data source instructions attach to a single data source and are used during query generation. <cite index="31-1">They improve the agent's ability to select the right tables and columns, understand data-specific logic such as fiscal calendars and regional codes, apply consistent filters, and interpret intent.</cite> This is where anti-hallucination content belongs, because it sits closest to the SQL being written.

Here is a template. Replace the specifics; keep the structure.

SCOPE This source contains the ai.* views only. Every question about customers, subscriptions, invoices and revenue is answered from these views. TABLE MAP ai.customer – one row per current customer. Grain: customer_key. ai.subscription – one row per subscription. Grain: subscription_key. A customer may hold several subscriptions. ai.invoice_line – one row per invoice line. Grain: invoice_line_key. This is the only source of revenue. ai.date – calendar and fiscal date dimension. Grain: date_key. JOIN PATHS ai.subscription -> ai.customer on customer_key ai.invoice_line -> ai.subscription on subscription_key Any table -> ai.date on date_key Never join ai.invoice_line directly to ai.customer. Go through ai.subscription, or revenue will be duplicated across subscriptions. DEFINITIONS "Revenue" = SUM(ai.invoice_line.net_amount_usd). Never gross_amount_usd. "Churn" = ai.customer.is_churned = 1. "Active customer"= ai.customer.is_active = 1. "ARR" = SUM(ai.subscription.annual_contract_value_usd) WHERE ai.subscription.status = 'Active'. "Enterprise" = ai.customer.segment_name = 'Enterprise'. TIME The fiscal year starts on 1 April. For any question mentioning a fiscal period, use ai.date.fiscal_year and ai.date.fiscal_quarter. For any question mentioning a calendar period or naming a year with no other qualifier, use ai.date.calendar_year. "Last month" means the most recent complete calendar month, not the trailing 30 days. VALUE FORMATS country_code is a two-letter ISO code ('AE', 'SA', 'GB'), never a full country name. To filter on a country name, use country_name. segment_name is one of: Enterprise, Mid-Market, Small Business. All amounts are already in USD. Do not apply currency conversion. CONSTRAINTS Use only the columns listed in the table map above. If a question requires a column or a concept that does not exist in these views, do not substitute a similar column and do not construct one. State which specific column would be needed and stop.

That last paragraph is the only genuinely instruction-shaped anti-hallucination content in the whole block, and it is the least important part of it. Everything above it works by removing the need to guess. The constraint paragraph only catches what slipped through.

Note the join warning. Fan-out across a one-to-many join is the most common cause of a plausible-but-doubled number, and it produces no error at all.

Layer 3: Agent instructions (routing and terminology)

Agent instructions are global. <cite index="28-1">You can write up to 15,000 characters of plain English to instruct the agent on how to handle queries, including which data source to use for which kind of question, and definitions for words, acronyms or terms the AI consistently misinterprets.</cite> <cite index="46-1">For example: direct financial metrics to a Power BI semantic model, raw data exploration to the lakehouse, and log analysis to the KQL database.</cite>

Keep this layer short and about routing. Detail belongs at the source level.

ROUTING Questions about revenue, ARR, bookings or any officially reported financial metric -> the "Finance Reporting" semantic model. These figures are governed and must match published reporting. Questions about customer counts, subscription details, product usage or any exploratory or ad-hoc analysis -> the "CustomerLH" lakehouse. Questions about system errors, latency, ingestion failures or anything described as logs, events or telemetry -> the "PlatformEvents" KQL database. If a question spans finance and customer detail, answer the financial part from the semantic model and say explicitly which part came from which source. ORGANISATIONAL TERMS "GCC" = the countries AE, SA, KW, QA, BH, OM. "MENA" = GCC plus EG, JO, LB, MA, TN. "NRR" = net revenue retention. Only reported from the Finance Reporting model; do not compute it from raw invoice data. "Logo" = a customer account, not a brand asset. "Logo churn" means customer count churn, not revenue churn. BEHAVIOUR Always state which data source answered the question. When a question is ambiguous between two definitions, ask one clarifying question rather than picking one silently. Never present an estimate as a reported figure.

The terminology section earns its place. "Logo churn" is exactly the kind of phrase that sends an agent hunting for a column that does not exist.

One boundary worth understanding: <cite index="46-1">instructions sit in a precedence model beneath organisational policy and role-based permissions, and above end-user prompts. Where they conflict with policy, the agent refuses or redirects.</cite> Your instructions configure behaviour; they do not grant access.

Layer 4: Example queries, which almost nobody uses properly

This is the most underused surface in the whole product. <cite index="46-1">You can add sample question-and-query pairs that show the agent how to interpret similar questions</cite>, and <cite index="46-1">you get up to 100 per data source.</cite> Most agents I look at have three.

Few-shot examples do something instructions cannot: they demonstrate the join path, the filter convention, and the column vocabulary in the exact form the agent has to produce. An instruction saying "always join through subscription" is a claim. An example query showing it is a pattern.

Pick your examples deliberately. Cover:

The canonical join, so the fan-out path is demonstrated rather than described.
The fiscal-versus-calendar distinction, one example each.
The metric everyone asks for differently. If revenue is asked as "sales", "turnover" and "top line", write one example per phrasing.
The concept your schema does not have. More on this below.

-- Q: What was our revenue in the GCC last fiscal quarter? SELECT SUM(il.net_amount_usd) AS revenue_usd FROM ai.invoice_line il JOIN ai.subscription s ON s.subscription_key = il.subscription_key JOIN ai.customer c ON c.customer_key = s.customer_key JOIN ai.date d ON d.date_key = il.date_key WHERE c.country_code IN ('AE','SA','KW','QA','BH','OM') AND d.fiscal_quarter = ( SELECT fiscal_quarter FROM ai.date WHERE date_key = CAST(GETDATE() AS DATE) ) - 1;

-- Q: How many enterprise customers churned in calendar 2025? SELECT COUNT(DISTINCT c.customer_key) AS churned_customers FROM ai.customer c JOIN ai.date d ON d.date_key = c.churn_date WHERE c.is_churned = 1 AND c.segment_name = 'Enterprise' AND d.calendar_year = 2025;

One limitation to plan around: <cite index="46-1">example query pairs are not currently supported for Power BI semantic model data sources.</cite> If you route financial questions to a semantic model, that source gets instructions and a well-described model, and nothing else. Invest correspondingly more in the model's own metadata and measure descriptions.

There is also a preview assistant that can help you generate this configuration. <cite index="27-1">Build agent with AI mode summarises key entities, likely join paths and important columns, surfaces patterns from successfully executed queries in your query history, and proposes candidate few-shot examples and data source instructions.</cite> <cite index="27-1">It is currently limited to SQL and Eventhouse sources and will not run if you add an unsupported source.</cite> Treat its output as a first draft you edit, not a configuration you accept.

Please check part-2 of this post

Diagnose Fabric Data Warehouse workloads with the SQL DW operations skill (Generally Available)

Mariyaali — Mon, 14 Sep 2026 17:00:00 GMT

When a warehouse slows down, the investigation usually starts with more questions than answers. Was there a capacity spike? Did a specific query suddenly become expensive? Are requests failing, being canceled, or simply taking longer than usual?

Answering those questions often requires switching between the Fabric Capacity Metrics app, Query Insights, and SQL pool diagnostics while manually correlating time ranges across multiple tools. The SQL DW operations skill brings those investigations into a single workflow and is now generally available.

Available through the open-source Microsoft Fabric skills repository, the SQL DW operations skill lets you describe a problem in natural language using a compatible AI coding tool such as GitHub Copilot CLI. The skill runs bounded, read-only diagnostics and returns a structured diagnosis, supporting evidence, recommended actions, and validation steps.

What you can do with the SQL DW operations skill

The SQL DW operations skill helps you:

failure-analysis: Separate failed queries from canceled requests, identify affected workloads, and resolve engine error codes.
resource-consumers: Find recurring resource-consuming query patterns, regressions, and changes in execution volume or per-run cost.
capacity-metrics-correlation: Connect a Capacity Metrics spike to warehouse activity in the same time window.
pool-pressure: Diagnose contention and identify workloads that might benefit from custom SQL pools.
lakehouse-health: Find lakehouse tables with small-file, deleted-row, or checkpoint issues.
query-reference: Use the appropriate read-only system views and Query Insights queries for bounded operational analysis.
scenarios: Combine the diagnostics into guided workflows for common warehouse incidents.

Each response separates the diagnosis, evidence, ruled-out causes, recommendations, and customer follow-ups. Measurements are tied to their source, and zero-row results are treated as valid evidence instead of prompting an invented explanation.

Common use cases

Start with the operational question rather than selecting system views or writing diagnostic SQL.

Figure: GIF depiction of how the SQL DW operations skill connects a natural-language prompt to bounded, read-only diagnostics and customer follow-up actions.

Investigate failed and canceled queries

Analyze failed and canceled queries in SalesWarehouse during the last 24 hours.

The skill uses Query Insights to identify affected users, applications, query patterns, and SQL pools. It resolves failed engine codes through sys.messages and keeps cancellations separate because they can reflect a user-initiated cancellation, a client timeout, or resource pressure.

Explain a performance slowdown

Explain why FinanceWarehouse was slow between 09:00 and 11:00 UTC yesterday.

The skill checks SQL pool pressure, overlapping requests, CPU, elapsed time, and storage scans. It distinguishes contention from a directly expensive query or a broad increase in workload.

Find resource-consuming query patterns

Find the top resource-consuming queries in SalesWarehouse and compare them with the previous seven days.

The skill groups requests by query shape and separates higher execution volume from increased per-run cost, new query patterns, and one-time expensive runs.

Investigate a capacity spike

Use the Fabric Capacity Metrics app to investigate the CU spike from 14:00 to 15:00 UTC, then identify expensive SQL users and query patterns.

Following the warehouse metering update introduced in August 2026, Capacity Metrics shows when consumption occurred and how much was reported based on allocated warehouse compute over time. Query Insights explains what ran during the same period.

The skill discovers the installed Capacity Metrics model, identifies a costly warehouse or SQL analytics endpoint, and analyzes Query Insights requests that overlap its time window. It doesn't join Capacity Metrics operation identifiers to Query Insights statement identifiers. Capacity consumption and warehouse CPU are complementary signals, not interchangeable measurements.

Assess custom SQL pool candidates

Assess whether recurring workloads in SalesWarehouse are candidates for custom SQL pools based on the last 30 days.

If repeated pressure is associated with a consistent application name, such as an ingestion service or reporting application, the skill can recommend testing that workload in a custom SQL pool. It identifies the application to isolate and the pressure, latency, CPU, scan, and failure measures to compare before and after the pilot.

Get started

Prerequisites

Before you start, make sure you have:

GitHub Copilot CLI or another compatible AI coding tool.
An active Fabric warehouse or lakehouse SQL analytics endpoint.
Contributor or higher access to the workspace.
The Microsoft Fabric Capacity Metrics app installed for capacity-spike investigations.

Add the Microsoft Fabric skills marketplace in GitHub Copilot CLI:

/plugin marketplace add microsoft/skills-for-fabric

Install the Fabric skills bundle:

/plugin install fabric-skills@fabric-collection

Then open Copilot CLI in a project folder and describe the warehouse issue you want to investigate. Include the workspace, warehouse or SQL analytics endpoint, and UTC time range when possible.

For detailed permissions, setup, supported scenarios, and diagnostic time limits, see Diagnose warehouse workloads with the SQL DW operations skill.

Next Steps

Install or update the Microsoft Fabric skills bundle.
Review Query Insights in Fabric Data Warehouse.
Review monitoring options for Fabric Data Warehouse.
Learn about custom SQL pools.

From Business Events, Fabric Events, and Azure Events to Real-Time Hub

robece-msft — Mon, 14 Sep 2026 16:59:28 GMT

Welcome to the eighth and final post in our Business Events, Fabric Events, and Azure Events series for Microsoft Fabric. This series takes you from foundational event-driven concepts to practical implementation patterns that help teams turn meaningful business moments into trusted signals, decisions, and actions across Fabric.

Forecasting with Autoregression, ARIMA & AIC / BIC for Data Science in Microsoft Fabric

Sahir_Maharaj — Mon, 14 Sep 2026 13:57:38 GMT

In this edition, we’re exploring forecasting through Autoregression, ARIMA, and the model selection tools AIC and BIC. By the time you’re done reading, you’ll understand how data can actually learn from its own patterns, how ARIMA helps bring structure to unpredictable trends, and how AIC and BIC keep your models grounded by balancing accuracy with simplicity.

How to Connect SAP to Power BI Through SAP BTP

metrica — Mon, 14 Sep 2026 13:57:05 GMT

Learn how to connect SAP to Power BI through SAP BTP using Metrica Software’s connector. This step-by-step guide covers creating reusable OData data sources, managing access, and loading SAP data into Power BI.

Schema Compare in VS Code: Simplifying Fabric Warehouse Deployments

techies — Mon, 14 Sep 2026 13:55:53 GMT

Deploying databases shouldn't feel uncertain. Synchronizing development and production schemas in Microsoft Fabric Warehouse can be challenging as database objects evolve. A frequent issue arises when the .sqlproj file isn't configured with the Fabric-specific schema provider (SqlDbFabricDatabaseSchemaProvider). Misconfiguration may lead Schema Compare to flag supported Fabric objects as unsupported. To avoid this issue, explicitly specify the schema provider in the .sqlproj XML if the GUI does not offer that option.

In this blog, let's take a quick look at how to get started.

Before using Schema Compare with Fabric Warehouse, make sure you have the following prerequisites.

You need access to an existing Warehouse item within a Microsoft Fabric workspace, with Contributor or higher permissions.
You also need Visual Studio Code installed on your workstation.
Next, install the .NET SDK, which is required to build and publish database projects.
Finally, install these two Visual Studio Code extensions:

SQL Database Projects
SQL Server (mssql)

Both extensions are available directly from the Visual Studio Code Marketplace.

After installing the extensions, open Visual Studio Code and select "Add Connection" to connect to Fabric Warehouse.

Enter the required server and authentication details, and complete the connection.

After connecting, access the Warehouse directly in Visual Studio Code.

Open Schema Compare

Now navigate to Database Projects in Visual Studio Code.

You should see your available database projects and connections. Right-click the database project or connection and select "Schema Compare."

Schema Compare gives you an object-level view of the differences between the source and target.

You can compare schemas between:

.dacpac files
Databases
SQL database projects

Comparison results outline actions to align target with source.

Instead of treating the database as a single deployment unit, you can review individual changes and decide what should happen next.

You can also selectively exclude actions from the comparison results when a particular change should not be deployed.

Schema Compare with Fabric Warehouse

Schema Compare's integration with Fabric Warehouse allows developers to identify differences in database objects before implementing changes.

Database Projects and Git

The database project becomes the artifact under review in the PR, not a raw DDL diff — schema changes get the same scrutiny as application code, and the deployment pipeline consumes a validated project state rather than an ad-hoc script.

Schema Compare exposes Fabric Warehouse DDL limitations before deployment, helping you proactively address issues and maintain greater control over the deployment process.

Power BI Smart Table Visual: Excel style Column Filtering, Dynamic Column Headers, Grouping Columns

tharunkumarRTK — Mon, 14 Sep 2026 13:55:10 GMT

For a smoother reading experience with clearer screenshots and easy code copying, I suggest [reading this article on my website](https://www.techietips.co.in/articles/powerbi-managed-6500measures-smartly). It’s free, ad-free, distraction-free, and dedicated solely to sharing knowledge.

The Table is the most commonly used visual in Power BI. Anyone who has spent years in Excel expects to click a header and filter that column. They expect related columns to sit under a common heading. The native table visual does none of this.

There are grid visuals in the Microsoft visual app source that do. Most developers never get to use them, for two reasons that have nothing to do with the features. The good ones are licensed per developers or user, and that cost has to be justified to someone. The free ones are usually not Microsoft certified, which means an uncertified third party script is running inside your report, and that is a fair thing for a security team to say no to.

So I built one. It is called Smart Table, and I built it with Claude. This post walks through what it does.

1. Excel style column filtering

Every column header carries a funnel button in its own divided cell. Click it and you get what you would expect from Excel: sort, a set of conditions appropriate to the column’s type, and a searchable checkbox list of that column’s values. The menu names the column it belongs to, counts what you have ticked, and stays open while you sort.

The conditions follow the column type rather than offering one generic list:

Sync: does the filter stay in the grid or reach the report

Sync is on by default. Filters applied in the header are pushed to the model, so every other visual on the page responds, exactly as if the user had used a slicer. Turn Sync off: Filtering then narrows this grid only.

Global search

The toolbar has a search box with its own operator dropdown: contains, is exactly, starts with, ends with, does not contain. It searches across every column at once.

Power BI: Smart Table Excel Style Column Filtering

2. Dynamic column headers, driven by a measure

A column header name is normally a static label. Here it can be a DAX measure.

Arrival Month is set to April, and the Key Metrics headers read Lead Time (Avg: 30.00), Room Nights (Avg: 1.00), ADR (Avg: 160.50). Change the month and they recompute.

The header honors the filter context, including the filters applied inside the grid itself. Filter Property down to Resort Hotel with the funnel and the averages in the header follow.

Power BI Smart Table: Measure Driven Column Headers (Dynamic Column names) - YouTube

3. Column groups, without any extra tables

Four groups across eleven columns: Reservation, Channel, Room Type, Key Metrics. Each one is a spanning header above its columns with its own colour.

The thing to notice is what is not involved. No disconnected table. No field parameters. No two table visuals stacked on top of each other. No shapes placed in the background: which means your column groups move to the right along with your cursor.

Power BI Smart Table: Grouping columns in a table without using a static table and a matrix visual - YouTube

Known limits

All of these come down to one idea: some filtering reaches the model and some is local to the grid.

30,000 rows That is the Power BI cap for a table mapping with this data reduction algorithm. Header filters push a real model filter, so filtering down to a workable set works fine. Do not point it at an unfiltered fact table.
Value lists cap at 2,000 distinct values per column**, built from the loaded rows. On a high cardinality column like a guest name, use the search box in the menu.
Ends with, Does not end with, and blank selections filter the grid only.** Power BI’s advanced filter operators have `StartsWith` and `Contains` but no `EndsWith`, and a blank has no equivalent in a model side `In` list. The menu tells you when you pick one of these.
Bookmarks restore the data but not the ticked checkboxes.** The model filter is persisted by Power BI, so the rows come back correctly. The header checkboxes just will not show as ticked.

I wrote this to make one point: the gaps in the built-in visuals are not permanent. Build your own, and with vibe coding that is no longer a months-long project.

I am not sharing the .pbiviz file. It is an experiment, not tested for production grade reports. Feel free to checkout my [website](https://www.techietips.co.in) and reach out to me if you are interested to do these experiments along with me,

I would like to continue developing this visual and will write detailed blogs in features explaining its features and implementations.

I hope you learned something new. Feel free to share your thoughts in the comments section.

Happy Learning!!!

My button & slicer disappeared in Power BI Service. Here's why.

Hans-Georg_Puls — Thu, 10 Sep 2026 21:28:14 GMT

Imagine you have created a beautiful table in Power BI with a slicer and a button placed on top of it, overlapping the table area.

The slicer is used to search for specific content. The button becomes clickable when exactly one row is selected and opens a detail page for that row.

In Power BI Desktop, everything works perfectly.

However, as soon as you publish your report to the Power BI service and start interacting with it, strange things may happen.

If you select one row, both the slicer and the button disappear.

If you leave the table area, both reappear.

However, as soon as you hover over them, they disappear again.

Even more strangely, if you move in and out of the table a few times, it may happen that the button is available again.

As it turns out, this is documented behaviour, not a bug. In reading view, Power BI brings the selected visual to the front layer, above anything that overlaps it. Selecting a table row selects the table, so the button and the slicer, which actually sit in front of the table, end up behind it and disappear.

What I can't explain is the button becoming available again after moving in and out a few times. I haven't found anything documented on that.

The good news: this default behaviour can be changed.

To change it, there is an option. It's called 'Maintain layer order' and can be found in a visuals format pane under Properties > Advanced options.

Switch it on for the visual that jumps to the front, in my case the table.

Have you run into this one? And on which visual did you end up switching it on?

Ask vs. Act: Fabric Data Agents and Fabric Operations Agents Explained (with Industry Scenarios)

FarhanJeelani — Thu, 10 Sep 2026 21:26:37 GMT

Feature status (September 2026): Fabric data agents and operations agents are both generally available. Investigator insights in operations agents is still in preview. Always check Microsoft Learn for the latest.

The core difference in one line

A Fabric data agent waits for a question and answers it from governed data. A Fabric operations agent doesn't wait for anyone: it continuously watches live data against business goals and raises its hand, or triggers an action, when a condition is met.

A simple mental model: the data agent is the expert analyst you can message at any time; the operations agent is the control-room operator who never goes off shift.

Fabric data agent: conversational analytics on governed data

The data agent is a configurable Fabric item that lets anyone ask plain-English questions about data in OneLake. Behind the scenes, it decides which source can answer the question, then generates and runs a read-only query: SQL for lakehouses and warehouses, DAX for Power BI semantic models, and KQL for KQL databases (including Eventhouse). It can also work with ontologies and Microsoft Graph.

One data agent can combine up to five sources. Authors improve accuracy by selecting relevant tables, writing agent instructions in business language (for example, "route revenue questions to the Finance semantic model"), and adding example question-and-query pairs, up to 100 per source (examples aren't currently supported for semantic models).

Security is where the data agent is deliberately conservative. Every query runs with the asking user's own credentials, so row-level and column-level security still apply, and Microsoft Purview controls such as DLP and access restriction policies are respected. It never creates, updates, or deletes data, and it doesn't trigger notebooks or other workflows.

Once published, the agent can be consumed well beyond Fabric: in Microsoft 365 Copilot, Teams, Azure AI Foundry, or as a tool inside Microsoft Copilot Studio agents, where the integration is now generally available through the Fabric IQ Data MCP tool.

It's built for conversational insight, not bulk extraction: responses are capped at 25 rows and 25 columns, and unstructured files like PDFs aren't supported.

Fabric operations agent: autonomous monitoring with human-in-the-loop action

The operations agent lives in the Real-Time Intelligence workload. Instead of answering ad-hoc questions, you give it business goals, instructions, a knowledge source (an Eventhouse KQL database or an ontology), and the actions it's allowed to recommend.

From that configuration, the agent generates a playbook: the business entities and properties it will track and the rules that define what "needs attention" looks like. You can build this manually or describe your intent to the built-in Copilot chat ("monitor the turbines and alert me when motor temperature gets too high") and refine the proposed rules iteratively. Each rule is backed by an inspectable query that the agent evaluates every five minutes, using either state conditions (such as is above 80, which keeps signaling while true) or transition conditions (such as crosses above 80, which signals only on the change).

When a rule is met, the agent sends a Teams message, to a person or a channel, through the Fabric Operations Agent Teams app, summarizing what it saw and what it recommends. Beyond messaging, you can configure actions that run a Fabric notebook or trigger a Power Automate flow, which opens the door to ServiceNow tickets, SAP work orders, emails, or any other Power Automate connector. For anomalies, Investigator insights (preview) adds correlated patterns and a likely root-cause summary directly in Teams.

Two governance details matter. First, each operations agent gets its own Microsoft Entra Agent ID, so its activity is auditable separately from human users. Second, it runs in delegated mode with its creator's permissions: when a recipient approves a recommendation, the action executes with the creator's access. Choose agent owners deliberately.

Side-by-side comparison

Dimension	Fabric data agent	Fabric operations agent
Core purpose	Answer questions about data	Monitor data and drive action
Interaction model	Pull: a user asks	Push: the agent notifies
Trigger	A human prompt	A rule condition being met
Data sources	Lakehouse, warehouse, semantic model, KQL database, mirrored database, ontology, Microsoft Graph (up to 5)	Eventhouse KQL database or ontology
Configuration	Tables, instructions, example queries	Goals, instructions, playbook rules, actions
Output	Natural-language answers, small tables, summaries	Teams alerts with recommendations, notebook runs, Power Automate flows
Write / action capability	Strictly read-only	Executes approved actions
Identity	Asking user's credentials (RLS/CLS enforced)	Entra Agent ID using creator's delegated permissions
Time orientation	"What happened, and why?"	"What's happening now, and what should we do?"
Where users meet it	Fabric, Teams, M365 Copilot, Copilot Studio, Foundry	Primarily Teams (alerts, approvals, investigations)

Industry scenarios

1. Manufacturing: from vibration spike to work order

A discrete manufacturer streams CNC machine telemetry (spindle vibration, motor temperature, cycle time) through Eventstream into an Eventhouse.

Operations agent: Its goal is to minimize unplanned downtime. A rule watches for vibration crossing above tolerance on any machine. When it fires, the maintenance supervisors' Teams channel receives an alert with the machine, recent readings, and a recommendation to schedule an inspection. On approval, a Power Automate flow creates a work order in the maintenance system.

Data agent: Preparing for a quarterly review, the plant manager asks: "Which lines had the most unplanned downtime last quarter, and how did OEE compare to target?" The agent queries the production lakehouse and OEE semantic model and returns a ranked answer, with no report-building required.

2. Retail and CPG: protecting a flash promotion

During a weekend promotion, point-of-sale and inventory events land in Eventhouse in near real time.

Operations agent: It monitors store-level inventory for promoted SKUs and flags any item dropping below safety stock. The regional replenishment lead gets a Teams recommendation to trigger an emergency transfer from a nearby store or distribution center, executed through Power Automate after approval.

Data agent: On Monday, the category manager asks: "How did the promotion perform versus the same weekend last year, by region, and which stores ran out of stock?" The agent blends the sales semantic model with lakehouse inventory history to answer in seconds.

3. Energy and utilities: pipeline pressure and grid load

An oil and gas operator ingests SCADA readings (pressure, flow rate, compressor status) from field assets; a utility does the same for substations.

Operations agent: Rules detect pressure exiting its normal operating range at a pumping station, or transformer load crossing a critical threshold during a heatwave. The control room is alerted with context and a recommended response, such as dispatching a field technician via a flow. Human approval stays in the loop. Note that operations agents complement, and never replace, certified safety instrumented systems.

Data agent: An asset integrity engineer asks: "Which ten assets had the most pressure excursions in the last 12 months, and when were they last serviced?"

4. Financial services: payment gateway degradation

A bank streams card authorization events by gateway, merchant category, and channel.

Operations agent: Its goal is protecting payment success rates. When a gateway's decline rate crosses above its threshold, the payments operations team gets an alert with a breakdown and a recommendation to reroute traffic to a secondary processor, an action that runs only after an engineer approves it.

Data agent: A relationship manager asks: "Which of my corporate clients saw failed payments above 2% this month?" Because the agent runs under the manager's own identity, row-level security ensures they only see their own portfolio.

5. Pharma logistics: cold-chain integrity

A logistics provider moving vaccines and biologics collects IoT temperature and GPS data from refrigerated containers.

Operations agent: A transition rule fires the moment a shipment's temperature exits the 2–8°C range. The control tower is notified with location and excursion details, a notebook action flags the batch for quality review, and a flow alerts the receiving site.

Data agent: A quality lead asks: "Which lanes and carriers had the highest excursion rates this year?"

Better together: detect, decide, explain

The strongest architectures use both agents on the same data foundation. Picture an airport operations team. An operations agent watches baggage-system events in Eventhouse and alerts the duty manager when belt throughput drops below target during a peak arrival wave. The manager approves the recommended action: rerouting bags to an alternate belt via Power Automate.

Then, without leaving Teams, the manager asks a Copilot Studio agent grounded by a Fabric data agent over the same Eventhouse and historical lakehouse: "How often has this belt degraded at this time of day over the past 90 days?"

The operations agent handles detection and response. The data agent handles explanation and learning. Because both point at the same governed data in OneLake, the alert and the analysis never disagree about the numbers.

How to choose

Choose a data agent when the need is exploratory and question-driven: executive self-service, analyst acceleration, or bringing trusted data into Copilot experiences. Choose an operations agent when you have a specific, repeatable business process with measurable conditions, where minutes matter and someone should be told (or something should happen) without anyone having to ask.

A quick test: if you hear "someone should check this dashboard every hour," you need an operations agent. If you hear "I just need to know X before Friday's meeting," you need a data agent.

Practical tips before you build

For data agents: curate tightly by selecting only the tables users need, write instructions in business terms, and invest in example queries, which are often the biggest accuracy lever. Use Git integration and deployment pipelines to promote agents from development to production.

For operations agents: scope one agent to one business process, flatten nested JSON columns in Eventhouse tables before configuring, and review each rule's generated query before starting the agent. Remember that approvals execute with the creator's permissions, trial capacities aren't supported, and agents consume capacity units, so track them in the Fabric Capacity Metrics app.

For both: confirm your tenant admin has enabled the required Copilot and Azure OpenAI settings, including cross-geo AI processing where your capacity region requires it.

Wrapping up

Fabric data agents and operations agents are two sides of the same coin. The data agent democratizes understanding: anyone can ask a question and get a governed answer. The operations agent operationalizes response: the business stops depending on people watching dashboards and starts reacting to events as they happen.

Start with the scenario that hurts most today, build on a shared OneLake foundation, and grow from there.

How are you using these agents in your industry? Share your scenarios in the comments!