Cross-Tenant Delegated OneLake Shortcut in Microsoft Fabric | End-to-End Tutorial
In this video, I walk through how to architect and implement a Cross-Tenant Delegated OneLake Shortcut in Microsoft Fabric. Discover how organisations can securely access and consume data stored in another Microsoft Fabric tenant without physically copying, moving, or replicating the data. Through a practical, step-by-step demonstration, you'll learn how to: ✅ Configure connectivity between two Microsoft Fabric tenants ✅ Implement delegated authentication for secure access ✅ Create a Cross-Tenant OneLake Shortcut ✅ Share data across organisational boundaries while maintaining governance and security ✅ Create Semantic model and Power BI report in the consumer tenant 🔍 What You'll Learn ✅ Understanding Cross-Tenant Delegated OneLake Shortcuts ✅ Microsoft Fabric multi-tenant data architecture ✅ Producer and Consumer tenant architecture design ✅ Configuring cross-tenant access in Microsoft Fabric ✅ Setting up delegated authentication using an Organisational Account or Service Principal ✅ Configuring workspace permissions and OneLake security ✅ Creating a Cross-Tenant OneLake Shortcut ✅ Accessing data across tenants without data duplication ✅ Security considerations and best practices for cross-tenant data sharing 🎯 Who Should Watch? This tutorial is ideal for: 🔹 Microsoft Fabric Data Engineers 🔹 Data Architects 🔹 Cloud Engineers 🔹 Analytics and Platform Teams 🔹 Anyone building secure multi-tenant data-sharing solutions If you're learning Microsoft Fabric and want to understand how OneLake Shortcuts enable governed data sharing across organisational boundaries, this is a tutorial you don't want to miss. 💡 One of the biggest advantages of OneLake Shortcuts is the ability to eliminate unnecessary data duplication while maintaining security, governance, and a single source of truth across tenants. 🎥 Watch the video https://www.youtube.com/watch?v=ihlWLRiia7Q&t=693s and let me know your thoughts in the comments. 👍 If you find the content valuable, please Like, Share, and Subscribe for more practical tutorials on: ✅ Microsoft Fabric ✅ Azure Data Engineering ✅ Data Architecture ✅ Modern Data Platforms #MicrosoftFabric #OneLake #OneLakeShortcut #CrossTenant #DelegatedAuthentication #MicrosoftFabricTutorial #DataEngineering #DataArchitecture #MicrosoftAzure #AzureDataEngineering #Lakehouse #DataSharing #ServicePrincipal #DataSecurity #CloudComputing #FabricLakehouse #DataPlatform #BigData #MicrosoftMVP #DatabricksMVP #AbiolaDavidFabric Data Agents Solve the Last-Mile Problem Between People and Governed Data
Fabric Data Agents can make governed data easier to question, understand, and use. This article explores what they solve, what they do not, and why trusted data, strong semantic models, and real-world testing are essential.Is Your Power BI Report Slow? 5 Data Modeling Mistakes You Should Check First
Power BI performance issues aren’t always caused by large datasets; poor data modeling can also slow down reports, slicers, and DAX measures. In this article, we’ll explore five common data-modeling mistakes that impact performance and practical ways to optimize your Power BI reports. 1. Building One Giant Flat Table The Mistake One of the most common approaches is putting everything into a single table: Customer information Product information Transaction details Location Dates Categories Measures Status fields At first, this seems simpler because there are fewer relationships to manage. But a single, wide table can introduce data duplication and make the model harder to optimize. Why It Can Hurt Performance Power BI's VertiPaq engine uses columnar storage and compression. The efficiency of that compression depends heavily on the characteristics of your columns. A large flat table can contain: Repeated descriptive values High-cardinality columns Unnecessary columns Duplicated customer or product attributes For example, imagine a fact table containing 10 million transactions where Customer Name, Customer City, Customer State, and Product Category are repeated across every transaction. You're storing the same descriptive information over and over again. The Fix: Build a Star Schema Instead of one giant table, separate your model into fact tables and dimension tables. Fact tables Sales Transactions Encounters Claims Inventory Dimension tables Date Customer Product Provider Location The fact table should contain the events or transactions you want to analyze, while dimension tables provide the descriptive attributes used for filtering and grouping. For example: DimDate | | DimCustomer ---- FactSales ---- DimProduct | | DimLocation This structure reduces redundancy, improves model organization, and aligns well with how Power BI's storage engine is designed to work. Pro Tip Don't just remove columns randomly. Instead, ask: Does this column need to exist in this table? If a descriptive attribute belongs in a dimension, move it there. 2. Using Bidirectional Relationships Everywhere The Mistake You create a relationship and immediately set the cross-filter direction to Both because it makes filters appear to "work everywhere." It seems convenient. But as your model grows, bidirectional relationships can introduce unnecessary complexity. Why It Can Hurt Performance With single-direction filtering, filters typically flow from the dimension to the fact table: DimCustomer → FactSales With bidirectional filtering: DimCustomer ↔ FactSales Filter context can now travel in both directions. When multiple tables and relationships are involved, this can create: Ambiguous filter paths More complex query behavior Unexpected filtering Difficult-to-debug totals Additional performance overhead in complex models This becomes especially important when your model contains multiple fact tables or many-to-many relationships. The Fix Use single-direction relationships by default. For a typical star schema: Dimension → Fact Use bidirectional filtering only when you have a specific reason for it and have tested its impact. If you need bidirectional behavior for a particular calculation, you can sometimes handle it within the measure using CROSSFILTER() rather than changing the relationship behavior across the model. Pro Tip If your report suddenly starts showing unexpected totals or filters are affecting tables you didn't expect, inspect your relationships. Relationship direction is often one of the first places worth checking. 3. Putting Multiple Business Processes Into One Fact Table The Mistake You have Sales, Returns, Inventory, Support Tickets, and other business processes, and decide to put everything into one massive fact table. At first, this may seem convenient. But there's a fundamental modeling problem: Different business processes usually have different grains. What Is Grain? The grain defines what one row represents. For example: One row = one sales transaction One row = one product shipment One row = one customer support ticket One row = one patient encounter These represent different business processes with different grains. Trying to combine them into one fact table can result in: Many blank columns Multiple flags indicating row types Complicated DAX Duplicate information Difficult-to-understand logic Larger and harder-to-maintain tables You may eventually find yourself writing logic like: IF( TransactionType = "Sales", ..., IF( TransactionType = "Return", ..., ... ) ) That's often a sign that one table is trying to do too many jobs. The Fix: Use One Fact Table Per Business Process Instead, create separate fact tables: DimDate / \ / \ FactSales FactReturns \ / \ / DimProduct Both fact tables can use shared dimensions such as: Date Customer Product Location These are often called conformed dimensions. This allows you to analyze different business processes consistently without forcing them into a single fact table. Pro Tip Before creating a fact table, write one sentence describing its grain: "One row represents ______." If you can't clearly complete that sentence, your table design probably needs another look. 4. Composite Models: When to Mix Import and DirectQuery The Mistake: Assuming you have to choose one storage mode for your entire model. Why It Matters Import mode loads data into memory using VertiPaq, generally providing fast report interactions. However, the data depends on refreshes. DirectQuery queries the source system when data is requested, allowing fresher data but making performance more dependent on the source database, network, and query complexity. Composite models allow you to combine Import and DirectQuery tables within the same semantic model. When to Consider a Composite Model Use a composite model when different parts of your data have different requirements. For example: Power BI Model | ┌───────────┴───────────┐ ↓ ↓ Import Mode DirectQuery ↓ ↓ Dimensions / Summary Large Fact Table Relatively stable Frequently changing You might: Keep relatively small, frequently used dimension tables in Import mode. Keep very large or frequently changing fact tables in DirectQuery. Use aggregation tables to answer common summary queries from Import-mode data while retaining access to detailed DirectQuery data. Incorporate native queries within DirectQuery sources to offload complex transformations directly to the underlying database engine. You can also extend models using DirectQuery for Power BI semantic models and Analysis Services, allowing live connection sources to be combined with imported data. Technical Note Composite models can introduce additional complexity, particularly when relationships cross storage modes. Test the model with realistic report queries and user interactions rather than assuming that combining storage modes will automatically improve performance. Pro Tip Don't choose DirectQuery simply because your dataset is large. Ask two questions: How fresh does the data need to be? Can Import mode meet that requirement? If your data can be refreshed every few hours or once a day, Import may provide a much better user experience than DirectQuery. 5. Creating Dozens of Duplicate Time-Intelligence Measures The Mistake Your model starts with: Sales Profit Quantity Then you add: Sales YTD Profit YTD Quantity YTD Then, Sales MTD Profit MTD Quantity MTD And eventually, Sales PY Profit PY Quantity PY Before long, your Fields pane contains dozens—or even hundreds—of similar measures. The calculations may be correct, but maintaining them becomes increasingly difficult. A change to your time-intelligence logic may require updating many individual measures. The Fix: Calculation Groups Calculation groups allow you to define reusable calculation logic once and apply it to multiple measures. For example, instead of creating a separate YTD measure for every metric, you can create a YTD calculation item that works with: Sales Profit Quantity Revenue Visits Claims The same approach can be used for: YTD MTD QTD Previous Year Year-over-Year Variance Calculation groups can be created directly in Power BI Desktop from Model view. They can also be managed using external tools such as Tabular Editor. A simplified concept looks like this: Base Measure ↓ Calculation Group ↓ YTD / MTD / PY / YoY For example, a Previous Year calculation item could use: CALCULATE( SELECTEDMEASURE(), SAMEPERIODLASTYEAR('Date'[Date]) ) The SELECTEDMEASURE() function allows the same calculation item to be applied to different base measures. This can significantly reduce duplicated measures and make your semantic model easier to maintain. Important Note Calculation groups are primarily a modeling and maintainability technique. They aren't a magic performance switch. If an underlying measure is inefficient, putting it inside a calculation group won't automatically make it faster. The main benefit is reducing repetitive calculation logic and making your semantic model easier to manage. Pro Tip If you find yourself creating the same calculation pattern repeatedly for different measures, that's a good signal to investigate calculation groups. A Quick Power BI Performance Checklist Before adding more visuals or rewriting your DAX, check these five areas: ✅ Model Structure Am I using a star schema? Have I removed unnecessary columns? Are descriptive attributes stored in dimensions? ✅ Relationships Are relationships mostly single-direction? Do I have unnecessary bidirectional relationships? Are there ambiguous filter paths? ✅ Fact Tables Does each fact table represent a clear business process? Is the grain clearly defined? Am I mixing different grains in the same table? ✅ Storage Mode Do I genuinely need DirectQuery? Could Import mode handle the requirement? Would a composite model or aggregation strategy make sense? ✅ DAX & Semantic Model Am I creating many duplicate measures? Can calculation groups reduce repeated time-intelligence logic? Are my base measures already optimized? Final Thoughts Power BI performance isn't just about how many rows you have. It's about how efficiently those rows are modeled, stored, filtered, and queried. A well-designed model can give you: Faster report interactions Faster refreshes Better scalability Simpler DAX Easier maintenance More predictable results So, the next time your Power BI report starts feeling slow, don't immediately start deleting visuals or rewriting every DAX measure. Start with the model. Check your table structure. Check your relationships. Check the grain of your fact tables. Check your storage mode. And check whether you're duplicating calculation logic unnecessarily. Sometimes the biggest performance improvement comes from fixing the model underneath it.
Build a Fabric IQ Declarative Agent for Microsoft 365 Copilot Chat
Did you know Microsoft 365 Copilot Chat natively supports Fabric IQ? You can query your Power BI reports and semantic models right from the general chat, without any custom setup. Ask a business question in plain English, and M365 Copilot automatically discovers the most relevant Fabric artifact you have access to, inspects its schema, runs queries, and returns a cited answer alongside your broader Microsoft 365 context. When to build a dedicated agent The discovery step works well and can often identify the right items on the first try, but it's not without tradeoffs. If you rely on a curated set of reports and semantic models and want fast, repeatable answers with less guesswork, we can do better. A dedicated agent lets you: Skip the discovery lag and ambiguity. In general chat, M365 Copilot may fan out searches across emails, Teams chats, OneDrive decks, and all your workspaces before settling on a Power BI item, adding several seconds of latency. If you have a dozen reports about "Sales" or "Revenue", it may pick the wrong one or return inconsistent answers from one day to the next. Agents let Copilot bypass tenant-wide search by routing queries to the correct report or model immediately. Create repeatable, controlled experiences without interference from Office content. Because general chat blends all your Microsoft 365 content, an answer might pull numbers from an outdated email thread or Word document instead of your authoritative Power BI report. Scoping the agent to your trusted Power BI reports and semantic models, and only the M365 sources you explicitly enable, ensures every user gets consistent, governed answers. Show native zero-prompt starters and improve follow-ups. Guide users with tailored conversation starters from the moment they open the agent. Provide them with smarter, context-aware follow-up suggestions after each answer. Control presentation and charting logic. You decide how metrics are defined and presented to users, how tables are formatted, and influence how charts are styled and labeled. Package and share easily. A declarative agent is easy to distribute across your organization in a few clicks. In this blog post, I'll walk you through how you can create your own agent for Microsoft 365 Copilot Chat. We'll build it as a declarative agent and use GitHub Copilot to assist with the development. How the Microsoft 365 Copilot declarative agent uses routing instructions and Fabric IQ MCP to select among sales, finance, and inventory reports or semantic models. What we're building Here is our AdventureWorks Insights declarative agent in action, greeting users with tailored zero-prompt conversation starters, and then answering directly from our curated Power BI semantic model: AdventureWorks Insights declarative agent zero-prompt home screen showing tailored conversation starters. AdventureWorks Insights declarative agent ranking five sales territories, with sales, share of all sales and a source citation. How declarative agents work Under the hood, a declarative agent runs on the same orchestrator and foundation models as M365 Copilot Chat. It is configured through a lightweight set of files that define: identity and behavior (name, instructions, constraints, and conversation starters), knowledge (approved grounding sources), capabilities and actions (built-in tools like Code Interpreter or MCP-based plugins), and host metadata (manifest.json and icons for packaging and distribution). A declarative agent project is packaged from a single appPackage/ folder alongside your Agents Toolkit project configuration: my-fabric-iq-agent/ ├── m365agents.yml # Build & validate workflow ├── env/.env.dev # Environment variables └── appPackage/ ├── manifest.json # App identity & icons ├── declarativeAgent.json # Starters, capabilities & actions ├── agent_instructions.md # Routing & chart rules (max 8,000 chars) ├── ai-plugin.json # Fabric IQ MCP plugin manifest └── color.png / outline.png # 192x192 & 32x32 icons Copy-paste prompt to get started You don't need to write these JSON files by hand. Unzip the starter template (.zip) (or use an empty folder), then copy the instructions below and paste them as-is into GitHub Copilot App, CLI, Claude Code, or your favorite harness. The coding agent will interview you for your target reports and preferences, inspect your schemas, and generate the entire project for you. Before you start, think about what you want your agent to do: Capabilities: turn on CodeInterpreter so the agent can write Python to render custom charts. If your users want to blend Power BI numbers with strategy decks or team discussions, enable OneDriveAndSharePoint, GraphConnectors, or TeamsMessages. Instructions, conversation starters & follow-ups: clickable starter cards that launch your most common analyses without typing, paired with instructions that guide smart follow-up suggestions and routing. See section below for how to write effective instructions. Connectors: during authoring, your coding agent inspects your items via the Fabric IQ MCP server (https://fabriciq.svc.cloud.microsoft/v1/mcp/fabriciq), and wires the finished agent's ai-plugin.json to the M365 Fabric IQ integration (known as AskPowerBI). You can also attach additional MCP servers or API plugins if your workflow spans other systems. Branding & packaging: Your agent's short name (max 30 chars), descriptions (short max 80 chars, full max 4,000 chars), two icons (color.png at 192x192 and outline.png at 32x32), and an accent color bundled into an installable M365 app package. Instructions to copy and paste Build a Microsoft 365 Copilot declarative agent backed by the Fabric IQ MCP server (m365agents.yml, env/.env.dev, and appPackage/: declarativeAgent.json, agent_instructions.md, ai-plugin.json, manifest.json, color.png, outline.png). Template: Fetch https://community.fabric.microsoft.com/blog/community_blog/build-a-fabric-iq-declarative-agent-for-microsoft-365-copilot-chat/5377379 (or unzip my-fabric-iq-agent-template.zip) and reuse its JSON templates; skip external doc fetches unless troubleshooting. Workflow: 1. Gather inputs: Ask which Power BI reports/models to include (names, URLs, or discover via Fabric IQ MCP), agent name, and optional capabilities beyond `CodeInterpreter` (`OneDriveAndSharePoint`, `TeamsMessages`, `GraphConnectors`). Wait for my reply. 2. Inspect in parallel: Call `GetReportMetadata` and `GetSemanticModelSchema` in parallel (both accept a Report GUID; use `DiscoverArtifacts` only to resolve names). Run one batched `ExecuteQuery` only if sample values or date ranges are missing. 3. Scaffold strict JSON (no `//` comments): Create `declarativeAgent.json` (v1.8, `"instructions": "$[file('agent_instructions.md')]"`, 4-6 starters), copy `ai-plugin.json` (v2.4, `AskPowerBI`) verbatim, create `manifest.json` (v1.25) + `m365agents.yml`, and generate 192x192 `color.png` & 32x32 `outline.png` (`agent_instructions.md` max 8,000 chars; `name.short` max 30; `description.short` max 80). 4. Write `appPackage/agent_instructions.md` (max 8,000 chars): (1) Artifact Routing Table with GUIDs & rules to skip `DiscoverArtifacts`, (2) 4-6 Example Questions with follow-up context, (3) Producing Answers rules, (4) Charting Guidelines (lollipop/dumbbell/line, light background, business axis titles). 5. Package & validate (min 90s timeout): Run `npx --yes @microsoft/m365agentstoolkit-cli package --env dev && npx --yes @microsoft/m365agentstoolkit-cli validate --env dev`, then help run `provision --env dev` or sideload `appPackage/build/appPackage.dev.zip`. Testing and distributing your agent Once your files are ready, run these commands from your project root using the Microsoft 365 Agents Toolkit CLI, or ask Copilot to run them for you: # 1. Build appPackage/build/appPackage.dev.zip (inlines agent_instructions.md) npx --yes @microsoft/m365agentstoolkit-cli package --env dev # 2. Validate schemas, limits & icons npx --yes @microsoft/m365agentstoolkit-cli validate --env dev # 3. Provision & sideload to M365 npx --yes @microsoft/m365agentstoolkit-cli provision --env dev Test in Copilot Chat. Open m365.cloud.microsoft/chat, select your agent from the right-hand rail, and click your zero-prompt conversation starters to verify routing, citations, and chart rendering. Share with your team or organization. Run npx --yes Microsoft/m365agentstoolkit-cli share --scope shared to grant access to specific security groups, or submit appPackage/build/appPackage.dev.zip to the Microsoft 365 Admin Center (Settings > Integrated apps) for tenant-wide catalog deployment. Every user's queries continue to respect their individual Power BI permissions and row-level security (RLS). Under the hood: configuring a declarative agent This section walks you through the various files that make up a declarative agent. You don't have to understand every detail, but it's helpful to understand how the pieces fit together. declarativeAgent.json This is the core manifest, it defines your agent's identity and capabilities: { "$schema": "https://developer.microsoft.com/json-schemas/copilot/declarative-agent/v1.8/schema.json", "version": "v1.8", "name": "AdventureWorks Insights", "description": "Answers sales, finance, and inventory questions using curated Power BI reports and semantic models.", "instructions": "$[file('agent_instructions.md')]", "conversation_starters": [ {"title": "Top markets by sales", "text": "Which markets bring in the most sales? Show the top five and their share of total sales."}, {"title": "Category breakdown", "text": "What product categories drive our leading market?"}, {"title": "Margin vs. stock check", "text": "Compare gross margin and current stock levels for our top product category."} ], "capabilities": [{"name": "CodeInterpreter"}], "behavior_overrides": {"default_response_mode": "Think deeper"}, "actions": [{"id": "fabric-iq-action", "file": "ai-plugin.json"}] } Note: Keep JSON strict (no // comments); package inlines agent_instructions.md (max 8,000 chars). Adding Fabric IQ connection via ai-plugin.json The "actions" entry above points to ai-plugin.json in the same appPackage/ folder. This file connects your declarative agent to the Fabric IQ M365 MCP integration (AskPowerBI) and maps Power BI source metadata into native M365 Copilot citations and sensitivity labels. You can drop this ready-made version straight into your project: { "$schema": "https://developer.microsoft.com/json-schemas/copilot/plugin/v2.4/schema.json", "schema_version": "v2.4", "namespace": "fabriciq", "name_for_human": "Fabric IQ", "description_for_human": "Answers questions grounded in curated Power BI reports and semantic models.", "description_for_model": "Ask data questions about the Power BI artifacts in instructions. Pass artifactId from the routing table.", "functions": [{ "name": "AskPowerBI", "description": "Answers a Power BI report or model question via DAX. Prefer Report GUIDs; include prior context on follow-ups.", "parameters": {"type": "object", "properties": {"artifactId": {"type": "string"}, "query": {"type": "string"}, "context": {"type": "string"}}, "required": ["artifactId", "query"]}, "capabilities": {"response_semantics": {"data_path": "$.content[*]._meta.artifact_citation", "properties": {"title": "$.Name", "subtitle": "$.Description", "url": "$.Url", "information_protection_label": "$.SensitivityLabelId", "thumbnail_url": "$.IconUrl"}}} }], "runtimes": [{ "type": "RemoteMCPServer", "auth": {"type": "None"}, "spec": { "url": "https://api.fabric.microsoft.com/v1/mcp/fabricaihub/integrations/m365", "mcp_tool_description": {"tools": [{ "name": "AskPowerBI", "description": "Answers a Power BI report or model question via DAX.", "inputSchema": {"type": "object", "properties": {"artifactId": {"type": "string"}, "query": {"type": "string"}, "context": {"type": "string"}}, "required": ["artifactId", "query"]}, "annotations": {"destructiveHint": false, "openWorldHint": false, "readOnlyHint": true}, "_meta": {"variants": "Fabric.Routing.M365.V1", "microsoft/response_semantics": {"data_path": "$.content[*]._meta.artifact_citation", "properties": {"title": "$.Name", "subtitle": "$.Description", "url": "$.Url", "information_protection_label": "$.SensitivityLabelId", "thumbnail_url": "$.IconUrl", "x-ms_unique_id": "$.ArtifactId"}}} }]} }, "run_for_functions": ["AskPowerBI"] }] } How to write efficient instructions Tip: Usually, it is best to put model-specific instructions and examples as close to the data source as possible. If your instructions are model-specific, like covering how to write DAX queries or which parts of your schema are relevant, consider adding them to the semantic model's custom instructions through Power BI's Prep data for AI experience. Keep them in your declarative agent's instructions only when you can't edit the underlying semantic model, or when you want rules that apply exclusively to this agent without changing how the model behaves elsewhere. When you need to define behavior at the agent level, write instructions to agent_instructions.md. We will structure it around four focused blocks. 1. Build an artifact routing table When you include exact artifact GUIDs and contents in a markdown table, the agent skips the discovery search call and routes straight to the right report or semantic model. ## Artifact Routing Table Match the question to the right source and pass Artifact ID directly to AskPowerBI: | Artifact Name | Type | Artifact ID | Key Content | | :--- | :--- | :--- | :--- | | **AdventureWorks Sales** | Report | `60763385-b565-411a-b266-05c22543d6e4` | `[Total Sales]`, `[Orders]`; `Sales Territory`, `Category`, `Fiscal Year` | | **Global Margin Model** | Model | `14c445e9-24b5-4ded-86b4-f2e1db5b0f34` | `[Gross Margin %]`, `[COGS]`, `[Net Revenue]`; `Cost Center`, `Category` | | **Supply Chain Tracker** | Report | `9b2184a1-4c90-4e12-8a15-7f8e9102bc31` | `[Units on Hand]`, `[Backorder Rate %]`; `Warehouse`, `Supplier` | Routing rules: - Sales, volume, and territory rankings to **AdventureWorks Sales**. - Cost, profitability, and gross margin to **Global Margin Model**. - Stock and warehouse availability to **Supply Chain Tracker**. - Cross-domain questions to query each artifact separately and align by `Category`. 2. Include example questions Give the model a few concrete examples so it learns how to disambiguate similar metrics and carry context across turns: ## Example Questions & Routing - "Which markets lead in sales?" to Query **AdventureWorks Sales** (`[Total Sales]` by `Sales Territory[Region]`). - "What drives the leading market?" to Keep prior territory and period; break down `[Total Sales]` by `Product[Category]`. - "How is gross margin trending for Bikes?" to Switch to **Global Margin Model**, keep `Category = "Bikes"`, query `[Gross Margin %]` by `Fiscal Year`. - "What is our revenue?" to Clarify gross `[Total Sales]` (Sales report) vs. `[Net Revenue]` (Finance model). 3. Define how to produce answers Tip: Only add custom answer-formatting rules if the out-of-the-box experience does not suit your needs. Fabric IQ's built-in instructions are already tuned to provide strong defaults for grounding, charts, citations, and formatting. If your audience requires a specific executive style, such as always leading with a one-sentence headline or suggesting cross-model follow-ups, you can layer those preferences in: ## Producing Answers - **Lead with the headline.** Start with a one-sentence finding naming the leader, exact figure, and share of total. - **State denominator and period.** Specify the date range and whether percentages are share of all sales or top N. - **Suggest follow-ups.** End with 2-3 next questions grounded in your routed artifacts. 4. Standardize charting logic Because CodeInterpreter is enabled, you can instruct the agent to render clean, publication-grade visuals automatically, without the user asking for a chart: ## Charting Guidelines - Include a chart for any comparison of 3+ items. - **Visual forms:** Sorted horizontal lollipop for rankings, dumbbell for 2-period comparisons, line for trends. - **Axis labels:** Use business titles (`Market`, `Sales ($M)`); never `X` or `Y`. - **Styling:** Light background, direct data labels, quiet gridlines, and `#b11f4b` accent for the leader. (Tip: In M365 Copilot Chat, users can also open the chart menu and select Display as interactive chart for native inline hover tooltips.)Scaling Machine Learning for Data Science with Dask in Microsoft Fabric
In this edition, we’re exploring why Dask has become such a practical tool for scaling machine learning in Python as your data grows. By the time you’re done reading, you’ll have a clear sense of how Dask fits naturally into a data professional’s way of thinking, without forcing you to give up the tools or habits you already rely on.Why To Move All Fabric Connections to Service Principal Authentication
The Problem with Personal Account Dependencies Many organizations start with data connections created using developer or analyst accounts because it is quick and convenient. Over time, this creates several challenges: Report refreshes depend on a specific employee account. Password changes can break connections. Employee transfers or resignations create operational risks. Troubleshooting ownership becomes difficult. In short, critical business reporting ends up depending on an individual's credentials. Why Service Principal? A Service Principal acts as a non-human identity that can authenticate securely against Fabric resources. Instead of: Report → User Account → Data Source The architecture becomes: Report → Service Principal → Data Source This means the connection belongs to the organization rather than to a specific user. No dependency on user accounts Easier onboarding and offboarding Better governance and security Reduced refresh failures caused by credential changes Centralized ownership of reporting assets Improved business continuity Implementation Approach Step 1: Create a Service Principal Create a Service Principal through Azure App Registration in Azure Portal. This will generate: Client ID Tenant ID Client Secret These credentials would later be used for authentication. Copy the secret Value, not its Secret ID, and store it in your approved secret store. The value cannot be retrieved after leaving the screen Please refer this great article for setting up your Service Principal : Link Step 2: Create a Security Group Instead of granting permissions directly to the Service Principal, add it to a dedicated security group. Create a security group in Azure Portal and and add this service principal in this se. Benefits: Easier permission management Better governance Future scalability If additional Service Principals are needed later, they can simply be added to the same group. Step 3: Grant Workspace Access Add the security group as a member of the required Fabric workspaces. This will ensure that the Service Principal is having sufficient permissions to access workspace artifacts and related resources. Step 4: Create New Connections Using the Service Principal Power BI Admin need to enable below 2 settings in the Admin portal for the security group we created: Service principals can access read-only admin APIs Service principals can call Fabric public APIs Step 5: Create New Connections Using the Service Principal For each report and semantic model connection: Create a new Fabric connection. Select Service Principal authentication. Provide: Tenant ID Client ID Client Secret (value) Save and validate the connection. If configured correctly, the connection should authenticate successfully. Step 6: Rebind Semantic Models Existing semantic models can be updated to use the newly created Service Principal connection instead of personal credentials. This is the step where ownership effectively moves away from individual users. How To Validate: Simply saving the connection is not enough, multiple validation checks are required to ensure the configuration success. Validation 1: Connection Test The connection was created successfully and authenticated without errors. Validation 2: Semantic Model Refresh Trigger a manual refresh using the Service Principal-based connection. The refresh should complete successfully. Validation 3: Refresh History Refresh history will show successful execution after the migration. This confirms that the authentication was working end-to-end. Validation 4: Cross-Workspace Testing The same approach can be repeated in another workspace and semantic model. The refresh must complete successfully there as well. This eliminats the possibility that the first success was due to a workspace-specific configuration. Validation 5: User Independence Check The biggest test: The connection no longer depended on the creator's account. The reporting solution could continue functioning even if the original developer was unavailable. Lessons Learned: A few observations: Using a security group makes permission management cleaner than assigning permissions directly. Test with a non-production semantic model first. Validate refreshes after every change. Document Tenant ID, Client ID, and ownership information securely. Standardize this approach for all future reporting projects to avoid technical debt. Final Thoughts Service Principal authentication is one of those improvements that users never notice, but administrators will appreciate for years. The migration does not change report functionality, visuals, or user experience. What it does change is reliability, governance, and supportability behind the scenes. If your Fabric environment still relies on personal accounts for report connections, now is a good time to review your architecture. Moving to Service Principal authentication can significantly reduce operational risk and ensure that critical business reports continue running regardless of personnel changes. The best infrastructure is often the infrastructure nobody notices. After this migration, our reports simply kept working, and that was exactly the outcome we want!Understanding Attention Mechanisms and Transformers for Data Science in Microsoft Fabric
In this edition, we’re exploring attention mechanisms and why they became such a big deal in modern deep learning. By the time you reach the end, you will have a clear sense of what problem attention was meant to solve, why older sequence models struggled, and how attention fits naturally inside transformer models using a torch-style way of thinking.Fabric IQ: from booked hours to next year's forecast
You might remember that the first time you asked an AI agent about next year's project hours, it told you it could not predict the future. Fair enough. It had historical data, not a forecast. In this build, we gave it one. Fabric Plan generated monthly forecasts from booked hours. We wrote those values back to a database, connected them to the business entities in an ontology, and let a data agent answer questions about them. That distinction matters. Predict produced the numbers. The data agent retrieved and explained them. The interesting part was getting those pieces to agree on what "forecast" meant. We used synthetic planning, procurement and maintenance data. Forecasting connected the planning model, ontology and agents. Live telemetry and graph exploration showed where the other pieces fit, and where they did not. Disclaimer: These are notes from a hands-on lab built on synthetic data, and the findings are our own. Fabric IQ features and limits change quickly, so treat the Microsoft Learn and official documentations in the references as the authoritative source. Put a forecast behind the question The starting point was a planning semantic model with actual booked hours by project, discipline and month. In Fabric Plan, closed periods took those actuals. Open periods held editable forecast values. We used Predict [1] to populate the open periods from historical data. For monthly forecasting, the current guidance calls for at least 24 months of history; some algorithms need more. Our sample history went back to September 2023. The forecast was then written to a Fabric SQL database and exposed through a OneLake shortcut in the gold lakehouse. We added a ProjectForecast entity to the ontology, linked to Project and Discipline. The data agents could now reach the forecast as data rather than being asked to invent it. The question became: Which projects have the most forecast hours in 2027? That has a queryable answer: select the forecast scenario, filter the year, group the detail rows by project and rank the totals. Writing back the forecast did not automatically add it to the original semantic model. We had to expose that new table deliberately. This demonstrated the workflow, not forecast accuracy. The data was synthetic, and we did not backtest the forecast against held-out actuals. A correct answer about stored predictions is a different test from whether those predictions will be useful. Two forecasts, two different questions The first problem was not statistical. It was vocabulary. Our planning semantic model already had a measure called Forecast Hours At Completion. It estimated total hours at completion from baseline hours and earned-value progress. It was not a month-by-month prediction. Asked for the 2027 forecast, the agent initially chose that measure, applied a 2027 filter and found nothing. The measure name sounded right. The meaning was wrong. We made the distinction explicit in the agent instructions: Monthly, yearly and future-period forecasts use the Plan forecast table. Estimates at completion and overruns use the earned-value measures. An unqualified "forecast" means the Plan forecast in this application, and the answer must say so. After that change, the next-year question returned the expected project ranking. The useful check was not whether the answer sounded confident. It was whether the selected source, filters and totals matched the query we expected. The at-completion estimate also used the current activity snapshot. A date filter did not turn it into a historical estimate. That was another definition worth stating rather than leaving the agent to infer. The writeback table needs rules Our writeback contained more than a tidy list of future values. It included actuals, forecast values, subtotal rows and blank cells. That creates several ways to get a plausible but wrong total. Summing detail and All subtotal rows double-counts hours. Selecting the wrong value column mixes up actuals and forecasts. Treating every blank as zero changes the meaning of missing or unpopulated values. For this table, the query needed to: Select the intended scenario and Hours Forecast value column. Exclude All subtotal rows from project, discipline, year and month. Filter the requested period and aggregate at the required grain. Preserve the distinction between a blank and a recorded zero. Those rules can live in source instructions, but a curated view is easier to reuse when several consumers need the same result. Either way, the agent needs to know what a row means before it starts adding rows together. Ontology adds meaning, not missing data The ontology integration with data agents [2] is in preview. It supplies entities, relationships, definitions and source bindings. The data agent uses that context to choose a source and generate SQL, KQL or DAX; the underlying source executes the query. That separation explained one of our early failures. The agent could see only part of the lakehouse, and the forecast binding pointed elsewhere. Adding business context could not compensate for tables it could not reach. We had to check the actual source configuration and table selection. There was a less obvious problem too. An AI-assisted ontology edit added the forecast entity but repointed a shared connection to the plan database. Other entities still depended on that connection reaching the gold lakehouse. The ontology opened normally; the saved definition exposed the mistake. We corrected the connection and bound ProjectForecast to the shortcut in gold. The lesson was practical: after changing an ontology, inspect the saved bindings and rerun representative questions. A successful edit message is not enough. Shared business meaning belongs in the ontology. Source-specific query guidance belongs with the source. Keeping those responsibilities clear is more useful than repeating the same join rules in every agent prompt. Test the answers, including the expected ones We used the data agent evaluation SDK [3], also in preview, to run question sets and save the results. SQL queries computed expected answers from the sample data, and a grader compared them with the agent responses. That made instruction changes testable. It also exposed a mistake in our test: a planning question used a definition of baseline hours that did not match the semantic model. The apparent agent failure was a disagreement about the expected answer. Query-derived expectations still need review. They stay tied to the data, but a wrong join or business definition will produce the wrong benchmark very consistently. For an ontology comparison, keep the sources, permissions, questions, runtime and expected answers constant. Comparing an agent with access to the full lakehouse against one that can see only a few tables tells you little about the value of the ontology. We also kept answer checking separate from presentation. Inspect the query, result and any generated code, not just the final paragraph. Save the configuration with the run so a later score is comparable. More guidance when needed, analysis after the query Two preview features helped beyond basic question answering. Topics [4] let the agent retrieve relevant sections of longer guidance rather than send all of it with every question. They currently apply to SQL sources on the preview runtime. Rules that always apply still belong in source instructions; detailed definitions can be grouped into topics. The run steps show which topic was opened. The code interpreter [5] works on retrieved data in a sandboxed Python environment. In the procurement demo, it produced a chart of late deliveries by supplier quality rating. That was easier to read than a long answer listing percentages. It can also combine results from multiple sources for analysis. That is different from a database query joining those sources, and it still requires compatible keys and grain. We did not treat a chart of synthetic data as evidence that supplier ratings caused delivery performance. The generated Python and its inputs are part of the answer worth checking. Use the graph for relationships We also materialized part of the ontology as a graph: Supplier, PurchaseOrderLine, WorkOrder, Project and Installation. This made it possible to follow a supplier's order lines to projects and its contractor work orders to installations. It was a useful way to explore connections without writing each join. A connection alone did not prove that a supplier had delayed a project; that still required dates and status filters. The graph projection [6] is in preview and has its own eligibility rules. Our Equipment entity, which also carried Eventhouse telemetry, was not eligible. We kept live readings in the Real-Time Dashboard and operations agent rather than forcing them into the graph. Making the graph work also uncovered a compatibility problem. We changed relationship mappings and some entity keys. Afterwards, that ontology would no longer attach to the data agent, while a copy from before the changes still worked. We used the graph-ready copy for exploration and the earlier copy for the domain agent. We did not isolate which change caused the failure. This was a workaround for our build, not evidence that every solution needs two ontologies. From a sensor reading to a Teams recommendation The operations agent [7] covered a different job: watching a signal rather than waiting for a question. Our preview setup monitored simulated pump telemetry in an Eventhouse. Every five minutes it evaluated peak vibration and alerted when the value crossed above 5 mm/s. We observed the Teams recommendation and used Investigate further to explore the alert. This was a simulated bearing-wear scenario, not a validated fault diagnosis. The threshold identified a condition to investigate. The transition rule mattered. "Crosses above" is not the same as "is above." For a repeatable test, we first let the agent observe a below-threshold window, then introduced the fault. Restarting the stream or leaving an empty window did not guarantee a new crossing. We configured a Power Automate action with human approval, but did not complete end-to-end verification of the approved action's result. Its parameters were fixed for the demo pump, not dynamically mapped for every asset. The confirmed result was the alert and investigation, not an automatically created maintenance request. The client still matters Fabric IQ appearing in several clients does not mean each client can reach every Fabric item in the same way. Microsoft 365 Copilot Chat [8] can use Power BI reports and semantic models. The Copilot Cowork plugin [9] currently reaches semantic models through their reports; it does not directly ground answers in lakehouses, Eventhouses, ontologies or data agents. Existing Power BI permissions and row-level security apply. Publishing a Fabric data agent as a Microsoft 365 agent is a separate integration path [10]. It should not be confused with the Power BI connection. We used Prep data for AI [11], which is in preview, for model-level AI instructions and schema selection. We also covered verified answers for recurring questions. Preparing the model does not make the separate forecast table available to a client that cannot query it. Before choosing a client, check which source it can reach and how the answer is produced. The Fabric IQ name alone does not settle that. What changed We started with an agent that could answer questions about recorded work. We ended with one that could also answer questions about a stored forecast. The change was not a more ambitious prompt. It was a forecast with a defined grain, an accessible table, business context and explicit routing between two different meanings of the same word. That is the useful part of this Fabric IQ build: planning produces values that can be queried alongside operational data. The forecast still needs validation, and the answer still needs checking. But "next year" no longer has to mean "there is no data." Try it yourself: sample data, semantic models and data agent setup on GitHub References (Microsoft Learn, accessed 16 October 2026) [1] Predict forecasts: https://learn.microsoft.com/fabric/iq/plan/planning-forecasting/planning-how-to-generate-statistical-forecasts-using-predict-feature [2] Ontology in a data agent: https://learn.microsoft.com/fabric/data-science/data-agent-ontology-sources [3] Evaluate a data agent: https://learn.microsoft.com/fabric/data-science/evaluate-data-agent [4] Data agent topics: https://learn.microsoft.com/fabric/data-science/data-agent-topics [5] Code interpreter: https://learn.microsoft.com/fabric/data-science/data-agent-code-interpreter [6] Ontology graph: https://learn.microsoft.com/fabric/iq/ontology/how-to-use-ontology-graph [7] Operations agent: https://learn.microsoft.com/fabric/real-time-intelligence/operations-agent [8] Fabric IQ in Copilot Chat: https://learn.microsoft.com/fabric/iq/connectors/microsoft-365-copilot-overview [9] Fabric IQ in Copilot Cowork: https://learn.microsoft.com/fabric/iq/connectors/cowork-overview [10] Data agent in Microsoft 365 Copilot: https://learn.microsoft.com/fabric/data-science/data-agent-microsoft-365-copilot [11] Prep data for AI: https://learn.microsoft.com/power-bi/create-reports/copilot-prepare-data-ai
Production-grade Fabric data agents: moving from Demo to Dependable
Getting a Fabric data agent to work in a demo is one thing. Supporting it once people depend on it is another. Using the Lakeshore Retail demo, this guide covers the practical work behind that transition: reliable data, ontology context, secure access, performance testing, guardrails, and day-two operations. Tutorial data - Lakeshore Retail. The inventory is a CSV reference, not an agent answer; benchmarks are proposed, not measured. 1. Define the service before tuning the prompt Start with verifiable questions: inventory below safety stock, inbound shipments, governed sales measures, and refrigeration readings over a defined window. Agree on definitions, units, time zone, snapshot selection, freshness, and an answer owner. Low stock is not a shortage forecast. Two high-temperature readings do not prove a continuous breach: cadence, gaps, quality, and boundaries matter. Store-level relationships do not identify which product occupied a refrigeration unit. Put durable calculations in governed measures, SQL, or KQL. Ordering and other write actions need separate authorization and approval. Figure 1. Reference design; guardrails do not replace authorization. Use the standard runtime for production. Ontology, preview runtime, Foundry integrations/agent guardrails, evaluation SDK, and data-agent source control have separate preview considerations. The core's GA status does not extend to every dependency. 2. Use ontology for meaning, not as a security boundary LakeshoreOntology is now attached to the LakeshoreDataAgent draft, with Standard selected. It supplies entities, relationships, and mappings. SQL, KQL, or DAX executes against underlying sources under user permissions. Figure 2. Tutorial ontology, not proof of query correctness. The integration is preview. Attachment here does not confirm a service-wide fix to the documented new-experience issue. Keep shared definitions in the ontology. Explicitly mount eligible underlying sources before adding source-specific instructions, descriptions, or examples. These settings do not write back to the ontology. 3. Establish one reference answer For the fixed 2026-08-28T18:50:00Z snapshot, select positions where: LocationType = STORE StorageClass = FROZEN OnHandQuantity < SafetyStockQuantity The 48-position inventory fixture has exactly three matches, all Vanilla Bean Ice Cream (P-FRZ-001): Figure 3. A calculation from tutorial files, not a live agent result. For test INV-001, assert exact Store/Product keys, count, quantities, filters, and timestamp, not just three city names. The fixture's InventoryStatus = 1 marks low stock; sample instructions call it healthy. Resolve this first; the reference uses quantities. Do not silently alter data or treat August as current. 4. Prepare and wire the sources Curate objects without losing keys, joins, dates, or measures. Smaller is not automatically better. Use governed Power BI measures, Prep for AI, and Verified Answers. DAX generation uses model metadata and Prep for AI, not agent instructions. SQL/KQL source instructions/examples are separate. AI scoping and hidden fields are not authorization. Column flags do not narrow a selected source table; use supported model configuration or curated objects, not destructive edits to shared models. In development: Check prerequisites: supported paid F2+/P1+ capacity, tenant settings, and same-region agent/source capacities. Use Sources > Add sources > Add an ontology to attach LakeshoreOntology. Review its entities and Ontology data sources. Mount eligible sources for SQL/KQL configuration; prepare the semantic model separately. Set routing, clarification, and response rules, including units, identifiers, and honest failures. Inspect the fixed inventory question's source, generated query, and result. Publish to a restricted test audience; repeat as real consumers. Figure 4. The supplied attached-source draft. It does not establish query, security, or performance success. Validate the published version; republish runtime changes. Standard does not freeze service-managed LLM upgrades. Model Read permits semantic-model queries; authoring needs broader rights. 5. Benchmark the whole request Schema width and preview runtime are experiments, not guaranteed gains. Measure authentication, planning, query generation, source execution, response generation, and wrapper/guardrail work. Fast engine execution does not establish that capacity will fix conversation latency. Figure 5. Proposed experiments, not benchmark results. Vary one factor before combinations: Import, Direct Lake, and DirectQuery have no universal speed ranking. Filter telemetry early. Advanced DAX value indexing currently needs Q&A enabled. Fix questions, data, identities, and versions. Separate new chats/follow-ups and cold/warm runs; interleave baseline/candidates. Never clear shared production caches for a test. Use approved synthetic load with representative volume, cardinality, skew, joins, and freshness; tutorial data cannot prove sizing. Record sample size, p50/p95, timeouts, failures, throttles, correctness, and resources. Count all attempted eligible requests when reporting correct completion within budget, including timeouts. Leave caching off until permission-aware keys, freshness, expiration, and revocation invalidation are proven. 6. Add Foundry without moving the authorization boundary A prompt/hosted agent can use Fabric for orchestration, safety, and escalation, not repair measures, grant access, or replace Fabric's query-generation model. Use defense in depth: Application: authenticate, allowlist tools/arguments, bound calls/concurrency/deadlines, and validate available structured results. Foundry: supported safety controls and orchestration. Fabric and sources: enforce the identity's permissions. Data/semantic layers: business meaning. Use delegated end-user identity for interactive access; test two differently entitled consumers through the whole route. Never silently use the owner's identity. The Fabric tool and Fabric IQ integration differ; the former requires the same tenant. Deployment-principal support does not establish query-principal support. The public published-agent MCP pattern is: https://api.fabric.microsoft.com/v1/mcp/workspaces/{workspaceId}/dataagents/{dataAgentId}/agent Verify OAuth, consent, and networking; private-link routes may need workspace-specific hosts. Keep tokens out of code/logs. Moderation is route-specific, not automatic for arbitrary MCP bridges or nested calls. Test input, tool calls, retrieved content, and output on your route. Safety approval does not prove accuracy or authorization. For hosted guardrails, prove filtering with negative tests. invocations needs invocations_moderation; invocations_ws has no content-safety moderation. Agent-level groundedness/spotlighting are unsupported. Implement and test any missing required control, or block release. Review Code Interpreter/action tools for isolation, retention, and approval. Exclude preview dependencies if policy requires it. 7. Test behavior, not convincing prose Use approved references and holdouts separate from tuning. Assert numbers, dates, filters, and authorized scope deterministically. Test ambiguity, no data, outages, expired identities, follow-up filters, threshold boundaries, gaps, injection, denied fields, and cross-user isolation. Check totals/errors for disclosures too. Test consumers: workspace Admin, Member, and Contributor bypass RLS. Admin success is not an RLS pass. The conversational cap is 25 rows by 25 columns. Compute totals in the source, not from truncated output; offer an authorized report/export route for complete datasets. The preview evaluation SDK uses question/expected_answer columns and a published target. Review detailed steps alongside assertions. Judge scores cannot override failed access checks. Example targets: all critical cases pass; broader correctness >=95%; no disclosures observed in the negative suite; 95% of attempted eligible requests correct within 30 seconds at five concurrent users. These are proposals, not measurements or guarantees. 8. Release a service someone can operate Figure 6. Proposed release gates, not an existing pipeline. Separate dev/test/prod. Version configurations, ontology/models, AI preparation, examples, policies, code, and evaluations. Data-agent source control is preview. Verify target bindings, permissions, region, and freshness. Publish and smoke-test consumers before expanding access. Practice rollback to a known-good release; it does not necessarily restore data, permissions, tools, or the service-managed model. Regress after service changes. Assign business, data, application, security, and operations owners. Monitor outcomes, p95, throttling, freshness, quality, and cost with correlation IDs/versions. Restrict trace access and retention; never log tokens or unrestricted results. Bound transient retries, honor Retry-After, and never retry denials as outages. Add deadlines, circuit breakers, and escalation. Distinguish no matches, no access, and failed queries. Review compliance/geographic boundaries for Foundry, external services, logs, and exports. Fabric's up-to-28-day history retention does not define external retention. Respect Purview/source policies. AI consumption includes instructions, examples, and history; source queries consume separately. Monitor AI Query and engines, budget Foundry/application costs, and compare cost per correct, authorized answer within target. Require evidence for scope, data quality, published bindings, consumer access, correctness, guardrails, load, rollback, ownership, and honest failures. Missing evidence blocks release. Dependable means useful within scope and supportable when things change.
