Fabric March 2026 Feature Summary
Welcome to the Fabric March 2026 Feature Summary—and welcome to FabCon! As we kick off FabCon, this update captures the momentum we’re seeing across the Fabric platform and the conversations happening with customers and partners right now. March brings a wide range of enhancements across governance, data engineering, real-time intelligence, data science, extensibility, and AI—all designed to help teams build, operate, and scale end‑to‑end data solutions with confidence. Many of the capabilities highlighted here reflect direct feedback from the community and real‑world usage we’ve learned from—including insights shared leading up to (and during) FabCon. We are eager to share what’s new and to continue the conversation throughout the week. If you haven’t already, check out Arun Ulag’s hero blog “FabCon and SQLCon 2026: Unifying databases and Fabric on a single, complete platform” for a complete look at all of our FabCon and SQLCon announcements across both Fabric and our database offerings. Contents https://youtu.be/xhrSMNNX5ho?si=aTWiAMq0PvtTxej8 Events and Announcements Don’t miss the next Monthly Data Days Sessions On March 26 we have a special edition of Fabric Data Days featuring two topics. Join us at 8 AM Pacific for a session on getting started with Fabric IQ. Then at 3 PM Pacific we’ll discuss mapping and spatial analytics in Fabric. Register now! Couldn't make it to Atlanta or just want more FabCon + SQLCon? Join us in Barcelona this September. FabCon Europe is happening again in 2026. Mark your calendars for September 28 – October 1, 2026. Register now to access Super Early Bird pricing! Fabric Platform OneLake Catalog Govern for admins (Generally Available) In today’s data-driven world, effective data governance is crucial to ensure the integrity, security, and usability of data. OneLake catalog is available for Fabric admins, providing tools and insights to govern and secure data estates within Fabric in one place. Figure: OneLake Catalog: Govern for Admin view Figure: OneLake Catalog: Govern for admin—view more report. OneLake Catalog search API and MCP tool (Preview) OneLake Catalog’s Search API brings cross-workspace discovery to code. Instead of traversing workspace-by-workspace and “listing everything,” a single search request can locate matching items across your accessible estate based on catalog metadata and the user’s permissions. Search is designed to help even when the exact name isn’t known. Free-text matching includes the item’s display name and description, so a keyword you remember is often enough to find the right entry. Results can be filtered by the item's type to narrow down the scope of your search. The set of supported metadata signals and filters is expected to grow, enabling richer and more targeted discovery scenarios. The catalog search capability is also included as a built-in tool in the Fabric Core MCP server so AI agents can reliably locate the right Fabric asset as part of a broader workflow, then continue with follow‑up actions using other tools. Workspace tags (Generally Available) Fabric tags add meaningful metadata so people can find the right content faster and organize it consistently. That capability is available for workspaces. Workspace tags add shared context (like team, project, or cost center) at the workspace level, helping teams discover and manage workspaces more efficiently, while also enabling scalable governance through APIs. Figure: Workspace settings screen showing applied workspace tags Workspace tags are built on the existing Fabric tags model: tags are defined once, then applied on items and workspaces. Workspace admins can apply and remove tags in workspace settings, making it easy to add shared context at the workspace level. A workspace can have up to 10 tags applied. Workspace tags are surfaced in key discovery experiences so they’re usable in day-to-day navigation: Workspaces can be filtered by tags both in the workspaces list and in OneLake Catalog Explorer, a tags indicator also appears in the workspaces list and in OneLake Catalog Explorer next to every tagged workspace. Tag names are shown on the workspace screen itself, making the workspace context immediately visible. Tagging can be retrieved and managed at scale using APIs, enabling consistent application and reporting across workspaces. Data loss prevention policies for Fabric—Extending restrict access to structured data in OneLake (Preview) When handling sensitive data, it might be challenging to find the right balance between federating data and keeping it secure and compliant. Data Loss Prevention (DLP) policies enable organizations to detect sensitive data and surface it to users and admins when it is found. The Restrict Access action allows you to restrict access to your data once the sensitive information is detected. DLP Restrict Access reduces the risk of exposure to unauthorized users, without slowing analytics or collaboration. Customers can scale Fabric with confidence, meeting compliance requirements while enabling secure, enterprise-wide data sharing. With this release, you’ll be able to apply access restrictions through DLP on: Warehouses KQL databases SQL databases Lakehouses (previously supported) Semantic models (previously supported) Cosmos DB and mirrored databases are coming soon. Figure: Fabric warehouse with a restrict access indication Admins can ensure that sensitive data is protected consistently wherever it lives and however it is accessed. Learn more about restrict access in DLP. Lakehouse Signals in IRM (Generally Available) Microsoft Purview Insider Risk Management cross-references millions of signals across all your products, to create comprehensive profiles of potentially unethical behavior inside your organization. Using Lakehouse indicators in Insider Risk Management enables security teams to detect and investigate risky data activity in OneLake with greater precision and context. By incorporating Fabric Lakehouse signals directly into insider risk policies, security teams can correlate data access and movement with DLP, labeling, and audit signals in a single investigation experience—reducing blind spots and accelerating response to potential data exfiltration or misuse. This provides stronger protection for high‑value analytics data while maintaining built‑in privacy controls and avoiding the operational overhead of deploying separate monitoring tools. Figure: Lakehouse indicators used within the IRM tool Learn more about Fabric indicators in Insider Risk Management. Quick policy for data theft for Fabric (Generally Available) A new quick policy for the Data Theft rule is available for Fabric. This streamlined experience makes it easier to set up protection against data exfiltration scenarios, helping security teams take action faster when sensitive Fabric data is at risk. Learn more about IRM quick policies. Insider Risk Management PAYG Usage Report (Generally Available) The Microsoft Purview Insider Risk Management pay-as-you-go feature usage report is designed to provide transparency to customers, enabling more accurate budget planning and policy tuning. IRM admins can check the distribution of PAYG processing units billed across workloads (Fabric), sub-workloads (Power BI, Lakehouse), and indicators (downloading Power BI reports, etc.) to fine-tune their policies and plan PAYG budgets accordingly. Figure: Pay-as-you-go Usage Report Purview DSPM for AI for Fabric Copilots and data agents (Preview) As AI adoption accelerates, organizations need built‑in protections to keep data safe. With Purview Data Security Posture Management (DSPM) for AI, customers gain visibility and control over AI interactions. DSPM for AI helps teams spot sensitive data risks in AI prompts and responses, identify risky AI behavior, and apply consistent governance using familiar tools like DSPM, Insider Risk Management, Audit, and eDiscovery—so organizations can move faster with AI, without compromising security or compliance. bric_Copilot Figure: Purview DSPM for AI report showing Data Agent interaction in Fabric Learn more about DSPM for Fabric Copilots. Branched workspace with Git integration (Preview) Branched workspace is a new developer experience designed to simplify how teams work with feature workspaces during a branch‑out flow. With clearer visual cues and richer context, developers can easily understand workspace relationships and work more confidently when branching and iterating on features. This feature will be released by the end of March 2026. Figure: Fabric workspace tree showing the new relation between workspace and branched workspace Follow our new Git developer experiences in Microsoft Fabric (Preview) announcement. Selective branching with Git integration (Preview) Fabric Git Integration Branch-out with selective branching introduces a more focused branch‑out experience in Fabric. Developers can select only the items they need for a feature, reducing clutter in the target workspace, improving reliability, and accelerating time‑to‑code. By working with a smaller, purpose‑built workspace, developers can iterate faster and with greater confidence. Figure: Branch-out selective branching dialog Follow our new Git developer experiences in Microsoft Fabric (Preview) blog announcement. Compare code changes with Git integration (Preview) The new compare code changes experience helps developers confidently sync their Fabric workspace with a connected Git branch by clearly showing what changed before taking action. It provides a familiar code‑compare experience that highlights the exact differences since the last sync—down to the item and file level—whether the change originated in the workspace or in the repository. This makes it easier to review updates, understand their impact, and resolve conflicts by comparing workspace and Git versions side by side before committing, updating, or undoing changes. Figure: Git Integration code compares changes dialog Follow our new Git developer experiences in Microsoft Fabric (Preview) blog announcement. Connection reference item type in Variable Library (Preview) The new connection reference item type in Variable Library introduces a new way to manage external data connections in Microsoft Fabric. This new variable type lets you reference existing connections—such as Azure SQL or Snowflake—by storing a connection ID in the Variable Library, instead of embedding static connection strings in code. Figure: Variable Library “connection reference” item type option Connection reference variables work seamlessly with CI/CD and Git, enable safer environment‑specific configuration across dev, test, and prod, and ensure only authorized connections can be selected through the UI. This makes it easier to build, deploy, and manage Fabric solutions with cleaner configuration, stronger governance, and improved portability across CI/CD stages. Bulk import and export items definition APIs (Preview) These APIs enable you to programmatically export, import, and synchronize Fabric item definitions across workspaces at scale—all through the Fabric REST API. Every Fabric item—whether it’s a Notebook, Report, Semantic Model, Data Pipeline, or KQL Dashboard—has an underlying item definition: a portable schema containing the item’s full configuration and content (encoded in Base64). The Import & Export Batch APIs let you: Export item definitions individually or in bulk from any workspace Import (create) items from definitions into a target workspace Update existing item definitions in-place for continuous deployment List & paginate through all items in a workspace for batch operations Key scenarios Workspace migration: Moving items across workspaces, tenants, or regions is one of the most common requests from Fabric customers. The batch APIs let you export all items from a source workspace into a portable JSON manifest, then import them into any target workspace. This is invaluable for replicating environments across different tenants and cloning a production workspace for testing purposes. CI/CD and DevOps integration: To support enterprise DevOps practices in Microsoft Fabric, organizations can integrate the new Bulk Export and Import APIs into their CI/CD pipelines. Fabric item definitions can be treated as code—exported and versioned in Git using Fabric Git Integration or the bulk-export API, validated through pull request workflows, and promoted through a well-defined release process. When deploying across workspaces, the bulk-import API enables consistent, automated promotion into test and production environments using the underlying Fabric dependency logic that creates new items in the correct order, retains the original relations, and updates existing ones in place. Metadata backup and recovery: Schedule periodic batch exports to capture the full state of your workspace as versioned JSON manifests. Store them in Azure Blob Storage, a Git repository, or any durable storage. If something goes wrong, re-import the manifest to restore your workspace to a known-good state. Metadata scanning and lineage analysis: Tools that analyze report definitions to discover data lineage—such as which semantic model columns are used in each report—can extract hundreds of report definitions in bulk instead of one at a time, reducing scan time.Export (read) operationsMethod Endpoint Description POST /workspaces/{workspaceId}/items/bulkExportDefinitions?beta=true Export an item’s full definition as Base64-encoded parts. May return 202 for LRO. Import (write) operationsMethod Endpoint Description POST /workspaces/{workspaceId}/items/bulkImportDefinitions?beta=true Create/Update an existing item’s definition in-place. Ideal for CI/CD sync. Resources Full announcement: Bulk import/export items definition APIs API documentation: Fabric Items API Reference Comprehensive guide: Item Management Overview Item definition structure and formats: Item Definition Reference How to handle async operations with polling: Long-Running Operations Guide App registration and authentication setup: Microsoft Entra ID Documentation CI/CD tutorial using Bulk API: CI/CD tutorial using the Bulk Export and Import APIs Fabric CLI v1.5—Power BI Scenarios, CI/CD Deployments, and DX Improvements The Fabric CLI v1.5 is the most scenario-driven update yet. Power BI developers can now trigger semantic model refreshes, rebind reports, and script end-to-end deployment workflows—all from the terminal, without portal context-switching. The release also adds a new deploy command for CI/CD, interactive REPL mode, JMESPath filtering, notebook export in multiple formats, Python 3.13 support, and expanded coverage for Fabric items. Many of these improvements are community-contributed, making the CLI a comprehensive open-source automation surface for Fabric. CI/CD deployments from the CLI—deploy workspaces in One Command A new deploy command integrates the Fabric CI/CD Python library directly into the Fabric CLI, enabling full workspace deployments—including item rebinding and configuration—from a single command. Teams can run deployments from their terminal, GitHub Actions, or Azure DevOps pipelines. Combined with Service Principal authentication and federated credentials for GitHub OIDC, this enables zero-touch, Git-based promotion workflows that fit modern DevOps practices—no custom scripts or additional tools required. For usage examples, refer to the CI/CD examples and setup guide. Fabric CLI as an Execution Layer for AI Agents Fabric CLI is designed to work well with AI agents. A structured agent instructions file and a dedicated Fabric CLI Skill provide AI assistants like GitHub Copilot and Claude with the context they need to generate correct CLI commands from natural language. Improved error messages with actionable guidance help agents self-correct, and the interactive REPL mode enables persistent terminal sessions for multi-step agent workflows. Using a CLI as the execution layer for AI agents is an emerging industry pattern—instead of agents calling raw REST APIs (which require extensive token-heavy context about endpoints, auth, and payloads), agents issue concise CLI commands that encapsulate that complexity, making AI-driven Fabric automation more practical and reliable. Learn more with Fabric CLI agent docs and AI assets on GitHub. Fabric Remote MCP Server: AI agents operate directly in your Fabric environment Fabric Remote MCP is a cloud-hosted MCP server that allows AI agents to perform real operations in your Fabric environment—create workspaces, manage permissions, work with item definitions, and more. No local installation is required. Agents authenticate via Entra ID and operate within your existing RBAC boundaries, with every tool invocation recorded in audit logs. The preview launches with capabilities spanning workspace management, item CRUD and definitions, and permission management. It works with any MCP-compatible client, including GitHub Copilot, Cursor, and Claude Desktop. Learn more in this blog post: Introducing Fabric MCP (Preview). Fabric MCP AI code assistants (Generally Available) The Fabric Local MCP is an open-source MCP server that runs on your machine. This solution integrates AI coding assistants with the comprehensive Fabric API, offering OpenAPI specifications, best-practice guidelines, item definition schemas, and example payloads to enable agents to produce precise, production-ready code while minimizing errors. OneLake tools enable live file operations including upload, download, table inspection, and item creation. This update introduces integrated authentication, automatic retry, production SLAs, and telemetry. Install via npx Microsoft/fabric-mcp in any MCP-compatible client—it works with VS Code, Claude Desktop, Cursor, and more. Fabric Local MCP on GitHub.Fabric Fabric Extensibility Extensibility (Generally Available) After six months in preview mode, gathering feedback, resolving bugs, and strengthening the platform, we’ve reached the next milestone. Partners and customers can build, validate, and publish custom Fabric workloads to production with full Microsoft support. Key highlights: All core capabilities are stable and supported: OneLake storage, native item lifecycle, Entra token acquisition, iFrame relaxation, Workload Hub publishing. The Starter Kit ships with production-ready UI components (ItemEditor, WizardControl, OneLakeView, and more) that reduce time to first workload. GitHub Copilot integration and a new DevContainer/GitHub Codespaces setup reduce setup effort—no local machine required. The first Fabric Extensibility Community Contest drew strong community participation, with real workloads already appearing in the Workload Hub. Learn more about Fabric Extensibility (Generally Available). CI/CD & remote support (Preview) Three new features further enhance the professional development experience for Fabric workloads. CI/CD Support Workload items are first-class citizens in Fabric's CI/CD platform. Items participate in Git integration and Deployment Pipelines with no custom tooling. Variable Library support means items automatically pick up workspace-specific configuration (e.g., the right Lakehouse reference) when promoted across dev, test, and production—no hard-coded IDs, no manual reconfiguration. Figure: CICD enablement for Hello World Sample Variable Library Support Items can be read from Fabric's Variable Library, allowing workspace-specific configuration (e.g., the right Lakehouse reference) to resolve automatically when an item is promoted across dev, test, and production stages—no hard-coded IDs, no manual reconfiguration, and no custom deployment hook logic required. An opt-in webhook that fires whenever a workload item is created, updated, or deleted—regardless of whether it happened through the UI, the REST API, or a CI/CD pipeline. It’s designed for licensing checks, infrastructure provisioning, and external system synchronization. There’s no impact on workloads that don't register an endpoint. Figure: Variable Picker within Fabric Cloud Shell Item Remote lifecycle notification API Workloads are no longer just passive objects sitting in a workspace. The Remote Lifecycle Notification API is an opt-in capability—there is no requirement to use it. If your workload does not need backend notifications, you simply don't register an endpoint, and everything works exactly as before. Fabric Scheduler / Remote Jobs This feature allows workload items to expose named job types that users can schedule directly from Fabric. When a scheduled job fires, Fabric calls a registered endpoint on your workload backend—passing along the item context and a delegated user token. For all these features, you’ll find samples in the Toolkit Starter Kit. Learn more about Fabric Extensibility CI/CD and remote capabilities in this blog post. What's new in workload management As the Fabric extensibility ecosystem grows, with partners publishing workloads and organizations building custom solutions, managing workloads at scale demands more than a single settings page. IT admins need centralized governance and a clear overview of what's being used across the organization, and workspace teams need self-service agility. Three key workload management features for Microsoft Fabric Extensibility will launch by April 1, 2026: Workload admin portal (Generally available) Add workload to workspace (Generally available) Workload management admin APIs (Preview) These will enhance governance through portal, API, and self-service capabilities. Admin portal: centralized admin workload overview (Generally Available) The Fabric Admin Portal now includes a dedicated Manage Workloads tab, a single pane of glass for workload governance across your organization. Centralized workload visibility: view all workloads available for assignment in your tenant in a single centralized view, including status information and workload details. Tenant assignment controls: manage workload assignment at the tenant and workspace level. Add workload to workspace (Generally available) The workspace-level workload assignment was previously introduced in Preview. It allows workspace admins to add workloads directly to one or more workspaces. How it works (Workspace admins): Navigate to the Workload Hub from the left menu in Microsoft Fabric or from your workspace settings. Browse or search for the workload you want to add. Select "Add Workload" and select "To Workspace" from the dropdown. Select your workspaces: search, check the workspaces you want, and use "View more/less" to manage the list. Select "Add": the workload is immediately available in your selected workspaces. Workload Management Admin APIs: Overview and Control at Scale (Preview) Capabilities For Fabric admins who need a programmatic view of their workload landscape, the new Workload Management Admin APIs provide governance and oversight across the tenant through a REST interface. List all workloads: view all workloads available to be added in the tenant, and view which workloads were added. List all workload assignments in the tenant. Drill down into a specific workload and view where it was added (tenant, workspace, capacity). Manage workload assignments (add or remove) to capacities, workspaces, and tenant. Self-service workload publishing (Generally Available) A frequent question from ISV partners using the Microsoft Fabric Extensibility Toolkit is: "How do I get started publishing?" Key features Self-service workload publishing is expected to be generally available by the end of March 2026. ISV partners will be able to publish workloads directly to selected customer tenants for private preview without requiring a manual submission request. This can accelerate time to market and support faster iteration with customers. Self-Service Workload Publishing gives ISV partners full control over their private preview journey: Publish to up to 20 customer tenants: share your workload with selected customers for testing and validation, no Microsoft certification required. Workload name reservation: reserve your globally unique workload name (e.g., Contoso.DataQuality) to protect your brand identity before formal publication. Automated validation: your workload package is automatically validated against manifest schema, naming conventions, and security requirements at upload time. Seamless path to general availability: once validated with customers, use the same workload package to pursue formal certification and publish to the global Workload Hub. OneLake Third-party support for OneLake security This month, we announced third‑party support for OneLake security, taking an important step toward interoperable data security. As customers increasingly build lake‑first architectures on open formats like Delta and Iceberg, they expect the freedom to use multiple analytics engines without copying data or redefining security. OneLake security addresses this need by enabling security to be defined once and enforced consistently wherever data is accessed. At the core of this capability is the authorized engine model. Security policies—including role‑based permissions, row‑level security (RLS), and column‑level security (CLS)—are centrally defined and managed in OneLake, while enforcement happens at query time inside the engine reading the data. Authorized third‑party engines securely retrieve the relevant metadata and effective security definitions through OneLake APIs and apply them during query execution. This ensures users see only the rows and columns they are permitted to access, while OneLake remains the single source of truth for access control. To support adoption, we’ve published implementation guidance and setup documentation for both engine builders and users. The APIs are designed to be engine-agnostic and easy to integrate by providing pre-computed effective access definitions. Looking ahead, we’ll continue evolving OneLake security APIs, including adding support for bitmap-based RLS enforcement. With this release, data vendors can integrate directly with OneLake security, customers can maintain a single security model, and users gain the flexibility to query OneLake data using the engines of their choice. OneLake file explorer (Generally Available) You can easily access and organize all your OneLake data from Windows using the OneLake file explorer. The file explorer lets you browse every workspace and data asset, and upload, download, or edit these files using the same familiar experience as OneDrive. By bringing data lakes into the Windows file system, the file explorer makes enterprise data more accessible for business users. Data Engineering Fabric Runtime 2.0 (Preview) Fabric Runtime 2.0 (Preview) is a next-generation runtime that is purpose-built for large-scale data computations in Microsoft Fabric and introduces key features and components that enable scalable analytics and advanced workloads. Apache Spark: 4.0 Components include Operating System: Azure Linux 3.0 (Mariner 3.0) Java: 21 Scala: 2.13 Python: 3.12 Delta Lake: 4.0 This screenshot demonstrates how you can switch to Runtime 2.0 at the Workspace settings and the Environment levels. Figure: Change runtime at the workspace settings level Explore the full documentation and start using Runtime 2.0 in Fabric. Custom Live Pools for Fabric Data Engineering Modern data engineering workloads are rarely one‑size‑fits‑all. Teams often need predictable performance, isolated resources, or customized configurations for critical production pipelines and high‑value interactive development. At the same time, Spark session startup times can degrade in real-world enterprise environments, especially when users have custom library dependencies. Workspaces or tenants are secured with Private Links or Managed Private Endpoints. In these scenarios, Spark clusters must be created on demand within strict network boundaries, and libraries need to be resolved and installed dynamically, adding noticeable startup latency. Custom Live Pools address this challenge by introducing dedicated, long‑lived Spark pools that stay warm inside your network boundary and come preconfigured with the required dependencies. With Custom Live Pools, Fabric Data Engineering now enables you to: Create dedicated Spark pools and schedule them tailored to your workload needs. Reduce session startup overhead by keeping pools warm with libraries preinstalled. Run reliably within Managed VNets and Private Link–enabled environments. ompute_configuration_panel_within_a_data_analytics_platfo Figure: Animated GIF demonstrating the setup of custom live pools in an Environment Because these pools are already provisioned within the workspace’s network boundary and fully initialized with dependencies, users can start working immediately, without paying the repeated cost of cluster spin‑up and library installation. Custom Live Pools are ideal for: Production pipelines that require consistent SLAs. High‑value interactive notebooks used by data developers. Teams operating in secure or regulated environments. How to set up a Custom Live Pool: Navigate to your Compute tab in your Environment. Select Spark pool and enable the option for Live Pool. Specify the Schedule, Time period of inactivity, and Retrigger frequency. Job concurrency and queue monitoring experience for Fabric Data Engineering As organizations scale their Fabric usage, understanding what’s running, what’s queued, and why becomes essential. The new job concurrency and queue monitoring experience delivers deep visibility into Spark workload execution across your environment. View active, queued, and completed jobs in a single place. Understand why jobs are queued and how concurrency limits are applied. Identify bottlenecks caused by capacity or concurrency constraints. Make informed decisions to tune workload scheduling and resource allocation. Figure: GIF demonstrating the new job concurrency and queue monitoring view in the Data Engineering/Science Spark settings page of Workspace settings Accessing workspace monitoring To view concurrency and queue signals for your specific workspace: Navigate to Workspace settings. Select Data Engineering/Science > Spark settings. Select Jobs to view the live view of your workspace level Spark queue and concurrency. Resource Profiles for Fabric Data Engineering Modern data engineering teams shouldn’t need to be Spark experts to get great performance. With Resource Profiles in Fabric Data Engineering, users simply describe what they’re trying to do, and Fabric automatically recommends the optimal compute configuration. Figure: GIF demonstrating the new resource profiles experience in workspace settings Simple inputs, smart recommendations Instead of tuning dozens of Spark settings, users provide a few high‑level workload details through an intuitive UI: Primary use case, such as a specific medallion layer (Bronze, Silver, or Gold) or task‑based optimization (read‑heavy or write‑heavy workloads). Typical data volume. Data characteristics, such as whether input data contains many small files. Maximum capacity units (CU) for the Spark pool. Once these inputs are provided, users select Get recommendation, and Fabric automatically generates an optimized configuration tailored to that workload. Based on the inputs shown above, Fabric recommends: The appropriate Resource profile. Node family and size. Autoscale and dynamic executor settings. Optimized Spark driver and executor cores and memory. A compatible runtime version. All recommendations are derived from proven best practices and internal performance tuning, removing guesswork and trial‑and‑error. Where to configure Users can enable and manage Resource Profiles from workspace settings: Go to Workspace settings > Data Engineering and Data Science > Resource optimization. Select or edit the optimized profile for the workspace. Rerun the Optimize for your use case flow as workloads evolve. Apply consistent configurations across all Spark workloads in the workspace. Once configured, all notebooks and pipeline‑triggered Spark jobs inherit these optimized settings automatically, without requiring per‑notebook configuration. Why this matters This experience enables: Performance by default: optimized compute without manual tuning. Consistency: the same performance characteristics across users and jobs. Better price‑performance: right‑sized resources aligned to workload intent. Lower operational overhead: fewer tuning cycles and support escalations. As workloads change over time, teams can simply revisit the optimization flow, update a few inputs, and let Fabric adapt the configuration—without rewriting code or Spark settings. Figure: Introduction to Resource Profiles Experience Figure: Recommendations generated based on user inputs To learn more about the Resource Profiles experience in Microsoft Fabric Data Engineering, refer to the Microsoft Learn documentation. Installing libraries with Quick mode in Spark Environment (Preview) Managing libraries shouldn’t slow down your development workflow. In Microsoft Fabric Environments, we’re introducing a more efficient way to iterate on libraries while keeping production workloads stable and reliable. Fabric Environments now support two complementary library installation modes that you can use side by side: Quick mode: a fast, on-demand installation path designed for development and experimentation, where libraries are installed when a notebook runs. This avoids heavy processing during the environment publishing and significantly reduces publish time and notebook startup latency when you’re iterating on lightweight or frequently changing dependencies. Full mode: a snapshot‑based installation path optimized for production workloads and pipelines, where libraries are fully resolved, validated against the Spark runtime, and published as a stable snapshot to ensure consistency and reproducibility. Figure: Add libraries in Quick mode and Full mode This new feature lets you move faster during development without compromising production stability. You can keep your core, production‑ready libraries in the snapshot‑based mode, while using the on‑demand path to quickly test new packages or iterate on custom libraries, all within the same Environment. Dynamic session sharing limit up to 50 for high concurrency Fabric High Concurrency Spark sessions enable both interactive exploration and large‑scale, pipeline‑driven notebook execution, supporting parallel, scheduled, and event‑driven workloads at enterprise scale. Customers often achieve higher density by packing notebooks into a shared High Concurrency (HC) session using session tags, effectively fitting up to five notebooks per session to control startup overhead and cost. While effective, this approach relies on static limits and manual tuning. With this update, Fabric Data Engineering allows the maximum number of notebooks attached to a High Concurrency session to be increased up to 50, enabling dynamic session sharing at much higher scale. Where to set the configuration You can set the configuration in the Environment item that your notebooks or pipeline‑triggered notebooks use: Go to Workspace → Environments Select the Environment attached to your notebook or pipeline Open Spark Properties Add the High Concurrency configuration Set spark.highConcurrency.max to a value between 2 - 50 Note: This update does not change the default limit of five. This enables: Interactive notebooks, used for exploratory analysis and collaboration. Notebook jobs triggered by pipelines, running in parallel within shared HC sessions. Dynamic adjustment of session sharing limits based on workload intensity, cost, and price‑performance goals. By increasing the session sharing limit, customers can: Improve session acquisition times during peak load. Increase notebook density without fragmenting sessions. Tune concurrency to match workload demand rather than fixed defaults. Achieve better price‑performance efficiency while preserving isolation and fairness across jobs. To learn more about increasing your session sharing limit in High Concurrency mode, please check out Microsoft Learn documentation. Data export settings for notebooks With data export settings for notebooks, Microsoft Fabric empowers administrators with explicit, tenant-level control over how data leaves notebooks. This feature helps ensure that interactive analytics do not inadvertently become channels for data exfiltration. Administrators can restrict the downloading of notebooks, preventing files that may contain sensitive data, credentials, or proprietary logic from leaving the environment. Additionally, they can disable downloads of rich output content, such as table results generated from DataFrames, within the notebook experience. By managing these controls, Fabric admins can effectively prevent unintended data exfiltration from interactive notebook workflows and consistently enforce security and compliance policies across all workspaces and teams. Figure: New data export tenant setting for notebooks Figure: New data export tenant setting enabled What users experience when downloads are blocked When an administrator blocks data export: The Download option is removed from the notebook UI. Users can no longer download notebook files or rich output content generated from DataFrames in the notebook experience. Interactive exploration continues in‑place, but data cannot be extracted outside Fabric through the notebook UI. Figure: Notebook with download controls disabled due to tenant-level enforcement This ensures that notebooks remain a secure analysis surface, rather than a data export mechanism, without disrupting day‑to‑day exploration inside the platform. Why this matters Notebooks often contain more than just code: Embedded datasets Derived analytical results Business logic Confidential insights By controlling export behavior at the platform level, Fabric helps organizations: Reduce risk of accidental data leakage. Meet regulatory and audit requirements. Standardize governance across teams and regions. Data Export Settings for Notebooks reinforce Fabric’s commitment to secure‑by‑default analytics, enabling powerful interactive experiences without compromising enterprise security posture. Session starts insights into Fabric Data Engineering Fast session startup is critical for interactive analytics, and Fabric’s Starter Pools are designed to deliver Spark sessions in ~5 seconds by default. However, when that target isn’t met, users have historically had little visibility into why. Session Start Insights closes that gap by making session acquisition transparent, debuggable, and actionable. Why sessions don’t always start in five seconds In practice, session startup delays are almost always driven by user‑side configurations, not platform regressions. Common causes include: Custom compute configurations that prevent reuse of pre‑warmed Starter Pools. Pre‑installed libraries or environment dependencies that require cluster customization. Managed VNets or private networking that force isolated cluster provisioning. Unexpected high regional demand triggering fallback to on‑demand clusters. What Session Start Insights delivers Previously, users could see that a session was “starting,” but not what was happening under the hood. With this feature, Fabric surfaces clear, explicit reasons for session startup behavior directly in the product experience: Whether the session was served from a Starter Pool or required an on‑demand cluster. The exact reason a fast‑path session could not be used (for example, libraries, networking, or custom configs). Where time was spent during session acquisition. Using the session detail view to diagnose delays Navigate to the notebook’s session status or monitoring pane. Open Session Details for the active or recent session. Figure: Notebook with Session Details option Review the delay reason and session source (Starter Pool vs. on‑demand) Figure: Notebook with Session Details pane showing session start details This makes it immediately clear whether the delay was: Expected due to configuration choices Related to libraries or networking Learn more about session start insights in the Microsoft Learn documentation. Z-order and liquid clustering support in the Native Execution Engine With the Native Execution Engine, Fabric Data Engineering continues to raise the bar on price‑performance leadership for large‑scale analytics. Beyond execution‑time optimizations, the engine now includes native support for Z‑Order and Liquid Clustering, allowing advanced data layout techniques to fully benefit from vectorized, C++‑based execution paths. This ensures that storage‑level optimizations and execution‑level acceleration work together, delivering compounding performance gains for real‑world analytical workloads. Why this matters Modern analytical queries frequently: Filter on multiple high‑cardinality columns. Scan large Delta tables repeatedly. Rely on selective predicates to narrow down results. Without intelligent data layout, even a highly optimized execution engine can spend unnecessary time scanning data. By combining the Native Execution Engine with Z‑Order and Liquid Clustering, Fabric ensures that: Related data is co located on disk, enabling aggressive file and row‑group skipping. Queries scan fewer files and fewer bytes. CPU‑efficient native operators are paired with I/O‑efficient data access. On a one‑billion‑row dataset, internal benchmarks comparing fallback execution versus Native Execution Engine with clustering showed: 20–32 seconds absolute runtime reduction per query. Roughly 20%–27% improvement across multiple clustered column combinations. Performance gains observed consistently across different predicate shapes and data distributions. This brings a compounding performance effect: faster scans, fewer CPU cycles, and lower cost per query, without requiring users to rewrite Spark code or change query semantics. This helps deliver strong price-performance for analytics workloads. How users enable and use this 1. Enable the Native Execution Engine Users must first ensure that the Native Execution Engine is enabled for their Spark workloads (at the workspace, environment, or session level). Once enabled, supported Delta operations automatically run through native execution paths. 2. Use Z‑Order or Liquid Clustering on Delta tables Users can apply clustering using standard Delta Lake commands: Define Liquid Clustering at table creation or apply it to existing unpartitioned tables Use OPTIMIZE … ZORDER BY for multi‑column access patterns To learn more about the Z-Order and Liquid Clustering support or Native engine, refer to the Microsoft Learn documentation. Copilot for data engineering and data science Microsoft Fabric notebooks now include a context-aware Copilot experience designed to support you across the full notebook lifecycle. By automatically understanding your workspace environment—including attached Lakehouses, notebook structure, and runtime behavior—Copilot provides assistance that stays aligned with how your notebook is built and executed. It’s easy to get started with no session startup required. Choose the Copilot icon on the toolbar to open the chat panel. Copilot can help accelerate notebook development by generating and refining code, explaining unfamiliar logic, and assisting with larger notebook workflows. For more complex tasks, Copilot can first propose a plan and then help implement it across the notebook, allowing you to move from idea to working solution more quickly. Copilot also improves the troubleshooting experience when notebook executions fail. Instead of navigating long stack traces or ambiguous error messages, you can use Copilot to analyze failures, identify likely root causes, and review suggested fixes directly within the notebook. Figure: Fix with Copilot provides error summary and suggested fixes Throughout this process, built-in guardrails ensure you remain in control. Copilot suggestions are transparent, and proposed code changes can be reviewed before being applied. Together, these capabilities help teams reduce development friction, resolve issues faster, and build more reliable data workflows. Try the new Copilot experience today. To learn more, visit the Copilot for Data Engineering and Data Science documentation. Fabric notebook custom agent inside VS Code The Fabric notebook custom agent is a Fabric-native AI development agent embedded in the Fabric Data Engineering VS Code extension. It helps data engineers build, debug, and publish Microsoft Fabric notebooks and Spark workloads. Unlike generic coding assistants, this agent operates with full awareness of the Microsoft Fabric workspace, runtime, environments, and Lakehouse resources. It ensures every action—code generation, execution, artifact management, and publishing—is context-aware, validated, and safe for enterprise environments. Prior to the introduction of the Fabric Notebook custom agent within the VS Code extension, there were notable limitations in how language models understood and interacted with the Microsoft Fabric environment. For instance, when users provided a prompt such as "read the parquet file from the current default Lakehouse and save it to a delta table," the language model was unable to interpret what was meant by "default Lakehouse." As a result, it would generate standard Spark code without leveraging the built-in spark variable available within the notebook, which is essential for initializing and managing Spark sessions in the Fabric environment. With this new agent, the following code will be generated and ready to run. # Read parquet file from default lakehouse df = spark.read.parquet("Files/green_tripdata_2022-08.parquet") # Write to delta table in dbo schema df.write.mode("overwrite").format("delta").saveAsTable("dbo.raw_green_tripdata_202208") This custom agent should be automatically activated once the Notebook is open. Figure: Fabric notebook custom agent For more detail, refer to the Author notebook inside VS Code documentation. Tenant switching inside Fabric Data Engineering VS Code extension ISV and partners often collaborate with multiple end customers, each typically operating within their own dedicated Microsoft Fabric tenant. To address this need for flexibility, the Fabric Data Engineering VS Code extension now enables tenant switching. With this enhancement, ISVs and partners can easily transition between different customer projects within the same VS Code window, eliminating the need for repeated sign-in processes. This streamlined experience simplifies managing multiple projects and improves overall productivity for professionals working across diverse customer environments. To switch to a different tenant, select the currently signed-in Fabric user in the status bar and pick the target tenant from the list. Figure: Switch Fabric tenant inside VS Code Enable new kernels inside Fabric Data Engineering VS Code extension Users can now run Fabric notebooks within VS Code using a variety of new kernels. Previously, running notebooks required users to specify the language of each cell using cell magic commands and rely on PySpark as the execution environment. With this enhancement, three additional kernels have been introduced, allowing users to select their preferred programming language directly at the kernel level. This eliminates the need for cell magic commands and streamlines the process, enabling notebooks to be executed in Python, Scala, or Spark SQL natively within VS Code. Choose Microsoft Fabric Runtime from the top-level kernel list. The available languages then appear in the second panel. Figure: Microsoft Fabric Runtime entry Figure: Supported Fabric notebook languages in VS Code For more detail, please refer to the documentation Author notebook inside VS Code. Support for multiple schedules in Fabric materialized lake views MLVs now support multiple named schedules per lakehouse. Previously, all MLVs shared a single schedule, and teams needing different refresh timings resorted to notebook-triggered refreshes. This workaround bypasses dependency management, centralized error reporting, and retry logic; failures can persist for weeks undetected. Each named schedule now targets a specific subset of views. A finance pipeline can refresh hourly while an analytics pipeline runs every six hours, with no scripting required. When a schedule fires, Fabric refreshes upstream dependencies in order, runs independent views in parallel, surfaces errors centrally, and skips overlapping runs. Figure: Schedules panel for materialized lake views, showing configured schedules and available actions For more information, refer to the Schedule a materialized lake view run documentation. PySpark support for Fabric materialized lake views (Preview) MLVs now support PySpark authoring (Preview), letting data engineers create, refresh, and replace MLVs from Fabric notebooks using the DataFrameWriter API. Previously, teams wrote definitions in Spark SQL, which made custom cleansing logic, UDFs for business rules, and procedural transformations harder to express. With PySpark authoring, MLVs gain access to the entire Python ecosystem. A gold-layer MLV can score transactions against a fraud detection model, standardize addresses using a geocoding library, or validate records against external regulatory rules. All existing MLV capabilities, including data quality constraints, table properties, and scheduled refreshes, work identically with PySpark-authored definitions. Full refresh only today; optimal refresh is coming soon. For more information, refer to the PySpark reference for materialized lake views (Preview) documentation. Move data from source to Lakehouse in a few moves using Copy job Getting data into your Lakehouse should be straightforward. For many customers, the first interaction with Microsoft Fabric begins right after creating a Lakehouse and selecting Get data. With this update, Copy job appears at the top of the Get data experience in Lakehouse, making it a more discoverable way to bring data into Fabric. Whether you’re onboarding your first dataset or scaling ingestion across multiple sources, Copy job can help you move data with minimal setup so you can focus on insights instead of configuration. Fabric notebooks now support lakehouse auto‑binding when used with Git, making notebooks far more portable across environments such as dev, test, and prod. Instead of hard‑binding a notebook to a specific lakehouse, Fabric automatically resolves the correct lakehouse as the notebook moves across Git‑connected workspaces, reducing manual rebinding and environment‑specific fixes. This feature is opt‑in and must be enabled from the notebook settings page. Once enabled, it applies to all lakehouses referenced in the notebook, including the default and any additional lakehouses. The configuration is stored in a system‑managed notebook-settings .JSON file in the Git repo, which should not be edited manually. Overall, lakehouse auto‑binding helps teams focus on versioning notebook logic while keeping data and environment configuration cleanly separated. Try it out in just a few steps: Create or open a Lakehouse. Select the Get data dropdown in the ribbon. Figure: Start ingesting data into a Lakehouse directly from the Get data dropdown using Copy job Select New Copy Job You’ll be redirected to the Copy Job experience, where you can choose the source data you want to ingest from. In just a few clicks, your data is copied into the Lakehouse and ready for exploration, analysis, and downstream analytics. Learn more: What is Copy Job in Data Factory – Microsoft Fabric Notebook supports Lakehouses auto-binding in Git Fabric notebooks now support lakehouse auto-binding when used with Git flow, making notebooks more portable across environments such as dev, test, and prod. Instead of hard-binding a notebook to a specific lakehouse in the original workspace, auto-binding lets the notebook automatically resolve the linked lakehouse as it moves across Git-connected workspaces. This reduces manual rebinding and environment-specific fixes. This feature is opt‑in and must be enabled from the notebook settings page. Once enabled, it applies to all lakehouses referenced in the notebook, including the default and any additional lakehouses. Figure: Entry of auto-binding setting in notebook The configuration is stored in a system‑managed ‘notebook-settings .json’ file in the Git repo. Overall, lakehouse auto‑binding helps teams focus on versioning notebook logic while keeping data and environment configuration cleanly managed. Notebook Resources Folder Support in Git Notebook projects often depend on more than just notebook code—such as reusable Python modules, configuration files, or small supporting assets. Fabric notebooks now support committing the built‑in Resources folder to Git, enabling true end‑to‑end source control for notebook‑based projects. These resources are versioned alongside the notebook and automatically restored during Git sync. To support real‑world workflows, this feature includes fine‑grained controls. Teams can define Git exclusion rules or use standard .gitignore files inside the Resources built in folder to avoid tracking large files, temporary assets, generated outputs, or test data. Figure: Define resources in git settings in notebook The feature is disabled by default to ensure safe adoption and does not introduce noticeable performance impact during commit or sync. The support for Environment resources folder, deployment pipelines, and public APIs is coming soon. Learn more: Notebook source control and deployment - Microsoft Fabric Fabric notebook public APIs (Generally Available) Fabric Notebook Public APIs enable notebooks to be managed and executed programmatically as first‑class assets. The APIs provide full CRUD support—enabling teams to create, update, list, and delete notebooks at scale—making them ideal for CI/CD and automated environment management. In addition, notebooks can be executed on demand via the Job Scheduler API. You can parameterize notebook runs, customize session configuration, specify environments and lakehouses, monitor execution status, and cancel runs if needed. Secure service principal authentication is also supported. A key enhancement is the ability for notebook runs to return exit values, enabling conditional branching and richer orchestration in pipelines. Together, these APIs unlock seamless integration with Fabric pipelines, external schedulers, and enterprise automation platforms. Learn more: Items - REST API (Core) and Job Scheduler - REST API (Core). Improved Copilot completion for Fabric notebooks We’re introducing upgraded Copilot completion in Fabric notebooks to deliver a faster, more accurate, and more intuitive coding experience. With this update, auto-completion is closer to what developers expect from VS Code‑style inline suggestions, helping you stay in flow while writing notebook code.You can enable the feature from the Copilot completion button in the notebook status bar. It supports both Python and PySpark notebooks. Figure: How to enable copilot completion A More Natural, Inline Coding Experience The upgraded auto‑completion is designed to work inline as you type, offering context‑aware code suggestions that better match your intent. Whether you’re writing Python logic, data transformations, or helper functions, Copilot now provides suggestions that feel more predictable, relevant, and easy to accept—reducing friction compared to earlier experiences. Faster and More Responsive Performance has been a key focus of this upgrade. Auto‑completion now responds more quickly, reducing latency between keystrokes and suggestions. This makes Copilot feel less intrusive and more like a natural extension of the editor, especially during rapid iteration or exploratory development. Higher‑Quality Suggestions That Fit Notebook Workflows Beyond speed, the quality of suggestions has improved. Copilot is better at understanding notebook context, including surrounding cells and in‑progress code, resulting in completions that require less manual editing. The goal is simple: help you write correct, readable code with fewer interruptions and less back‑and‑forth. Designed for Everyday Notebook Development This upgraded auto‑completion brings Fabric notebooks closer to the editing experience developers are already familiar with, while remaining optimized for data engineering and analytics workflows. Learn more by exploring Develop, execute, and manage notebooks - Microsoft Fabric. Create files in the notebook resources folder Fabric notebooks now let you create and manage files directly in the built‑in Resources folder, making it easier to develop and maintain notebook dependencies. You can create and edit Python modules, configuration files, and other lightweight assets alongside your notebook code and use them directly within the notebook. Figure: Entry of creating new file in notebook resources folder To learn more, refer to How to use notebooks - Microsoft Fabric. Data Science and AI Fabric data agents (Generally Available) Data sources: Build and consume data agents on a broad set of data sources, including Lakehouse, Warehouse, semantic models, Eventhouse, SQL databases, and mirrored databases. Configurations: Configure data agents using agent-level instructions, data source–specific instructions, and example queries to tailor behavior to your scenarios. Publish and share: Publishing and sharing data agents within Microsoft Fabric is generally available, making it easier to operationalize and collaborate on data agents. Figure: End-to-end data management workflow from creation through consumption This release also includes diagnostic, Git integration, and deployment pipelines as part of Microsoft Fabric’s Application Lifecycle Management (ALM) capabilities, enabling troubleshooting and lifecycle management of your agents! Advanced security and governance in data agents (Preview) Data agents in Microsoft Fabric (Preview) include capabilities that elevate security and governance standards. Through integration with Microsoft Purview, organizations gain access to comprehensive auditing, eDiscovery, data lifecycle management, communications compliance, and classification. These tools capture prompt and response telemetry along with user context, supporting enterprise protection and regulatory compliance. Additionally, we are introducing outbound access protection support for the Data Agent artifact to help mitigate sensitive data exfiltration risks and adhere to strict security policies at the individual workspace level. With these updates, organizations can monitor, control, and safeguard all data agent interactions more effectively. Data source enhancements for data agents (Preview) The latest preview brings significant enhancements to data agent source capabilities in Microsoft Fabric. Users can now connect Graph as a data source, enabling them to model and analyze complex relationships within their data for richer, AI-driven insights. Additionally, support for KQL user-defined functions (UDFs) and SQL functions is available, allowing for more sophisticated and efficient querying in KQL- and SQL-enabled sources. These enhancements make data agents more flexible and powerful, supporting faster analytics and expanded scenario coverage. Insider Risk Management PAYG Usage Report (Generally Available) Multimodal support for AI functions in Fabric enables notebook users to apply AI capabilities directly to their unstructured data—including PDFs, images, and text files. With just a few lines of code, users can perform tasks such as summarization, classification, sentiment analysis, and more, all within their existing workflows. This capability is designed to work across both pandas and Spark, making it easy to bring AI-driven insights to a wide range of data science and analytics scenarios in Fabric. Figure: Load files into a table or classify insurance claim images with multimodal AI functions Check out the multimodal AI functions documentation to learn more. AutoML in Fabric (Generally Available) AutoML delivers a fully production-ready, low-code machine learning experience. In addition to the core AutoML capabilities powered by FLAML—automated model selection, feature engineering, and hyperparameter optimization—the end-to-end UI experience, making it easy to configure experiments, monitor training progress, compare models, and deploy the best performer directly from the interface. With integrated experiment tracking, reproducibility, and seamless deployment workflows, teams can confidently move from raw data to high-quality predictive models faster—while maintaining transparency, governance, and control within Fabric. Figure: AutoML includes the fully integrated UI experience for configuring experiments, tracking model performance, and deploying models end to end Check out the AutoML in Fabric documentation to learn more. Data Warehouse Fabric Data Warehouse recovery (Preview) In fast moving production environments, an item gets dropped accidentally due to an incorrect script. Suddenly, critical reports are broken, and teams are asking the same question: “How fast can we recover?” With dropped warehouse recovery in Microsoft Fabric, a deleted warehouse no longer means starting over. You can now restore a dropped warehouse together with everything that goes with it—data, schemas, snapshots, permissions, and saved queries—in minutes, without rebuilds, re‑ingestion, or complex restore workflows. Figure: Warehouse recovery in action—from drop to restore in minutes There’s no need to recreate environments, rerun pipelines, or scramble through backups. Recovery is simple, predictable, and designed to bring your warehouse back exactly as it was before the drop. This capability is built for the realities of modern analytics: rapid iteration, frequent deployments, and shared production environments. Instead of turning accidental deletes into prolonged outages, Fabric makes recovery a routine, low-stress operation. No panic. No need to rebuild. Just built in resilience—designed for real-world production analytics. To learn more, refer to the manage workspaces documentation. Alerts and actions Microsoft Fabric Data Warehouse provides operational intelligence closer to the data by integrating SQL queries with Fabric Activator rules. Traditionally, identifying an issue in query results is only the first step. Teams then need separate systems or manually follow up to notify the right people and act. With this integration, Data Warehouse makes it possible to define rules directly from SQL query outputs, so changes in data can trigger alerts and downstream actions automatically. This unlocks a simpler way to monitor business-critical conditions using familiar SQL workflows. Teams can create queries that detect scenarios such as SLA risks, failed processes, unusual trends, or threshold breaches, then attach rules that evaluate results continuously and respond in real time. The result is a more proactive analytics experience, where insights move beyond the warehouse and are acted on immediately. Figure: Create rules on SQL query results to detect data issues, monitor KPIs, and automatically trigger alerts or Fabric workflows. Analyze unstructured text using T-SQL AI functions (Preview) Microsoft Fabric Data Warehouse extends modern analytics beyond structured and semi‑structured data by introducing built‑in AI functions for working directly with the unstructured text. Traditionally, processing free‑form content such as notes, logs, or comments requires external services or complex pipelines. With these new capabilities, Fabric Data Warehouse enables text extraction, classification, sentiment analysis, and transformation directly in T‑SQL language, allowing data engineers and analysts to keep AI‑driven text processing inside the warehouse. The new AI functions simplify common text analytics scenarios using familiar SQL patterns. You can extract structured insights from unstructured text, analyze sentiment in feedback or messages, and classify content such as application logs or incident reports using contextual understanding rather than fragile rules or expressions. Fabric Data Warehouse also supports text transformation scenarios, including summarization, grammar correction, and translation, making it easier to standardize and enrich text data as part of existing data preparation workflows. The following visual is example of processing unstructured text in the Comments table: Figure: Analyzing comment text with AI functions This query enriches each user's comment by determining its sentiment; labeling the type of feedback or intent; and extracting key discussion signals such as the main topic, user intent, and any requested action using built-in AI functions. For advanced scenarios, Fabric Data Warehouse enables custom prompt-based processing through a generic ai_generate_response(instructions, text) function. This function allows teams to define precise transformation or extraction rules as prompts; apply domain specific logic; and reuse AI behavior consistently across queries and pipelines. Together, these capabilities significantly broaden the scope of data warehousing scenarios supported in Fabric, unlocking new ways to analyze and operationalize unstructured text using T-SQL. Refer to AI functions in Fabric Data Warehouse to learn more about analyzing text with built-in AI functions. ANY_VALUE aggregate Fabric Data Warehouse provides the ANY_VALUE() aggregate, which lets you return an arbitrary value from each group in T-SQL query. This is especially useful when you need to group results by a key (such as GeographyID) but you still want to project descriptive attributes (such as city name, and country) that are functionally the same for every row in that group. An example of such a query is illustrated in the following picture, demonstrating how ANY_VALUE() aggregate can be used to get the values from the group that are not changing. Figure: Using ANY_VALUE() to project descriptive columns while aggregating trips by GeographyID In this pattern, city, state, and country don’t add meaning to the aggregation because they’re constant for a given GeographyID. Adding these columns in the GROUP BY clause or applying more complex or costly aggregates like MIN or MAX is unnecessary overhead and makes queries harder to read and maintain. ANY_VALUE keeps the grouping logic minimal and the intent clear: aggregate by the key and simply carry through the descriptive columns. Refer to the ANY_VALUE function in Fabric Data Warehouse documentation to find additional scenarios where it helps simplify grouping and aggregation logic. Fabric warehouse custom SQL pools (Preview) Custom SQL pools for Fabric Data Warehouse gives administrators finer-grained control over how SQL compute resources are allocated across workloads. Custom SQL pools build on the warehouse’s autonomous workload management by letting you define your own isolation boundaries, explicitly assign resources, and route queries based on application context. With Custom SQL Pools, you can create multiple isolated SQL pools within a single workspace and allocate a percentage of available compute to each. Queries are routed to the appropriate pool ensuring that critical workloads get the resources they need without being impacted by other activities in the warehouse. Figure: Custom SQL Pool Configuration Key benefits include: Predictable performance for critical workloads—Reserve compute for business‑critical reporting or dashboards, so they aren’t disrupted by ad‑hoc queries or background processing. Flexible workload isolation without added complexity—Allocate resources where they matter most without needing to split workloads across multiple workspaces or scale capacity just to protect one workload. Custom SQL Pools are especially useful when multiple applications share a single Fabric warehouse or SQL analytics endpoint and have different performance or priority requirements. As your capacity scales up or down, your pool allocations automatically scale with it, preserving the relative resource distribution you’ve defined. Learn more about custom SQL pools in Fabric Data Warehouse: Custom SQL Pools - Microsoft Fabric | Microsoft Learn. SQL Audit Logs (Generally Available) SQL Audit Logs for Fabric Data Warehouse enable organizations to capture and analyze database activity for security monitoring, compliance, and forensic analysis. Figure: Configuring SQL Audit Logs With this release, we are expanding support and improving accessibility: Support for SQL Analytics Endpoint auditing. Direct access to audit files stored in OneLake, through OneLake Explorer. Ability to download or copy audit files through OneLake Explorer. Ability to open the .xel audit files directly in SQL Server Management Studio (SSMS) for deeper investigation. These capabilities make it easier for security and compliance teams to perform detailed investigations, long-term retention, and external analysis of workflows. For configuration steps and usage details, see the documentation: SQL Audit Logs in Fabric Data Warehouse - Microsoft Fabric | Microsoft Learn COPY INTO and OPENROWSET support for OneLake sources (Generally Available) Previously, this capability supported Lakehouse sources only. With this release, we are expanding support to all OneLake items (except Warehouses). This enables much more flexible ingestion scenarios, including: Using partner workloads such as COPY Jobs. Using staging areas across different Fabric items. Loading data stored anywhere in OneLake-backed items. Customers can now leverage OneLake as a unified staging layer for ingestion workflows while maintaining a consistent SQL experience. Figure: Executing COPY INTO from OneLake sources For full usage examples and configuration guidance, see the documentation: Ingest Data into Your Warehouse Using the COPY Statement - Microsoft Fabric | Microsoft Learn COPY INTO (Transact-SQL) - Azure Synapse Analytics and Microsoft Fabric | Microsoft Learn Outbound Access Protection (OAP) support for Warehouse (Generally Available) Outbound Access Protection for Fabric Data Warehouse provides stronger data exfiltration protection for enterprise environments. Warehouse now supports connector rules that allow organizations to control which external sources the warehouse can access. Customers can define rules to allow access to: Specific Azure Data Lake Storage Gen2 accounts Other Fabric workspaces Approved external connectors This expands the model introduced during Preview, where access was limited to OneLake and local workspace sources only. With connector rules, organizations can enforce controlled and auditable outbound connectivity, helping meet strict governance and compliance requirements. Figure: Supporting OAP Data Connection Polices For details on configuring connector rules, see the documentation: Workspace outbound access protection for data warehouse workloads - Microsoft Fabric | Microsoft Learn Full query text available in Query Insights Query Insights now show the full SQL query text, removing the previous 8,000‑character truncation. The complete query text is available in: The command column of queryinsights.exec_requests_history. The Query Details pane in the Query activity tab. This makes it significantly easier to understand what ran, especially for large, auto‑generated queries from BI tools, ORMs, or complex workloads. You can now retrieve the full query text directly using: SELECT distributed_statement_id, submit_time, total_elapsed_time_ms, command FROM queryinsights.exec_requests_history ORDER BY submit_time DESC; This query takes the most frequently executed query (from Frequently Run Queries) and pulls every historical execution with the full, untruncated SQL text, making it easy to understand exactly what is running and how often: WITH TopQuery AS ( SELECT TOP 1 query_hash FROM queryinsights.frequently_run_queries ORDER BY number_of_runs DESC ) SELECT erh.query_hash, erh.distributed_statement_id, erh.submit_time, erh.total_elapsed_time_ms, erh.status, erh.allocated_cpu_time_ms, erh.data_scanned_remote_storage_mb, erh.command AS full_query_text FROM queryinsights.exec_requests_history AS erh JOIN TopQuery AS tq ON erh.query_hash = tq.query_hash ORDER BY erh.submit_time DESC; Because the full statement is preserved, users can: Immediately understand what logic was executed, not just which query ran. Compare query text across executions to detect subtle changes or regressions. Correlate performance issues with specific joins, filters, or aggregations. This enhancement removes a major gap between observability and action, making Query Insights a more complete tool for day-to-day production troubleshooting. Live connectivity in Migration Assistant for Fabric Data Warehouse (Preview) The live connectivity in Migration Assistant for Fabric Data Warehouse lets you migrate object metadata by connecting directly to your source system into a new Fabric warehouse. This helps you accelerate migration and reduce upfront prep by eliminating the need to generate and upload a DACPAC for the metadata step. The object metadata of schemas, tables, views, functions, and stored procedures gets migrated to warehouse. Figure: Migration using direct connection to the source system Learn more about Migrate with a Direct Connection. Simplify data access with data sources (Generally Available) Fabric Data Warehouse lets you define external data sources that act as named references to locations in your lake (for example, a root folder of a Fabric Lakehouse or an Azure Storage account). External data sources (Preview) were introduced in preview in October 2025, now they are generally available and fully integrated into the Fabric experience, with the full IntelliSense and Copilot support in the SQL query editor. The following visual shows how to create a reusable reference to the Fabric Lakehouse root folder that represents a landing zone where you can store the files before ingesting them in warehouse: Figure: Creating external data source in Fabric Data Warehouse. In the following visual, you can see how you can query files using short, relative paths that are resolved against the data source root: Figure: Accessing files in the referenced data source using the relative path Using a data source keeps queries clean and portable. You can write easy to remember relative paths (like /Files/bronze/logs/*.jsonl) instead of embedding long, environment-specific URLs everywhere in the code. This makes scripts simpler to maintain and easier to share across workspaces and environments. Find more examples about the reading files form the lake in OPENROWSET(BULK) (Transact-SQL) documentation page. Real-Time Intelligence Business Events in Microsoft Fabric (Preview) With Business Events, organizations can move from observing what happened to acting on what matters, in real time. It enables organizations to respond faster, operate more intelligently, and scale real-time decision making across analytics, automation, and AI. You can generate business events from user data functions (UDFs) and notebooks. Once generated, a single business event can power multiple downstream actions, such as: Trigger alerts and automations with Activator, responding immediately via email or Teams. Execute custom logic using user data functions, reacting programmatically to business events. Run analytics and workflows in notebooks, using events to drive downstream analysis. Provide real‑time context with AI and ML, enriching models with governed business signals. Integrate with Spark jobs, dataflows, and Power Automate, enabling distributed processing and business process automation. With Business Events in Real-Time Hub, you can explore, define, and act on critical business signals for the whole organization in a unified experience. Figure: Business Events creation experience For more information about this feature, please refer to the documentation: Business Events in Microsoft Fabric. Building event-driven, real-time applications on database changes with Fabric Eventstreams Deltaflow (Preview) Building intelligent systems that react quickly to operational database changes is simpler with this update. With the release of DeltaFlow, Fabric Eventstreams can seamlessly capture inserts, updates, and deletes from operational databases, transform them from their raw Debezium format, and make them available to downstream event-driven applications using Activator and for real-time analytics in Eventhouse. There’s no need for custom Debezium/JSON processing code or to explicitly manage destination tables through source table schema changes. Easily connect to, ingest from, and transform raw CDC feeds into analytics-ready form. to_connect_to_a_CD Figure: Enabling DeltaFlow when connecting to an Azure SQL database Detect, fetch and register source database & table schemas in the Eventstream schema registry as they evolve. Figure: Automatic registration and use of source table schemas Automatically manage tables in analytics store as they continuously evolve with source schema changes without breaking pipelines. Figure: Automatically created Eventhouse tables with analytics-ready shapes For more information about these features, please refer to the document Building real-time, event-driven applications with Database CDC feeds and Fabric Eventstreams DeltaFlow (Preview). Real-time stream processing with Fabric Eventstreams and Spark notebooks (Preview) This update brings together Fabric Eventstreams and Spark Structured Streaming, making it easier for Spark developers and data engineers to work with real-time data in Microsoft Fabric. These enhancements enable you to access streaming data in Eventstreams directly from Spark notebooks, supporting low-latency processing and end-to-end real-time AI pipelines. Easily discover Eventstreams and real-time sources available through the Real-Time Hub, right from within Fabric notebooks. Figure: Real-Time Hub view inside Fabric notebook—discover Eventstreams in seconds Connect to and process streaming data within minutes using auto-generated PySpark code snippets. Figure: Auto-generated PySpark snippet in a Fabric notebook for an Eventstream Load and use existing notebooks from the Fabric Eventstreams portal. Figure: Load a Spark notebook as an Eventstream destination—reuse and collaborate Securely connect to any Eventstream from a Fabric Spark job without connection strings and secrets, using the enhanced Spark adapter for Eventstreams. Securely connect to any Eventstream from a Fabric Spark notebook using the enhanced Spark adapter—without connection strings or secrets, and with built-in auto-retry support. For more information about these features, please refer to the blog post Bringing Together the world of Real-time Intelligence and Spark Structured Streaming (Preview). Anomaly Detector full-item experience Introducing a refreshed Anomaly Detector full‑item experience that makes it easier to create, run, and explore anomaly detection workflows from end to end. Instead of working through disconnected steps or modal flows, you now get a single, full‑page canvas that brings configuration, analysis, and results together in one place. The updated layout follows Fabric’s shared item experience, so navigation and interactions feel consistent with the rest of the platform. With this new experience, you can more quickly define detection scenarios, run analyses, and immediately dig into detected anomalies without losing context. Clearer sectioning and action cues guide you through the process—from selecting signals and parameters to reviewing anomaly trends and individual data points. Results stay visible alongside configuration, making it easy to iterate, compare outcomes, and refine your setup in real time. Figure: An updated Anomaly Detection UI that simplifies setup, visualizes trends and anomalies in real time, and highlights high‑confidence events for faster investigation The full item Anomaly Detector experience also sets the stage for deeper investigation and richer insights. By consolidating analysis and results into a unified view, you can spend less time navigating and more time understanding what’s changing in your data. Whether you’re monitoring operational metrics or exploring unexpected behavior in real-time signals, these improvements help you move from detection to insight faster and with greater confidence. Operations agent playbook improvements and messages This month sees several improvements for operations agents’ ability to monitor your data and take actions. Based on usage and feedback we’ve heard so far, operations agents are better at mapping between your instructions and the fields in the Eventhouse you connect them to. You’ll also see they can build different types of rules to monitor the specific conditions in your data, including comparing string values in the data and counting data points over time. Finally, you’ll also see better messages if the agent can’t generate a playbook based on the data, goals, and instructions you’ve configured. In those cases, the LLM will try and describe the issue for example not being able to ground a field it inferred from your instructions to a field in the Eventhouse KQL database, or the parameters for a condition or action not being clear in the instructions. For example: This makes it easier for you to debug and unblock the agent configuration. Finally, we’ve updated our best practices and sample for how to give guidance and steering to the operations agent for it to follow your instructions. Learn more about this in the Operations Agent Best Practices and Limitations documentation. Live update for Real-Time Dashboards Real-Time Dashboards now support Live update, a feature that automatically refreshes dashboard visuals when new data is ingested into your underlying data sources. Instead of relying on fixed-interval refresh - which polls your data source on a set schedule regardless of whether new data exists - Live update uses a lightweight background query to detect when data arrives and triggers a refresh only when needed. This event-driven approach offers several benefits. Your dashboards stay current without the compute overhead of constant polling, making it particularly valuable for high-frequency data monitoring scenarios where you need to see data the moment it arrives. For organizations running multiple dashboards or monitoring large data volumes, Live update reduces compute load by eliminating unnecessary refresh cycles during quiet periods. Dashboard viewers also gain flexibility with the ability to pause live updates temporarily. If you're investigating a specific data point and don't want the visuals to change, you can pause updates to analyze the current state without interruption, then resume when you're ready to return to real-time monitoring. Dashboard editors can enable Live update through the dashboard settings, with configuration options including Live update (recommended), manual update only, or a fallback refresh interval for visuals that don't support ingestion detection. Learn more about configuring Live update for your dashboards, with the What is Real-Time Dashboard? documentation. Eventstream SQL Operator (Generally Available) During preview, the Eventstream SQL Operator introduced SQL-based stream processing in Fabric, enabling customers to transform live event data using familiar SQL with rich authoring, preview, and debugging capabilities. Write to multiple destinations from a single SQL operator Consolidate your real-time processing logic into one streamlined SQL block and route results to multiple destinations in a single step. This simplifies pipeline design, making it more efficient and lowers operational overhead. The updated authoring experience makes it easy to add multiple destinations directly within the SQL editor and preview results for each output independently. During testing, dedicated output previews let you validate transformations before you publish. Figure: Route data to multiple destinations from one SQL operator. Event ordering and late event arrival handling Configure event ordering policies directly within the SQL operator to handle late-arriving and out-of-order events. Define thresholds for how long to wait for delayed data and ensure accurate, event-time–correct processing—even in the presence of network delays or asynchronous producers. These policies help build more resilient real‑time pipelines that reflect how data behaves in the real world—not just in perfect conditions. Learn more about Fabric Eventstream SQL Operator. Together, these enhancements make Eventstream SQL Operator more powerful, more intuitive, and ready for production‑grade real‑time workloads. Anomaly Detection as a source in Eventstream Anomaly Detection can be added as a source in Fabric Eventstream, allowing you to publish anomaly events directly into your Eventstream for processing and action. You can add Anomaly Detection as a source from either Eventstream or Real-Time Hub. You can enrich your anomaly events by adding business context and additional information. Further, you may route real-time events to downstream workloads for automated alerting and dashboard visualization. Where to Add This Source You can add Anomaly Detection as a source in two ways: From Eventstream – Create a new Eventstream, select Anomaly detection events as a source. Figure: Adding Anomaly Detection source within Eventstream From Real-Time Hub – Navigate to Real-Time Hub, find the Fabric events and select Anomaly detection events. Figure: Adding Anomaly Detection events in Real-Time Hub Once added, anomaly events flow seamlessly into your Eventstream, ready for transformation and routing to downstream workloads. Get Started Anomalies are now another streaming event—ready to be transformed, enriched, and acted upon. Try out Anomaly Detection as a source in Eventstream today and unlock the power of real-time anomaly pipelines. Learn more about this in the Operations Agent Best Practices and Limitations documentation. Data series colors for real-time dashboard visuals When monitoring operational data, color choices matter. A status indicator showing “Critical” in red and “Healthy” in green communicates meaning instantly—viewers can interpret the visual without reading legends or labels. With data series colors, you can make these intentional choices rather than accepting system defaults. To configure data series colors, switch to Editing mode, select the Edit icon on your tile, and expand the Data series colors section in the Visual tab of the formatting pane. From there, you can select a color for each data series in your visual. Figure: Visual formatting: data series colors setting Figure: Color palette for setting visual elements colors Learn more about customizing your Real-Time Dashboard visuals, refer to the Customize Real-Time Dashboard visuals documentation. Use Copilot to create visuals in real-time dashboards (Preview) Dashboard editors can now use Copilot to create and edit visuals in Real-Time Dashboards using natural language. When you're in Edit mode, open the Copilot pane while creating a new tile or editing an existing one. Describe the insight you need - for example, "Show me the top 10 repositories by push events this week" - and Copilot generates the KQL query, returns the data, and suggests a visual that fits your results. Figure: Real-time dashboard visual in edit mode after Copilot answer has been applied You can accept Copilot's suggestion, refine your question with follow-ups like "Group by event type" or "Filter to the dotnet organization," or edit the query directly. Once you're satisfied, add the visual to your dashboard and use the no-code formatting options to customize its appearance. To learn more, refer to the Copilot-assisted real-time data exploration documentation. Instantly run and preview functions in Microsoft Fabric Eventhouse: no code required (Preview) Previously, working with an Eventhouse function involved manually writing KQL queries. You needed to enter the function name, provide parameters in the right format, and execute the query just to see what results you would get. If you wanted to view the function's body or metadata, you had to run a separate command. That's no longer the case. With the new Preview Functions capability in Microsoft Fabric Eventhouse, you can open the function definition, run the function, and instantly preview its results, with no manual KQL, no parameter guesswork, and no extra commands. Why this matters Eventhouse functions are powerful, but environments evolve. Databases grow, teams change, and you often inherit functions you did not write. Instead of guessing what a function does or manually building a query just to test it, you can: View the function definition instantly. Run the function and preview results with a single click. Test parameterized functions interactively. Browse your function list with search and sorting. This removes friction from everyday workflows. Whether you are exploring unfamiliar logic, validating outputs before building reports, or troubleshooting unexpected results, you get clarity in seconds instead of minutes. How to view or preview a function: In DB Explorer, expand Functions and select a function. A read-only version of the function opens. Select Preview results to instantly run the function and see the output. If the function has parameters, enter your values and preview the results based on your input. The preview shows up to 100 records, providing a quick snapshot of the function’s output. Figure: DB Explorer with the Functions folder expanded and a function selected. The function opens in read-only mode, and the Preview results option is available to run the function and display the output, with fields provided to enter parameters You can view a complete list of all stored functions, including their folder, description, and optional sorting. Built-in search makes it easy to find specific functions, making navigation and discovery simple even in large databases. Figure: Functions list with all available functions in the database, with options to sort, search, and open a menu with additional actions The new Run & Preview Functions feature in Microsoft Fabric Eventhouse lets you instantly inspect function definitions and preview results without writing KQL or handling parameters manually. Quickly explore, test, and manage all your stored functions in one place, saving time and reducing friction. Learn more with the Stored functions list documentation. Workspace monitoring dashboard templates in Microsoft Fabric Eventhouse (Preview) Fabric workspace monitoring provides rich telemetry across your workspace assets, including Eventhouses, Power BI Semantic Models, Data Engineering (GraphQL), and Mirrored Databases. The workspace monitoring data is stored in an Eventhouse, part of Fabric Real-Time Intelligence. To help you turn this data into actionable insights, we have created ready-to-use real-time dashboard templates with out-of-the-box visualizations. Currently, two templates are available: one for Eventhouse items and one for semantic models. From any Workspace Monitoring Eventhouse, users can create these dashboards directly. To get started, go to your Workspace Monitoring Eventhouse, open the upper ribbon, and select Fabric Monitoring. Figure: The ribbon in the Workspace Monitoring Eventhouse allows you to create out-of-the-box dashboards for monitoring From there, choose to create: Eventhouse Monitoring Dashboard—to monitor Eventhouse items in your workspace. You can track: Ingestion results and logs. Commands and queries monitoring. Metrics related to Eventhouse performance. Semantic Model Monitoring Dashboard—to monitor Semantic Models in your workspace. You can use semantic model logs to: Identify periods of high or unusual Analysis Services engine activity by capacity, workspace, report, or user. Analyze query performance and trends, including external DirectQuery operations. Track semantic model refresh durations, overlaps, and processing steps. Monitor custom operations sent using the Premium XMLA endpoint. Once created, the dashboards are ready to use immediately, giving you instant visibility in your workspace. You can also customize them to fit your specific needs. These templates make it fast and easy to track, analyze, and act on workspace activity, all in one place. Learn more in the documentation: Visualize your workspace monitoring . Databases Database Hub in Fabric The Database Hub in Fabric is a new unified database management experience that brings databases across edge, on‑premises, cloud, and Fabric into a single, coherent view. It provides teams with one place to explore, observe, govern, and optimize their entire database estate. Built for scale, the Database Hub uses agent-assisted intelligence to continuously reason over estate-wide signals, surface what changed, explain why it matters, and guide teams toward the right next actions. With built-in observability, delegated governance, and Copilot-powered insights, database agents help teams move from insight to action faster, while humans remain firmly in control of goals, boundaries, and trust. The result is a simpler, more confident way to manage databases at scale today, and a foundation for increasingly autonomous, intelligent database operations over time. Sign up for early access. SQL database in Fabric Since reaching general availability in November 2025, SQL database in Fabric has seen rapid customer adoption as organizations modernize SQL workloads with less operational overhead and tighter integration with analytics and AI. Guided by customer feedback, the platform emphasizes simplicity, autonomy, security, and AI optimization. We are introducing a set of improvements and new features that make it easier to migrate, manage, and optimize SQL workloads in Fabric: Simplified migration with new assistant: The Migration Assistant in public preview helps SQL developers move SQL Server and Azure SQL workloads into Fabric by importing schemas, assessing compatibility, and guiding migration with minimal manual effort. Configurable autonomous management: While maintaining a SaaS-first approach, new options allow database-level control over vCore scaling, expanded compatibility levels, enhanced T-SQL features, and settings that ease application transitions without code changes. Support for all collations: All Azure SQL database collations are supported when creating a new SQL database in Fabric for enhanced global data compatibility and app development flexibility. Collations control text sorting and comparison in SQL databases, impacting filtering, searches, and multilingual content management. Users can specify collations seamlessly during database creation via the REST API across deployment methods. Check out the How to set a different collation for SQL database in Fabric demo and explore the sample code in Git repo. Enhanced data mirroring and security: Auditing and Customer Managed Keys are generally available. CMK for Fabric SQL lets you encrypt databases with your own Azure Key Vault keys to gain full control over key ownership, access, rotation, and compliance. Users can selectively manage which tables are mirrored to OneLake for immediate analytics access. AI and monitoring integration: SQL database in Fabric supports vector search with DiskANN and integrates with Azure AI Foundry for advanced semantic search and AI scenarios, alongside workspace performance dashboards for unified monitoring and optimization. Enhanced data recovery: In Fabric, when a database is deleted, it goes into a soft-deleted state into the Fabric Workspace’s Recycle Bin tab. Depending on the retention configured, the deleted database can be recovered from the Recycle bin while in retention. In addition to this Recycle bin experience, the Fabric SQL database also has the backup retention period configurable from 1-35 days. When the database is hard deleted from the Recycle bin, the backups are still available for the configured backup retention period. This new improvement allows you to restore the backup into a new database to any point in time within the restorable period. Cosmos DB mirroring with Private Link and VNET Cosmos DB mirroring with Private Link and VNET enables customers to mirror data from privately secured Azure Cosmos DB accounts into OneLake. This allows organizations to maintain consistent network security and compliance while supporting near‑real‑time analytics and AI workloads in Microsoft Fabric—strengthening Fabric’s enterprise readiness by design. Figure: Mirroring data from Azure Cosmos DB accounts secured with Private Endpoints or VNETs into OneLake To learn more, refer to the Cosmos DB Fabric Mirroring for Private Networks documentation. Data Factory—Copy Job Richer Change Data Capture (CDC) with Oracle, Fabric DW, and SCD Type 2 Copy job continues to improve the no‑code CDC experience with richer, enterprise‑ready replication patterns. This release introduces a set of enhancements in Copy job in Microsoft Fabric Data Factory that make CDC replication more powerful and easier to use without writing code: Oracle CDC source—Capture changes directly from Oracle databases. Fabric Data Warehouse sink—Replicate CDC data into Fabric Data Warehouse. SCD Type 2—Preserve full history with valid dating, and handle deletes as soft deletes. With built‑in SCD Type 2 and soft delete handling, Copy job automatically preserves every version of a record as it changes over time, instead of overwriting history. This makes it easy to answer point‑in‑time questions, support regulatory audits, and run accurate historical analytics—capabilities that traditionally require complex MERGE logic or custom code. Figure: Enabling SCD Type 2 in Copy job with One Click Learn more in the Change data capture (CDC) in Copy Job documentation. Every row is traceable with built-in audit columns Audit columns are additional metadata columns that Copy job can automatically append to every row it writes to the destination. These columns don't come from your source data—they're generated by the platform to describe the data movement itself. When you enable audit columns in Copy job, each row in your destination table can be enriched with information such as: Audit Column What It Captures Data extraction time The timestamp when the row was extracted from the source by a Copy job run File path The source file path the row was read from (applicable for file-based sources) Workspace ID The Fabric workspace ID where the Copy job resides Copy job ID The unique identifier of the Copy job item Copy job run ID The unique identifier of the specific Copy job execution Copy job name The name of the Copy job that moved the row Lower bound The lower bound value of the incremental window for the current run Upper bound The upper bound value of the incremental window for the current run Custom A user-defined static value—add any additional context your team needs. For example, you can add your source server name here Table: Audit column list With audit columns enabled, you can answer the following questions for any row in your destination table: When was this data extracted? Exact timestamp from when the row was read from the source. Where did it come from? Which file path, which data store. Which job moved it? Which Copy job from which Workspace, which specific run, by name and ID. What was the incremental scope? Lower and upper bounds tell you exactly what slice of data this run covered. No custom code. No expression authoring. Add as many audit columns as you want, and every row in every table your Copy job writes will include this metadata automatically. Figure: Setup audit column in Copy job. Figure: Output on destination data Learn more in What is Copy job in Data Factory - Microsoft Fabric. Workspace Monitoring for Faster, Scalable Troubleshooting As Copy jobs scale from a handful to hundreds, visibility becomes critical. Fabric Workspace Monitoring brings centralized, log‑level observability to Copy job executions, streaming detailed run data into a query able Monitoring Eventhouse inside your workspace. Teams can analyze failures, throughput, duration, and data volumes across all Copy jobs in one place—without inspecting jobs individually. With historical logs, cross‑item correlation, and integration with Data Activator for alerts, Workspace Monitoring helps DataOps teams detect issues earlier and troubleshoot faster at scale. Figure: PBI Report against Fabric Workspace Monitoring metric from Copy job Learn more in Workspace Monitoring for Copy Job in Microsoft Fabric - Microsoft Fabric Boost performance automatically with AutoPartitioning Moving large tables efficiently often requires careful partition tuning—but Copy job now does this automatically. With auto‑partitioning, Copy job detects large datasets and applies an optimal parallel read strategy without any manual configuration. This delivers dramatically higher throughput out of the box, whether you’re copying millions or hundreds of millions of rows. The system adapts dynamically based on data size and source characteristics, ensuring consistent performance across tables while eliminating per‑table tuning effort. Figure: Enabling auto partitioning in Copy job Learn more in What is Copy job in Data Factory More flexible incremental copy with new watermark column types Incremental copy is a core pattern for keeping analytics data up to date—but in real world systems, changes aren’t always tracked with a clean datetime column. To address this, Copy job now supports additional watermark column types, making incremental copy more flexible and applicable across a broader range of source systems Copy job now supports ROWVERSION, Date, and String (interpreted as datetime) watermark columns. This allows you to choose the column that best represents change in your source system, while Copy job continues to automatically manage state, checkpoints, and incremental windows. ROWVERSION enables precise and reliable change tracking in SQL‑based systems, capturing every insert and update without relying on application‑managed timestamps. Date watermark support works seamlessly with common columns like LastUpdatedDate or ModifiedAt, with built‑in delayed extraction to prevent data loss or overlap between runs. String (interpreted as datetime) support removes the need for custom queries or schema changes when timestamps are stored as strings, improving compatibility with real‑world schemas. These enhancements make incremental copy easier to configure, more resilient in production, and better suited for diverse enterprise data models—without adding complexity for users. Learn more in What is Copy job in Data Factory - Microsoft Fabric. Data Factory—Dataflow Gen2 Preview-only steps (Generally Available) This capability improves authoring performance without changing runtime behavior. Preview-only steps let you run specific transformations during data preview only, automatically excluding them from dataflow execution and refresh. That means your production logic stays exactly the same—while the authoring experience becomes faster, smoother, and more responsive. This capability addresses a common challenge when building Dataflow Gen2 items: iterating on logic can be slow when previews must evaluate full datasets. Preview‑only steps make it possible to temporarily reduce data volume or complexity during development, enabling faster validation of transformations without introducing conditional logic or modifying the final query definition. Common uses include filtering or isolating subsets of data to accelerate previews, testing transformation logic without waiting for full evaluation, and exploring new data sources while keeping production execution intact. Because preview‑only steps are ignored during refresh and run operations, they provide a safe way to optimize the authoring workflow without risking unintended changes in published outputs. Figure: The "Enable only in previews" option within the applied steps section Preview‑only steps are also integrated into specific authoring dialogs, including file system views and the Combine files experience. In these contexts, Dataflow Gen2 can automatically introduce preview‑only logic to limit sample data used during preview evaluation, further reducing load time while preserving the behavior of the final dataflow. With general availability, preview‑only steps become a standard part of the Dataflow Gen2 authoring model—helping teams iterate faster, validate transformations more efficiently, and maintain a clear separation between development‑time experimentation and production execution. Learn more: Preview only step in Dataflow Gen2 (Preview) - Microsoft Fabric. Fabric Variable Library integration (Generally Available) Throughout the last couple of months, we’ve removed some of the limitations such as: Variable limit: the previous limit was 50 variables. You can now reference as many variables as you need in your Dataflow. Power Query editor support and using a default value: you can now see how the variable gets evaluated within the actual Power Query editor. Not only that but within the Dataflow Gen2 experience, after enabling the input widgets through the Options menu, you will also have a new way that simplifies how you can reference variables with a complete no code experience fully integrated into the experiences and dialogs that you love from Dataflow Gen2: Figure: The Filter rows dialog within Dataflow Gen2 showing the input widget and the option to Select a workspace variable After selecting this option in any of the dialogs, the experience of selecting a Variable from a library will appear. Figure: The select variable dialog invoked from within a Dataflow Gen2 Be sure to test this improved experience and share your feedback. Learn more: Use Fabric variable libraries in Dataflow Gen2 (Preview) - Microsoft Fabric New data destinations Dataflow Gen2 continues to expand where curated data can land, supporting both lake‑first architectures and hybrid data estates. With new and updated destinations, teams can publish transformed outputs in the formats and platforms that best fit their downstream consumers—whether that’s open lake storage, lakehouse files, enterprise warehouses, or business‑friendly file formats. Azure Data Lake Storage Gen2 (Generally Available) Dataflow Gen2 now supports Azure Data Lake Storage Gen2 (ADLS Gen2) allowing teams to land curated outputs directly into their data lake using open formats and folder structures aligned to organizational standards. This enables lake‑first ingestion patterns for organizations that treat ADLS as their system of record, while still authoring transformations using low‑code Dataflow Gen2 experiences. Common scenarios include reusing curated outputs across Fabric (Spark and SQL) as well as external systems that are read directly from Azure Data Lake Storage Gen2. Figure: ADLS gen2 destination option Lakehouse files (Generally Available) Dataflow Gen2 can write outputs directly into the Files area of a Fabric lakehouse. This is useful when downstream consumers expect file‑based outputs rather than tables, or when teams need to align with existing folder and file conventions inside the lakehouse. This enables patterns where transformed extracts are consumed by Spark notebooks, pipelines, or external tools, while also supporting hybrid designs where some Dataflow outputs are tables and others are files within the same Fabric workspace. Figure: Lakehouse files option Snowflake databases (Preview) This enables transformed outputs to be published directly into Snowflake databases as part of Fabric‑based, low‑code transformation workflows. This supports hybrid data estates where Fabric is used for transformation while Snowflake remains the target platform for analytics or data sharing. This preview helps standardize transformations across platforms and enables analysts to departments to publish governed outputs into Snowflake without duplicating transformation logic. Figure: Snowflake destination option Excel files (Preview) Dataflow Gen2 is introducing the ability to write outputs as Excel files (Preview) for supported filesystem destinations such as SharePoint and ADLS Gen2. This makes it easier to support business processes that still rely on Excel, while keeping transformation logic centralized and governed in Fabric. Typical scenarios include publishing refreshed Excel extracts for operational reporting or legacy workflows, and standardizing Excel output formatting from a single Dataflow definition. Schema support in Fabric data destinations (Generally Available) As Dataflow Gen2 adoption grows, many teams run into organizational challenges when publishing tables into shared destinations. Without schema control, teams often resort to creating separate databases, warehouses, or lakehouses just to keep tables logically grouped—adding complexity and making collaboration harder. With this release, Dataflow Gen2 data destinations now support writing into specific schemas (where applicable). This capability is now generally available for destinations such as Fabric SQL databases, Lakehouses, and Warehouses, giving teams more control over how Dataflow outputs are structured and governed. What’s improved Better organization without extra destinations: Teams can organize tables by domain—such as finance, sales, or HR—using schemas instead of creating separate destinations for each area. This keeps environments cleaner while still enforcing logical separation. Smoother collaboration in shared environments: Multiple teams can publish tables into the same warehouse or SQL database while maintaining clear ownership and structure through schemas. This reduces naming conflicts and supports shared analytics models without friction. Figure: The connection settings for the Warehouse connector using the advanced options to set the Navigate using full hierarchy to true By aligning Dataflow Gen2 outputs with enterprise schema conventions, this enhancement makes it easier to support multiteam data platforms, improve governance, and scale Dataflow Gen2 usage across the organization without restructuring existing destinations. Learn more: Dataflow Gen2 data destinations and managed settings - Microsoft Fabric AI-Powered Prompt Transform (Generally Available) Fabric AI Prompt is integrating generative AI features into the low-code data transformation process. Authors can enrich and transform data using natural language prompts without building or managing machine learning models, while staying within the Dataflow Gen2 execution model. The AI Prompt capability is accessed from the Add column experience, where authors define a prompt and select columns to provide contextual input. This allows AI-driven enrichment to be expressed inline alongside existing Power Query transformations, keeping logic centralized and auditable. Figure: The AI Prompt dialog in Dataflow Gen2 Moving forward, all operations associated to the usage of AI Prompt within Dataflow Gen2 will be accounted towards an explicit AI meter with the operation name of “AI Functions”. Learn more: Fabric AI Prompt in Dataflow Gen2 (Preview) - Microsoft Fabric Publish experience UX + performance improvements (parallelized query validations) Publishing complex Dataflow Gen2 items can be time‑consuming, especially when dataflows contain many queries or multiple destinations. In these cases, validations are required before a dataflow can be published, and waiting for those checks to complete often slows down iteration and troubleshooting. With this release, we’ve improved the Dataflow Gen2 publish experience through a refreshed user interface and performance enhancements that parallelize query validations. By running validations concurrently, publish operations complete faster and surface issues earlier in the process. What’s improved Less time waiting: Dataflows with multiple queries and destinations publish faster, reducing idle time during validation and helping teams move through development and testing more efficiently. Clearer guidance during publication: Validation results are available sooner, making it easier to identify and resolve issues without repeated publish attempts or back‑and‑forth edits. Together, these improvements shorten the publish cycle, reduce friction when working with larger dataflows, and help teams iterate on Dataflow Gen2 solutions with more predictable and responsive feedback. Learn more: Dataflow Gen2 with CI/CD and Git integration. Save As Improvements: Scheduled Refresh Policies and Public APIs Save As continues to improve the migration experience to Dataflow Gen2 (CICD), especially for teams moving large numbers of dataflows across workspaces or tenants. One common challenge during migration is preserving refresh behavior—copied dataflows often require manual reconfiguration before they are production ready. With this release, Save As now supports Scheduled Refresh Policies for Dataflows Gen1, ensuring that refresh configurations are retained when copying a dataflow. This reduces post migration cleanup and helps teams move faster with fewer manual steps. In addition, we’re introducing a new public Save As API for Dataflows Gen1 designed for automation and bulk operations. This enables organizations to programmatically copy dataflows at scale, making it easier to support structured migration plans and repeatable rollout processes. What’s improved Streamlined refresh configurations: Dataflows created using Save As can now inherit scheduled refresh policies from the source dataflow, helping ensure consistent refresh behavior without re‑authoring schedules after migration. Automation at scale: The new Save As public API enables automated and bulk copy scenarios, allowing teams to migrate many Gen1 dataflows to Gen2 programmatically. This is particularly useful for multi‑workspace and multi‑tenant deployments where manual migration isn’t practical. Together, these enhancements reduce migration friction, minimize manual edits, and help teams adopt Dataflow Gen2 more efficiently—whether migrating a handful of dataflows or rolling out Gen2 at enterprise scale. Figure: Dialogs for the refresh and scheduling mechanism when using the Save as experience for Dataflow Gen2 Learn more: Save As Dataflow Gen2 documentation and public Save As API reference. SharePoint site picker in Modern Get Data and Data destinations (Preview) SharePoint Site Picker replaces manual URL entry with a browsable dropdown, so you can select the right SharePoint site directly instead of finding and pasting URLs Why this matters Eliminates manual URL copy-and-paste and context switching. Reduces connection errors caused by wrong URL formats. Surfaces Recent sites and Favorite sites instantly on dropdown open and enable you to search to find sites. Where the experience is available SharePoint site picker is available for SharePoint sources of Get Data in Dataflow Gen2, Pipelines, Copy Job, and Lakehouse shortcuts, and as a destination in Dataflow Gen2. How to use SharePoint Site Picker Simplified SharePoint Site Selection: Instead of copying URLs manually, use the Site URL dropdown to choose from Recent sites and Favorite sites. Figure: SharePoint Site Picker dropdown Quick Search Capability: Find related sites faster by typing in the dropdown search box. Once you select a site, you can load data into the Power Query editor for transformation. Figure: SharePoint Site Picker searched for results Learn more about SharePoint folder connector, SharePoint list connector, and SharePoint online list. Diagnostics download (Preview) Dataflow Gen2 diagnostics download provides a simple way to collect logs and diagnostic artifacts for both cloud-based and VNET gateway dataflows. Instead of rerunning refreshes or guessing at failures, you can download the information needed to investigate issues directly. This helps teams fail faster and fix issues sooner. Downloadable diagnostics make it easier to identify refresh failures, performance bottlenecks, and connectivity problems, including complex networking scenarios that rely on VNET data gateways. Figure: The recent runs dialog showing the new button at the bottom left of the dialog to Download detailed logs With clearer signals available upfront, support investigations are shorter and operational friction is reduced. Learn more: An overview of refresh history and monitoring for dataflows. Advanced Edit for destinations (Preview) The new Advanced Edit experience for Data Destinations enables editing of the underlying M logic that configures destination settings. This unlocks deeper customization, including the ability to leverage parameters to drive destination behavior—an important step for teams standardizing deployments across environments. Parameter-driven destinations: switch target schema/table, file paths, or naming conventions without rewriting queries. Unblock advanced scenarios that require destination settings not yet available in the simplified UI. Figure: The new Advanced editor for data destinations Learn more: Advanced edit for data destination queries in Dataflow Gen2 - Microsoft Fabric. Data destination validations during publish (Preview) Dataflow Gen2 now validates data destinations during publication, helping catch common issues earlier in the development cycle. These validations surface problems such as missing permissions, invalid destination settings, or naming conflicts before the first refresh runs. By shifting these checks to publish time, authors get clear, actionable errors sooner—when changes are easiest to fix. This shortens the feedback loop and reduces time spent troubleshooting runtime refresh failures after deployment. For creators, earlier validation means fewer broken dataflows entering production. This reduces operational noise, minimizes rework, and helps ensure that published dataflows are refresh ready and more stable by default. Learn more: Dataflow Gen2 data destinations validation rules. Evaluate query API (Preview) The Execute Query API (Preview) enables on-demand execution of Power Query logic in Dataflow Gen2 scenarios—without requiring a full scheduled refresh cycle. It’s designed for cases where you need to trigger transformations programmatically (or in response to events) and retrieve results quickly for downstream processing. Event-driven pipelines: run a transformation when new data arrives and push outputs to a destination or consumer immediately. Streaming and near-real-time scenarios: execute queries more frequently than a typical scheduled refresh to support operational dashboards and alerting workflows. Automation at scale: integrate with orchestration tools and scripts to run specific queries as part of broader ETL/ELT jobs. Faster debugging: re-run targeted queries to validate fixes without republishing the entire dataflow. Learn more: Execute Query API (Streaming) documentation (Preview). Data Factory Data Factory MCP (Preview) Dataflow Gen2 offers a suite of pipeline functions, including dataflow creation, M (Power Query) scripting, connection management, query execution, and refresh coordination. These tools are directly accessible to AI assistants. Access is available through platforms such as VS Code, Claude, ChatGPT, Gemini, or via the command line. Why it matters AI assistants create, test, and deploy dataflows through natural language—no browser tabs or manual configuration required. Iterative M development via execute_query lets the AI test transforms against live data before committing to a full refresh. MCP Apps provide guided UI forms (connection setup, gateway selection) inside the chat panel. Open source (GitHub), ships as a NuGet package, runs locally—credentials never leave your machine. Learn more: Data Factory GitHub repo. IBM Netezza ODBC Driver (Generally Available) As we move away from using the embedded Simba driver, customers now have a more dependable and supported option by using their own Netezza driver. This update ensures continued connectivity, long-term support, and a more future-ready experience for organizations using the Netezza connector. Customers do not need to install the new connector; you may reuse your existing connector but will need to install the new IBM Netezza ODBC driver. Figure: IBM Netezza Connector Selection in Fabric UI Reference the IBM Netezza ODBC documentation for more information Google BigQuery connector (Generally Available) This update reflects a shift to the newer GBQ connector as the supported, long-term path forward, providing customers with improved reliability, and alignment with our evolving security standards. With this update, customers can use a connector designed for durability, compliance, and future enhancements. Figure: Google BigQuery Connector in Fabric UI Additional details are available in the Google BigQuery connector documentation. QuickBooks Online connector retirement The QuickBooks Online connector is being retired and will no longer be supported as of March 2026. As part of our ongoing platform evolution, this change streamlines our connector portfolio and ensures our continued commitment to only the highest level of secure data connectivity. After retirement, customers will no longer be able to create new connections, and existing connections may no longer function. Lakehouse Maintenance activity in Fabric Pipelines (Preview) Keeping your Lakehouse healthy shouldn’t require a long checklist or manual scripts. The new Lakehouse Maintenance activity (Preview) makes it easy to automate common upkeep tasks directly inside Fabric Data Factory pipelines. Figure: The Lakehouse maintenance activity in Fabric pipelines With this activity, you can schedule and run actions like vacuuming old files, optimizing table layouts, and managing storage—all in a repeatable, governed workflow. It’s a simple way to keep performance high and storage costs in check, especially for teams managing large or fast‑growing datasets. Figure: The Lakehouse maintenance activity settings Whether you run maintenance nightly or as part of a broader DataOps process, this activity helps support reliable Lakehouse operations. Check out our Lakehouse Maintenance documentation. Refresh SQL endpoint activity in Fabric pipelines (Preview) The process of keeping your SQL analytics layer current is now simpler. The new Refresh SQL endpoint activity (Preview) lets you refresh your Lakehouse SQL endpoint on-demand or as part of your pipeline orchestration. Figure: The Refresh SQL endpoint activity You can trigger targeted refreshes after data ingestion, run coordinated refreshes alongside your transformations, or ensure downstream consumers always see the latest state. It’s built for operational consistency – especially for BI, reporting, and real‑time analytics scenarios that rely on predictable SQL performance. Figure: The Refresh SQL Endpoint activity settings. This activity gives you more control, less manual overhead, and a smoother end‑to‑end refresh experience. Check out the RSQL documentation for more details. Generate Pipeline expressions with Copilot (Generally Available) Writing expressions doesn't have to be time-consuming; simply describe your needs in natural language, and Copilot will generate pipeline expressions for you. Figure: Generate Pipeline expressions with Copilot Whether it’s building dynamic folder paths, conditional logic, string parsing, or parameterized values, Copilot now handles the expression authoring for you. This feature removes friction for both new users and power users – saving time, reducing errors, and making expression logic easier to understand. Workspace monitoring for Fabric Data Factory’s pipelines and Copy job (Preview) Operational observability continues to evolve in Fabric. We’re taking the first major step toward workspace‑level observability in Microsoft Fabric Data Factory. Until now, understanding how pipelines and copy jobs behave at scale often meant inspecting individual runs via Monitoring Hub. With the introduction of workspace monitoring (Preview), Data Factory begins a shift to a workspace‑wide view of operational health. The newest workspace monitoring updates bring clearer visibility and faster troubleshooting across your pipeline ecosystem. What’s available A workspace-wide view of item-level runs Rich filtering, sorting, and drilldown Insight into failure patterns, duration trends, and operational health Faster navigation—no need to click into each pipeline This gives DataOps teams a unified lens to understand performance and diagnose issues quickly. Figure: A view of your pipelines and Copy jobs within the workspace monitoring solution Coming Soon Activity-level L2 monitoring for pipelines. Copy job L2-level monitoring (Preview) for deeper insights and debugging. These improvements continue building toward a more comprehensive, intuitive monitoring experience for production workloads. Check out our docs on Enable Workspace Monitoring in Microsoft Fabric and Workspace Monitoring for Copy Job in Microsoft Fabric for more information on how to use this experience. Interval-based schedules The latest enhancement to Fabric Data Factory pipelines is the availability of interval-based schedules! This powerful new feature allows you to automate data workflows at regular non-overlapping intervals, like the popular tumbling window trigger in Azure Data Factory. Figure: Interval-based schedule configuration in Fabric Data Factory With interval-based scheduling, you can easily configure recurring pipeline runs that ensure timely data processing and seamless integration across your architecture. New Airflow APIs New Airflow Operators Apache Airflow jobs in Fabric Data Factory facilitate the execution of a wide range of Fabric artifacts through native operator integration. Users can run artifacts such as Notebooks, Spark job definitions, Pipelines, Semantic Models, and user data functions directly from their DAGs. Apache Airflow jobs now provide support for executing Copy jobs and dbt jobs! Figure: Airflow operators for Fabric items, including Copy job and dbt job execution To learn more, refer to Run a Fabric item using Apache Airflow DAG. PowerShell model for gateways (Generally Available) The PowerShell model for gateways now delivers fully supported, production-ready automation for gateway lifecycle, update, restore, and configuration management. This release introduces new commands for version discovery and upgrade control, along with reliability and usability improvements that make large-scale, script-driven gateway operations easier and more robust. Figure: Using the Gateway PowerShell module to manage gateway operations from the command line Learn more through the gateway PowerShell documentation and cmdlet reference on Microsoft Learn. Certificate and proxy support for VNet data gateway (Generally Available) Certificate and proxy support for VNet data gateway enables secure, compliant connectivity in enterprise environments. Organizations can use enterprise-issued certificates for gateway authentication and configure proxy routing when direct internet access is restricted. Together, these capabilities strengthen security, support corporate network policies, and expand deployment flexibility in controlled and regulated infrastructures. nshot_of_certificate_and_proxy_settings_for_a_Virtual_Network_Data_Gateway Figure: Configure certificate authentication and proxy for a Virtual Network Data Gateway Learn more through Manage virtual network (VNet) data gateways. Virtual network data gateway supports up to nine instances This update enables greater scalability and higher throughput for enterprise workloads. With expanded instance capacity, organizations can handle increased data movement and processing demands, improve parallel job performance, and enhance reliability for mission-critical tasks. This update provides more flexibility to scale gateway infrastructure in line with growing business needs. Figure: Virtual Network Data Gateway now supports scaling up to nine instances per cluster Learn more What is a virtual network (VNet) data gateway. SSIS Pipeline Activity (Preview) SQL Server Integration Services (SSIS) has been a cornerstone of enterprise data integration for decades, powering mission-critical ETL workloads across thousands of organizations worldwide. Invoke SSIS Package activity in Data Factory in Microsoft Fabric (Preview), provides the power of your existing SSIS investments directly into Fabric's unified SaaS analytics platform. Figure: Add an Invoke SSIS Package activity Many enterprises have significant investments in SSIS packages that orchestrate complex ETL workflows across on-premises databases, file systems, and cloud services. Until now, running these packages required either an on-premises SQL Server, or the Azure-SSIS Integration Runtime in Azure Data Factory. Both options meant managing additional infrastructure and staying outside the Fabric ecosystem. Figure: Invoke SSIS package activity configuration But the Invoke SSIS Package pipeline activity in Microsoft Fabric Data Factory changes this. It allows you to execute your existing SSIS packages directly from a Fabric pipeline, enabling true lift-and-shift of legacy ETL workloads into Fabric—no package rewrite required. There is no need for integration of runtime management or stopping and starting IRs; simply incorporate them into your pipeline. Seamlessly upgrade Azure Data Factory and Synapse pipelines to Microsoft Fabric (Preview) Microsoft Fabric Data Factory now offers a guided (Preview) migration experience to help you move existing Azure Data Factory (ADF) and Azure Synapse Analytics pipelines into Fabric—starting with an assessment-first approach so you can migrate intentionally and validate before switching production workloads. Review readiness and plan next steps: The assessment categorizes pipelines and activities so you can decide what to migrate now vs. what to fix or defer. You can also export results to CSV for offline review and remediation planning. Figure: Review pipeline and activity readiness results in Azure Data Factory (ADF only) Mount your factory to Fabric For Azure Data Factory migrations, you’ll mount your ADF into a Fabric workspace and then continue the remaining steps inside Fabric. Figure: Continue the migration flow in Fabric after mounting Migrate selected pipelines in Fabric In Fabric Data Factory, open the mounted factory (ADF) or chosen workspace (Synapse), then migrate the pipelines you want to migrate. Map linked services to Fabric connections and complete migration During migration, you’ll map ADF/Synapse linked services to Fabric connections. For guidance on creating and managing connections in Fabric, refer to Data source management. Figure: Map Linked Services to Fabric Connections Validate and promote After migration, validate connections and credentials, run end-to-end tests, and then re-enable triggers as needed. Pipelines migrate safely, with triggers disabled by default so you stay in control of execution. Learn more: Upgrade your Azure Data Factory pipelines to Fabric. Data Factory—Mirroring Mirroring for SAP (Generally Available) Built on top of SAP Datasphere’s Premium Outbound Integration, mirroring for SAP seamlessly integrates Fabric’s advanced mirroring engine with SAP Datasphere’s replication flows, unlocking connectivity through SAP’s native data extraction technologies. This means direct access to the full suite of SAP applications—whether it’s SAP S/4HANA (on-premises or cloud), SAP ECC, SAP BW, SAP BW/4HANA, or cloud solutions like SAP SuccessFactors, SAP Ariba, and SAP Concur. Mirroring capabilities allow you to: Eliminate data silos by bringing SAP data alongside other enterprise sources in OneLake. Maintain end-to-end data lineage and governance for compliance and auditability. Accelerate time-to-insight with near real-time data replication no custom ETL required. Figure: Mirrored database for SAP Learn more in Microsoft Fabric Mirrored Databases From SAP. Mirroring for Oracle databases (Generally Available) Mirroring for Oracle is now available in Microsoft Fabric, bringing a production‑ready, enterprise‑grade way to continuously replicate Oracle data into OneLake with no custom ETL pipelines. This milestone reflects strong validation from customers already running Mirroring for Oracle in production and marks a major step forward in Fabric’s zero‑ETL data integration strategy. With near real‑time data replication, customers can keep analytics, BI, and AI workloads continuously in sync with their operational Oracle systems. This release delivers improved stability, scale, and operational readiness, informed directly by customer feedback from public preview deployments. Mirroring for Oracle integrates natively with Fabric experiences like Power BI, Notebooks, and Lakehouses, enabling faster insights without disrupting existing Oracle workloads. As a fully supported capability, Mirroring for Oracle is now ready for broad enterprise adoption with long‑term investment from the Fabric platform team. Figure: Mirroring for Oracle creation steps Learn more at Mirroring for Oracle in Microsoft Fabric. Mirroring for Azure Database for MySQL (Preview) Mirrored databases now support Azure Database for MySQL. This capability enables you to directly replicate data from Azure Database for MySQL Flexible Server into Fabric in near real time, ensuring that information remains current, readily query-able, and seamlessly integrated throughout the analytics stack without the need for traditional ETL processes. Mirrored MySQL data is managed alongside other data sources, facilitating cross-source querying, unified reporting, and comprehensive analytics. Figure: Screenshot of configuring a mirrored database for Azure Database for MySQL Learn more in Microsoft Fabric Mirrored Databases for MySQL. Mirroring for SharePoint List (Preview) Mirroring for SharePoint Lists enables continuous replication of SharePoint Lists and Document Libraries into OneLake without building custom ETL pipelines. This capability keeps SharePoint data automatically synchronized in near real time, ensuring analytics in Fabric stay aligned while SharePoint remains the system of record. When mirrored, both list tables and document library metadata land in OneLake in an analytics‑ready format, with document libraries replicated via shortcuts and converted into Delta Lake tables. Figure: Mirroring setup for a SharePoint list Fabric automatically creates a mirrored database and a read‑only SQL analytics endpoint, providing a rich analytical surface over the replicated data. As changes are made in SharePoint—such as new columns or updated rows—those updates flow continuously into Fabric, keeping schemas and data in sync. This public preview unlocks a simple, unified way to analyze SharePoint operational data across Fabric workloads including SQL, Power BI, notebooks, and data engineering experiences. Extended Capabilities in Mirroring: Change Delta Feed and Snowflake Mirroring Support for Views (Preview) Optional enhancements that build on core mirroring to support more advanced, real‑world analytics scenarios. These capabilities are designed for customers who need more than basic replication—enabling faster freshness, incremental processing, and business‑ready data without building or maintaining complex ETL pipelines. Including Change Data Feed (CDF), which captures inserts, updates, and deletes at a granular level and applies them incrementally into OneLake, allowing mirrored data to stay continuously fresh without full reloads. Extended Capabilities also include Mirroring Views for Snowflake (with support for other sources coming soon), which replicate logical views from the source system into OneLake so that source‑defined business logic—such as joins, filters, and transformations—can be preserved directly in Fabric. Together, CDF and Views enable incremental pipelines, near real‑time analytics, and shaped datasets that are immediately ready for consumption across Fabric workloads. Extended Capabilities are enabled during mirror setup and operate on top of core mirroring, allowing customers to selectively opt into advanced functionality as their analytics and AI needs grow. Billing will be available as part of these extended capabilities starting April 1, 2026. More details about these capabilities and billing can be found on our documentation: Extended Capabilities in Mirroring – Overview. Mirrored database now supports up to 1000 tables To meet growing business demands and improve scalability, mirrored databases now support up to 1000 tables, raised from the previous limit of 500. This enhancement significantly expands the scale of datasets that can be mirrored from the source database, enabling customers to bring more comprehensive data into Fabric without fragmentation, drive deeper analysis and scale the data solution to meet evolving requirements. Learn more in Mirroring in Microsoft Fabric. That’s a wrap! Publishing this update on the first day of FabCon feels especially meaningful. The features in this release reflect not just ongoing platform investment, but the ideas, feedback, and candid conversations we continue to have with the Fabric community—in sessions, online, and across preview programs. Thank you for showing up, sharing your experiences, and helping shape where Fabric goes next. We encourage you to explore these updates, ask questions, and tell us what’s working—whether that happens here at FabCon, in community forums, or through ongoing feedback channels. We’re grateful to be building Fabric alongside such an engaged community, and we’re excited to keep learning from you throughout FabCon and beyond.Simplifying secure data access with Delegated OneLake Shortcuts (Preview)
Introduction Data rarely stays in one place. As organizations standardize Microsoft Fabric and OneLake, the same datasets need to be reused across teams, domains, workspaces, and increasingly across tenant boundaries. The challenge is no longer moving data; it is sharing it securely, consistently, and at scale without creating copies, breaking governance, or forcing every consumer to be individually provisioned at the source. OneLake Shortcuts already solve a large part of this problem. A shortcut presents data where people need it while the data stays in its original location, enabling a true zero-copy approach to distribution. By default, OneLake Shortcuts use pass-through authentication: when a user reads a shortcut, Fabric accesses the target data using that signed-in user’s identity, and the data owner controls access directly on the target. Pass-through is the right model for many collaborative scenarios, but customers have consistently told us it does not fit every access pattern. Two points came up frequently: Access management does not scale. When a curated dataset must be served to thousands of downstream users across multiple teams, the data owner becomes responsible for granting and maintaining every individual user’s permission on the source, an operational bottleneck that grows with every new consumer. Cross-tenant sharing is harder than it should be. Multi-tenant organizations told us that they need to access data residing in OneLake across their own tenant. These are not edge cases. They are everyday realities for enterprises building governed, reusable data products on Fabric. The preview of Delegated OneLake Shortcuts — including delegated sharing both within a tenant and across tenants — gives data owners a simpler, governed way to distribute data without compromising on security. Introducing delegated OneLake Shortcuts Delegated OneLake Shortcuts add a second authentication option to the existing shortcut experience you already know. Instead of accessing the target data as each signed-in user, a delegated shortcut accesses the target through a configured connection identity. That identity can be an organizational account, a service principal. This identity is attached to the shortcut, so all access to the shortcut reaches the target as the delegated identity. Delegated authentication is entirely optional and complements the default experience. If a user does not choose delegated authentication when creating a shortcut, the shortcut continues to use pass-through authentication exactly as before. The default flow is unchanged; delegation is simply there when you need it. How it works A delegated shortcut behaves like other external shortcuts in Fabric. When you create one, you sp, and that connection is used to browse and read the target data. This brings a familiar, governed connection model to OneLake-to-OneLake sharing. Identity delegation - Downstream users access the data through the delegated identity rather than their own, so the data owner no longer must provision each individual consumer on the source item. Secure access enforcement with OneLake security - OneLake security roles can be configured on both the data producer and data consumer delegated Shortcuts. At the time of this writing, table level security and column-level security are supported for delegated shortcuts, on both the target (where you are creating the shortcut) and the shortcut source (where data resides). Delegated permissions management - A shortcut can delegate as a fixed identity that represents a business unit. The central data owner controls what that identity can see, while the business unit manages OneLake security for its own end users, all while still honoring the controls applied to the delegated identity. Cross-tenant sharing Delegated shortcuts also work across Microsoft Fabric tenants. A cross-tenant delegated shortcut lets you create a OneLake shortcut to data that lives in another organization’s Fabric tenant. You provide a connection path to the external OneLake data and authenticate with an identity from that tenant; downstream users then access the external data through the configured delegated identity, without each user needing individual cross-tenant permissions. This makes delegated shortcuts a natural fit for multi-tenant enterprises, for example, sharing curated data between an organization’s test and production tenants, or between a parent company and a subsidiary using the same zero-copy, intersection-based security model that applies within a tenant. Difference between External Data Sharing and delegated Shortcuts Microsoft Fabric has External Data Sharing, a feature that enables Fabric users to share data from their tenant with users in another Fabric tenant. External data sharing can be used when the consumer has no identity in the producer's tenant, such as sharing across organizational boundaries with an outside partner or customer. This is ideal when you must share with partners or when ISVs must share data with their customers and don’t want to have the consumer identity in their tenant. Cross-tenant delegated shortcuts are used when the data consumer has an identity, such as an organizational account or service principal, in the producer’s tenant. For example, an organization can share data between its own test and production tenants, with access flowing through the configured delegated identity. Use cases Delegated OneLake Shortcuts are designed for the moments when the default pass-through behavior does not match the access pattern you want for a data product. Common scenarios include: Departmental data sharing at scale - Represent each department with a delegated identity, scope what that identity can see, and let department owners manage access for their own users instead of routing every request through the central data owner. Cross-tenant and subsidiary sharing - Share curated data between tenants — such as test-to-production or parent-to-subsidiary — with no data copies and the same delegated security model. Getting started Open the target Fabric item, such as a Lakehouse, and select Get data > New table shortcut. In New shortcut, select Microsoft OneLake, then choose the source you want to shortcut to. For cross-tenant data, select Enter connection details and provide the external OneLake path. For Connection method, select Delegated identity, then Connect. Choose an existing connection or create a new one by providing the OneLake path, a recognizable connection name, and an authentication kind (organizational account or service principal). Sign in to complete authentication. Browse the source, select the folders or tables to include, then review and create the shortcut. To switch an existing shortcut between pass-through and delegated authentication, delete and recreate it with the desired method. For detailed steps, refer to the OneLake Shortcuts documentation. Conclusion and next steps OneLake shortcuts are a foundational building block for zero-copy data distribution across Microsoft Fabric. Delegated OneLake Shortcuts extend that foundation to the scenarios enterprises care about most: serving curated data to large audiences, delegating access management to the teams closest to the users, and sharing securely across tenant boundaries. Together, pass-through and delegated shortcuts let organizations choose the right balance of control, scale, and simplicity for each data product. Pass-through keeps source-managed authorization per person for collaborative engineering. Delegated mode turns a shortcut into part of a governed publishing architecture: central teams retain ownership of the source, consuming teams avoid copying data, and downstream audiences access a managed experience rather than raw-path access — without ever giving up the governance benefits of unifying data in OneLake. Share your feedback, use cases, and questions in the Microsoft Fabric Community. Your input directly shapes the roadmap.SharePoint and OneDrive Shortcuts in OneLake (Generally Available)
For most enterprises, the largest and fastest-growing repository of business-critical information is SharePoint and OneDrive. Contracts, financial models, project plans, meeting notes, presentations, and compliance documentation accumulate across every team and department, rich with context that rarely makes it into an analytics workflow. This content has traditionally been invisible to data platforms. Bringing it into a data lake meant building pipelines, scheduling exports, managing duplicates, and reconciling governance across two separate systems. The result: valuable knowledge stayed locked in productivity tools while analytics teams worked with an incomplete picture. Now, SharePoint and OneDrive Shortcuts in Microsoft Fabric OneLake is now generally available. With this capability, organizations can now surface this data directly inside OneLake, without requiring traditional data movement pipelines in many common scenarios. Files stay exactly where they are in SharePoint or OneDrive, and Fabric workloads see them as a native part of the data lake. This is more than a connectivity feature. It is the bridge between the world where work happens and the world where data is analyzed. Sales forecasts stored in Excel can be joined with CRM transactions. Legal documents can be indexed and grounded in AI agents. Financial trackers can feed Power BI reports the moment they are updated. This helps reduce the boundary between collaboration content and enterprise analytics workflows. Customer Use Cases Organizations across industries are already finding practical, high-value ways to connect their Microsoft 365 content with Fabric analytics and AI. Below are example scenarios that illustrate potential use cases. Data Lake Unification Without Migration Many organizations maintain structured data in their Lakehouse alongside a parallel, untouched archive of unstructured content in SharePoint. Shortcuts eliminate the need to choose between the two. Data engineering teams can now create a unified view across both sources inside OneLake, enabling joins, aggregations, and AI workflows that span the full breadth of enterprise knowledge, without moving a single file. Combine SharePoint-hosted reference tables with Lakehouse transactional data in a single Spark notebook. Surface operational documents alongside structured metrics in a single Power BI semantic model. Avoid costly migration projects by referencing content in place and retiring redundant sync processes. Self-Service Analytics on Shared Documents Finance, HR, and operations teams frequently maintain planning workbooks, trackers, and reports in OneDrive and SharePoint. These files are updated regularly by business users who have no need or desire to interact with a data platform directly. Shortcuts let analysts consume this content without asking anyone to change how they work. Finance teams can surface quarterly budget workbooks directly in Power BI without any export step. HR can connect headcount trackers and org charts to workforce analytics dashboards. Operations teams can make procurement logs and vendor documents queryable alongside ERP data. Microsoft Fabric and Foundry: AI at Enterprise Scale For organizations building production-grade AI solutions on Azure, the combination of Fabric OneLake and Microsoft Foundry creates a powerful foundation. Shortcuts ensure that the rich, unstructured knowledge stored in SharePoint and OneDrive is available as a live, governed data source for Foundry-based agents and copilots. Connect SharePoint document libraries to Foundry knowledge stores without a separate ingestion pipeline. Keep AI knowledge grounding current automatically as SharePoint content is updated by business teams. Apply Fabric data transformations to prepare document content for structured AI consumption at scale. Features Supported in General Availability The following capabilities are generally available and supported for production use as of this release: Core Shortcut Capabilities Create shortcuts from any Fabric Lakehouse directly to OneDrive folders or SharePoint document libraries. Access files in place, without requiring explicit data duplication in many scenarios. Files remain governed by their existing SharePoint and OneDrive permissions. Live synchronization ensures that as content is added or updated in SharePoint or OneDrive, Fabric workloads see the latest version automatically. Shortcut Transformations Beyond simple file access, OneLake shortcuts include an optional transformation step that converts supported file types directly into Delta tables. This can reduce or eliminate the need for a separate ETL pipeline in supported scenarios, making document-resident data queryable by analytics engines. Supported file formats for transformation at GA: CSV, Parquet, and JSON. Transformed tables are kept in sync as new files arrive in the connected SharePoint or OneDrive folder. Transformed output integrates natively with Fabric warehouses, notebooks, and Power BI, enabling immediate analytics without additional data engineering. This capability fundamentally changes how customers are working with file-based data. A SharePoint folder containing hundreds of CSV exports from a business application can be transformed into a queryable Delta table in minutes, with minimal pipeline code required in most scenarios. General Availability Improvements In addition to the core shortcut capability, this GA release introduces two significant platform-level improvements that make shortcuts enterprise-ready for automation, scale, and cross-organizational scenarios. Service Principal and Workspace Identity Authentication OneDrive and SharePoint Shortcuts now support Service Principal (SPN) and Workspace Identity (WI) authentication, in addition to organizational account sign-in. This is a critical capability for production deployments. Reduces dependency on individual user credentials, preventing pipeline failures when team members change roles or leave the organization. Authentication is managed through Microsoft Entra ID, enabling consistent security governance and audit trails. SPN and WI authentication support higher API limits, directly reducing throttling in high-throughput scenarios. Cross-tenant access: service principals can be configured to access SharePoint and OneDrive content across organizational boundaries, enabling configurable partner and subsidiary data sharing scenarios, subject to tenant policies and security configurations. Metadata Caching and Performance OneLake now caches SharePoint metadata internally, reducing the frequency and impact of calls to the SharePoint API during query execution. This improvement can deliver the following benefits: Reduced throttling under high query volumes, particularly in multi-user or scheduled workload scenarios. Improved query performance for workloads that enumerate or filter large SharePoint folder structures. These improvements make shortcuts viable for production-grade pipelines and scheduled refresh scenarios that would previously have encountered reliability issues at scale. How to Get Started Creating a SharePoint or OneDrive shortcut in Fabric takes less than five minutes. The following steps apply to any Fabric workspace with at least one Lakehouse: Open a Lakehouse in your Fabric workspace. In the Explorer pane, right-click any folder and select New shortcut. In the New Shortcut dialog, select OneDrive or SharePoint tile from the list of external sources. Choose your authentication method: Organizational account for interactive scenarios, or Workspace Identity / Service Principal for automated and production workflows. Provide the SharePoint site URL and select or create a connection. If you don’t have root level access and prefer to provide the path directly, change the view to Path View by navigating to the top right corner. Browse to the folder or library you want to connect. Select one or more target locations and select Next. On the Transform page, choose whether to apply a transformation to convert supported file types (CSV, Parquet, JSON) into Delta tables. Select Skip if you only need file access. Select Create to finalize. Your shortcuts will appear immediately in the Lakehouse Explorer. From there, you can reference them in Spark notebooks, build Power BI reports directly on the data, run SQL queries through the Lakehouse SQL endpoint, or include them in Fabric pipelines and AI workflows. Learn more by exploring Create a OneDrive or SharePoint shortcut (Microsoft Learn documentation). We are excited to see what you build SharePoint and OneDrive Shortcuts in OneLake represent a step toward a world where every document in your organization is an active participant in your data and AI strategy. As the boundary between productivity and analytics continues to dissolve, Fabric is designed to be the platform that connects them. Share your feedback, use cases, and questions in the Microsoft Fabric Community forums. Your input directly shapes the roadmap.
Bring your Azure Monitor and AWS Glue data to OneLake! (Preview)
Microsoft Fabric is expanding the reach of Microsoft OneLake with two new mirroring capabilities: Mirrored Azure Monitor and Mirrored AWS Glue catalog. These previews make it easier to bring operational telemetry and tables from external catalogs into Fabric while minimizing data movement and integration complexity. Bring more of your data estate into OneLake Organizations manage data across clouds and platforms. As two examples, operational telemetry may live in Azure Monitor, and some lakehouse environments may use AWS Glue Data Catalog to organize Apache Iceberg tables stored in Amazon S3. Bringing these systems together traditionally requires custom ingestion pipelines, duplicate storage, and ongoing infrastructure management. Fabric mirroring provides a simpler model: it connects to external systems and reflects data or metadata into OneLake and Fabric experiences, helping teams analyze information across platform boundaries. Mirror your Azure Monitor data into Fabric Mirrored Azure Monitor brings tables from Log Analytics workspaces into OneLake without duplicating the operational data. It connects OneLake to the Log Analytics storage used by Azure Monitor, allowing teams to combine telemetry with business data already available in OneLake. This creates opportunities for: Operational analytics across application and infrastructure signals. Cross-domain reporting that connects service health with business outcomes. Real-time intelligence scenarios using KQL. AI-powered reasoning across operational and business data. Because the data remains governed by Azure Monitor and is accessed without replication, teams can avoid building a second ingestion and storage path merely to analyze telemetry in Fabric. Bring Iceberg tables from AWS Glue into OneLake Mirrored AWS Glue catalog is designed for organizations that use AWS Glue Data Catalog for their Apache Iceberg tables backed by AWS S3. Users connect OneLake to AWS Glue, select supported Iceberg tables, and have their Iceberg tables automatically show up in OneLake for use in Fabric. The experience is intended to enable: Unified discovery of Glue-cataloged Iceberg data with other OneLake data. Faster onboarding of cataloged data into Fabric analytics experiences. Cross-cloud analysis using Fabric workloads such as Power BI, Data Warehousing, Data Engineering, and Data Science. The underlying data remains in its existing storage location, while Fabric uses mirrored metadata and OneLake shortcuts to make supported Iceberg tables available across the platform. Open by design These capabilities continue Microsoft’s commitment to open data architectures and cross-platform interoperability. The mirrored catalog feature establishes a metadata-based, zero-copy pattern for bringing external catalog-managed data into OneLake. Mirrored AWS Glue catalog and Mirrored Azure Monitor extends the mirrored catalog approach to additional providers and environments. Get started Refer to the following steps to try out mirroring for these new sources today! Mirrored Azure Monitor To set up mirroring for your Azure Monitor tables from your Log Analytics workspace: Create a new Mirrored Azure Monitor item in Fabric. Connect it to a supported Log Analytics workspace. Select the tables you want to make available. Start exploring and analyzing the mirrored tables using Fabric experiences. Check out the Mirror Azure Monitor in Microsoft Fabric (preview) documentation for more guidance. Mirrored AWS Glue catalog To set up mirroring for Iceberg tables from your AWS Glue catalog: Create a new Mirrored AWS Glue catalog item in Fabric. Connect to AWS Glue Data Catalog using a supported authentication method. Select the Iceberg tables you’d like to mirror into OneLake. Start exploring and analyzing the mirrored tables using Fabric experiences. Check out the Mirrored AWS Glue catalog (preview) documentation for more information. Mirrored Azure Monitor and Mirrored AWS Glue catalog help bring more of your data estate into OneLake, reducing the complexity of traditional integration projects and making it easier to work across operational, business, and cross-cloud data. We want your feedback! Try the previews today! Share your feedback through the Fabric Ideas site and Microsoft Fabric Community!Building Common Data Architectures with OneLake in Microsoft Fabric
Introduction OneLake can be used as a single data lake for your entire organization, it provides ease of use and helps eliminate data silos. It can also simplify security while ensuring that sensitive data is kept secure. OneLake and Fabric provide several out of the box capabilities to keep data access restricted to only those that need it. This article will look at some common data architecture patterns and how they can be secured with Microsoft Fabric. Security structure It is important to understand the basic building blocks of security in Microsoft Fabric before getting started. Fabric provides many different places where security can be set. This allows for both flexibility and scale of security configurations. There are three main types of security in Fabric. Workspace roles Item permissions Compute permissions We will take a close look at each of these and how they interact. Workspace roles The first and least granular level of security in Microsoft Fabric is workspace roles. Workspace roles are pre-configured sets of capabilities that can be granted to users or groups at the workspace level in Fabric. When a user is assigned to a role, they receive all the capabilities of that role within the confines of the workspace. These workspace permissions then apply to all items within the workspace. Each workspace role contains permissions that allow users to perform certain actions. For this blog, we are focusing on data security and the data access granted by each role is outlined below. Role Can add admins? Can add members? Can write data and create items? Can read data? Admin Yes Yes Yes Yes Member No Yes Yes Yes Contributor No No Yes Yes Viewer No No No Yes Item permissions Next in the hierarchy are permissions that can be set on a specific item. Item permissions allow for adjusting the permissions set by a workspace role or giving a user access to a single item within a workspace without adding them to a workspace role. The easiest way to configure item permissions is to share an item with a user or group. During the sharing step, the user can choose which permissions to grant to the end user. Sharing the item always grants the user the Read permission for that item. Read allows users to see the metadata for that item and view any reports associated with it but not access the underlying data in SQL or OneLake. To grant just the Read permission, leave all the boxes unchecked. If the “Read all SQL endpoint data” is checked, users will be given the ReadData permission. ReadData gives access to all Tables in the item when accessing through the SQL Endpoint. Users will not be able to access OneLake directly. If the “Read all Apache Spark” box is checked, users will be given ReadAll. This permission allows users to access data in OneLake. This could be through direct OneLake access, Apache Spark queries, or the lakehouse UX. The last checkbox is not relevant for this blog, but you can learn about the Build permission here. Compute permissions (SQL and Semantic models) The last place permissions can be set is within a specific compute engine in Fabric, specifically through the SQL Endpoint or semantic models. The SQL Endpoint provides direct SQL access to Tables in OneLake, but it can have security configured natively through SQL commands. SQL security allows for more granular permissions such as table and row level security. However, the security set in this way only applies to queries made through SQL. Accessing OneLake data through a Spark query (users with the ReadAll permission) is not impacted by the security restrictions in SQL. Likewise, semantic models also allow for security to be defined using DAX and those restrictions apply to users querying through the semantic model or reports built on top. In the below example, a Lakehouse is shared with a user and Read access is granted. They are then given SELECT through the SQL endpoint. When that user tries to read data through Spark notebooks the access gets denied since they don’t have ReadAll, but reads made through SQL SELECT statements would succeed. Shortcuts Shortcuts are a feature of Microsoft OneLake that allow for data to be easily reused without making copies of the data. Shortcuts function like symbolic links where the data appears as if it is natively part of a Lakehouse, but the original data was not moved or copied. There are two primary types of shortcuts, and the security functions differently for each. OneLake shortcuts: Are shortcuts to another location in OneLake. These shortcuts require that the user accessing the shortcut has access to the location that the shortcut points to. For example, if Lakehouse1 has TableA that is a shortcut to Lakehouse2/TableB. Any user accessing TableA will need access to Lakehouse2/TableB to see any data. External (ADLS, AWS S3) shortcuts: Shortcuts to external locations outside of Fabric/OneLake use a service principal or account key to authenticate to the target destination. All users accessing the shortcut receive the same permissions as provided by the service principal/account key. To control access to data in external shortcuts, use the guidance from the Securing data section to configure permissions at the appropriate level within Fabric. This may require configuring SQL security to restrict access to the item. Another important detail about shortcuts is how they interact with SQL Endpoint and SQL Warehouse queries. SQL engines in Fabric use a delegated model when accessing shortcuts. This means that the SQL creator’s identity (the identity of the user that created the Lakehouse or warehouse) is checked against the shortcut target destination not the querying user. The diagram below shows which identity is used when accessing a shortcut to another Lakehouse based on which engine is being queried. Securing data Now that we understand the tools Fabric provides for configuring access, how should the pieces be setup to work together? The determining factor for setting access should be to always grant users access to data they need at the lowest possible level. If a user needs to read data from a report, they should only be given access to that report itself. With that guidance in mind, let us take a look at some common patterns. Data Mesh Data mesh is an architectural paradigm that treats data as a product, rather than a service or a resource. Data mesh aims to decentralize the ownership and governance of data across different domains and teams, while enabling interoperability and discoverability through a common platform. In a data mesh architecture, each decentralized team manages the ownership of the data that is part of their data product. Microsoft Fabric supports organizing data into domains and enabling data consumers to be able to filter and discover content by domain. It also enables federated governance, which means that some governance currently controlled at the tenant level can be delegated to domain-level control, enabling each business unit/department to define its own rules and restrictions according to its specific business needs. As a result, enterprise customers are empowered with the key tools they need to structure their tenant's data estate along the principles of a data mesh. Let's take a look at building a data mesh in Fabric. First, using the domains feature, tenant admins can manage the creation and assignment of any domains and the associated workspaces. Next, each data team has its own Fabric workspace. The workspace will store the data and orchestration needed to build out the final data products for consumption. The users that build and create data products in the workspace are given a read/write workspace role such as Contributor. This will let them interact with all the items in the workspace and create new ones as needed. Second, within the workspace teams will have Lakehouses that are consumed by different downstream teams. For example: data scientists, business analysts, and company leaders. To keep users aligned with their target experiences, each type of downstream user can be given access to a single Fabric data experience. Downstream User Fabric experience Data scientists Spark notebooks Business analysts SQL Endpoint Report creators Semantic models Company leaders Power BI reports Using Fabric artifact permissions, we can assign each user group to a single experience by sharing the correct items. The admin can share the Lakehouse with “Read all SQL endpoint data” for the business analysts. They can share the Lakehouse with “Read all Apache Spark” selected for the data scientists. Lastly, they can share the Power BI reports with the company leaders to ensure they have access to the polished end products. Because the “Read all Apache Spark” setting gives full access to the data of a Lakehouse, there might be cases where multiple Lakehouses are needed. For example, if some of the data is specific to only some data scientists due to PII or country specific data contents, lakehouses can be created per downstream consumption group. Using shortcuts, new lakehouses can be easily created and share data between them without creating additional data copies. For business analysts, granular security such as row and column level security can be configured directly in the SQL Endpoint for the Lakehouse. This ensures that data is kept secure at a granular level. This same security can be reused for Power BI reports built over the SQL Endpoint as well. Data mesh is unique in that each workspace will implement the same approach to managing security, as each team may need to consume data products from a variety of other teams. This interconnected approach is where the power of this architecture shines through. However, it’s important that each team understands how to correctly secure data since there is no central team managing all data security. Hub and Spoke Hub and spoke is a data architecture pattern that centralizes the data from different sources into a single hub, such as a data warehouse or a data lake. The hub serves as the source of truth for the data and provides standardized schemas and formats. The spokes are the various applications or services that consume the data from the hub for different purposes, such as analytics, reporting, or machine learning. The spokes can also perform transformations or aggregations on the data before presenting it to the end users. Hub and spoke aims to simplify the data integration and management process by reducing the complexity and redundancy of data pipelines. Like with the data mesh architecture, there is no single way to build a hub and spoke model. We will look at a common method of how to achieve this in Microsoft Fabric. Like the data mesh approach, workspaces are the core level for defining groups of related items and securing them. It is common for a hub and spoke team to have a single workspace where all data products are created and managed, but you can just as easily use multiple workspaces. For the data engineers and teams creating the central data products, granting the Contributor workspace role works best. Within the workspace, polished data products will be created for end users to consume. Some of these data products will need to be consumed by other teams for use in in-depth analysis and will require access to the underlying OneLake data. Others will consume data through a SQL warehouse or SQL Endpoint. Lastly, polished reports for company executives and decision makers are created and shared with those users. Using item sharing, the hub and spoke admins can share select consumption experiences or artifacts with the downstream teams that need access to them. For data science or ML teams, use “Read all Apache Spark” to give teams access to the OneLake data (raw files.) Teams can then leverage shortcuts to access the data in their own workspace without creating copies of it. For most business analysts, use “Read all SQL Endpoint data” to give access to the SQL queries. This can be augmented by defining SQL permissions. For businesses that don’t use lake data, you can simplify this by creating a Fabric warehouse instead a Lakehouse and share the warehouse directly. For users that need to consume sensitive data, creating pipelines against the SQL Endpoint will allow for RLS or other fine-grained security to be applied when that data is read. The data can be transformed or processed downstream without risking access to rows or columns that are not allowed for those users. For all other users, Power BI reports can be created and shared from the workspace. You can create reports specific to downstream teams that will need them. Recap Microsoft Fabric provides a robust set of tools for managing access to data, while leveraging the power of OneLake to simplify your data estate. In this guide we looked at some common architectures and how you can use the capabilities in Fabric to build those models and keep your data secure. Next steps Learn more about Fabric security features here. Microsoft Fabric security - Microsoft Fabric | Microsoft Learn Get started with a Microsoft Fabric free trial: Sign up for a Fabric free trial. Create a Lakehouse to get started with exploring the features in this article: Create a Lakehouse - Microsoft Fabric | Microsoft Learn
Announcing Mirroring Azure SQL Database in Fabric for Public Preview
In the era of digital transformation, advanced analytics and an AI driven world, data has emerged as the new oil, powering businesses, and driving decision-making. But what good is this oil if it’s not refined and ready for use when needed? Moreover, managing, and ingesting data into a central platform for analytics and AI is costly and cumbersome process. This is where the importance of near-real-time data replication comes into play. It’s not just about having data; it’s about having the right data at the right time. To address these challenges, we launched Mirroring in Microsoft Fabric at Ignite’23 for private preview. Today, we are excited to announce the public preview of Mirroring Azure SQL Database, Azure Cosmos Database and Snowflake data sources in Fabric, a new, simple, and frictionless way to replicate a snapshot of these source database in Fabric OneLake in Delta tables that keeps the data in sync in near-real time. The key benefits that Mirroring databases in Fabric enables are: Reduced total cost of ownership with zero compute to replicate along with generous amounts (terabytes) of storage based on the capacity size. Zero code with zero ETL Faster time to operational data, information to derive insights. This blog will explore the importance of Mirroring Azure SQL database in Fabric, discuss its main features and how it can transform your data strategy. https://www.youtube.com/watch?v=IKR796HlstA How does Mirroring Azure SQL Database in Fabric work? Mirroring Azure SQL Database in Fabric ensures that your source transactional SQL database is always up to date and available in the Fabric OneLake, providing a solid foundation for reporting, advanced analytics, AI, and data science. There is no complex setup or ETL for Mirroring. You setup the mirror from Fabric Data Warehousing experience by providing the Azure SQL server and database connection details, provide selections on what needs mirrored into Fabric, either all data or user selected eligible mirrored tables. And, just like that mirroring is ready to go. Mirroring Azure SQL database creates an initial snapshot in Fabric OneLake after which data is kept in sync in near-real time with every transaction when a new table is created/dropped, or data gets updated. Key features Mirroring for Azure SQL Database is built on the SQL’s Change Data Capture (CDC) stack optimized for lake-centric architecture. CDC stores changes locally in the database whereas Mirroring reads data from the harvested database transaction log and publishes the change data to OneLake storage. This change data is transformed into appropriate delta tables landing into Fabric OneLake. Moreover, DDL’s like add/drop column, alter table column <<datatype>>, drop table, rename table and rename column are also supported on actively mirrored tables. As a SQL database administrator or user, you can also check the status of Mirroring by using these public stored procedure and dynamic management views: To confirm if Mirroring configuration of the Azure SQL database is enabled correctly, execute the following public stored procedure. The key columns to look for here are the “table_name” and “state”. Any value of “state” column besides “4” indicates a potential problem. exec sp_help_change_feed; If you're experiencing mirroring problems, perform the following database level checks using Dynamic Management Views (DMVs) if data changes flow properly: SELECT * FROM sys.dm_change_feed_log_scan_sessions; If the DMV above doesn't show any progress on processing incremental changes, execute the below query to check if there are any problems reported: SELECT * FROM sys.dm_change_feed_errors; The Mirrored data can also be actively monitored from Fabric providing more insights into mirroring operations and when was mirrored data last refreshed. ric_for_Public_Preview From here on, the mirrored data in the delta format is ready for immediate consumption across all Fabric experiences and features like Power BI with new Direct Lake mode, Data Warehouse, Data Engineering, Lakehouse, KQL Database, Notebooks and co-pilots work instantly. Power BI Direct Lake mode Direct Lake mode is a fast path to load the data from the lake with groundbreaking semantic model capability for analyzing very large data volumes in Power BI. As Direct Lake mode also supports reading Delta tables right from OneLake, the Mirrored SQL database is Power BI ready along with Copilot. Cross-join Mirrored SQL Databases, Lakehouse's, Warehouses Data across any mirrored database (either Azure SQL DB, Azure Cosmos DB or Snowflake) can be cross joined as well enabling querying across any database, warehouse or lakehouse (either as a shortcut to AWS S3 or ADLS Gen 2 etc.) Explore Data Science and Data engineering insights Data scientists and data engineers can work with the mirrored SQL data that are created as shortcuts in Lakehouse. Summary & get started. To summarize, Mirroring Azure SQL Database in Fabric plays a crucial role in enabling analytics and driving insights from data by: Timeliness of Insights: Ensures that the most recent data is available for analysis. This allows businesses to make decisions based on the most current situation, rather than relying on outdated information. Improved Accuracy: The risk of discrepancies between the source and the replicated data is significantly reduced leading to more accurate analytics and reliable insights. Predictive Analytics and AI: Essential for predictive analytics and AI models that require the most recent data to make accurate predictions and decisions. To get started and learn more about Mirroring Azure SQL in Fabric, its pre-requisites, setup, FAQ’s, current limitations, and tutorial, click here to read all about it. We hope you enjoy using Mirroring Azure SQL Database in Fabric and we look forward to hearing your feedback and questions. Please stay tuned for more updates and new features coming soon.
Fabric September 2024 Monthly Update
Welcome to the September 2024 Update! Announcements We have a lot of exciting announcements to share with you for FabCon Europe! We've brought Copilot to Dataflows Gen2 and a richer Copilot experience when building and consuming Power BI Reports. With Real-Time Intelligence we have redesigned and enhanced user experience in the Real-Time hub. We announced the general availability of Fabric Git integration. You can sync Fabric workspaces with Git repositories, leverage version control, and collaborate seamlessly using Azure DevOps or GitHub. We now have an enhanced and redesigned left navigation experience in Real-time Intelligence with the new Real-Time hub user experience. In AI, we have released Copilot in Fabric experience for Dataflows Gen2 into general availability; allowing everyone to design dataflows with the help of an AI-powered expert. We also released Copilot in Fabric experience for Data Warehouse into preview. This AI assistant experience can help developers generate T-SQL queries for data analysis, explain and add in-line code comments for existing T-SQL queries, fix broken T-SQL code, and answer questions about general data warehousing tasks and operations. To learn more, read about all these announcements, and more in Arun's blog post Building an AI-powered data platform. Contents Power BI Core Announcing the availability of Trusted workspace access and Managed private endpoints in any Fabric capacity. Multitenant organization (MTO) (public preview) Announcing Git integration (generally available) A new design for Deployment pipeline (preview) Homepage improvements OneLake Access Databricks Unity Catalog tables from Fabric (public preview) Google Cloud Storage shortcuts and S3 Compatible shortcuts (generally available) REST APIs for OneLake shortcuts (generally available) OneLake SAS (public preview) Data Warehouse Copilot for Data Warehouse (public preview) Delta column mapping in the SQL analytics endpoint (public preview) Enabling SQL analytics endpoint on schema enabled Lakehouse’s (public preview) New editor improvements for Data Warehouse and SQL analytics endpoint Database Migration Experience (private preview) TSQL Notebook (public preview) Nested Common Table Expression (public preview) Data Engineering High Concurrency mode for Notebooks in Pipelines (public preview) Workspace Level Setting to Reserve Maximum Cores for Jobs in Fabric Data Engineering (public preview) Session Expiry Control in Workspace Settings for Notebook Interactive Runs (public preview) Spark Connector for Fabric DW – New Features T-SQL Notebook (public preview) Fabric Spark Diagnostic Emitter: Collect Logs and Metrics (public preview) Environment Artifact integration with Synapse VS Code extension Notebook debug within vscode.dev (public preview) Adding Python support in Fabric User Data Functions Invoke Fabric User Data Functions in Notebook Functions Hub is now available in Fabric User Data Functions Support for spaces in Lakehouse Delta table names Fabric Runtime 1.3 GA Native Execution Engine on Runtime 1.3 (public preview) Acceleration tab and UI enablement for the Native Execution Engine Fabric Spark Runtimes Release Notes Enable/Disable Functionality in API for GraphQL Public REST API of Livy Endpoint Data Science Announcing Public Preview: Share Feature for Fabric AI Skill Data Wrangler now supports Spark DataFrames and PySpark code generation (generally available) Announcing new usability improvements for Data Wrangler File editor in Notebook Real-time Intelligence Creating a Real time Dashboard by Copilot Adding a real-time dashboard to an org app Introducing A New Real-Time Hub User Experience New Streaming Sources Now Available via Eventstream Connectors Introducing Eventhouse as a new Destination in Eventstream Eventstream’s Integration with Managed Private Endpoint Introducing the new look and feel of KQL Database Set alerts on KQL Querysets with Data Activator triggers Data Exploration made easier with Top values feature Real-Time Dashboard lower than ever refresh rate Multivariate anomaly detection Real-Time Intelligence Copilot conversational mode New Rule Creation Experience We’ve made it easier to alert your teammates in Power BI Data Factory Dataflow Gen2 Copilot in Dataflow Gen2 (generally available) Incremental refresh for Dataflow Gen2 Certified connector updates Data pipeline Fabric Pipeline Integration in On-premises Data Gateway (generally available) Invoke remote pipeline in Data pipeline Spark Job environment parameters Mirroring Mirroring Azure SQL Database New Azure Data Factory Item Copy Job (public preview) Monthly Update Video Power BI You can now choose from a variety of themes Power BI Desktop, including the most requested Dark Mode! You can personalize your data visualization experience to match your preferences and working environment. In addition, we’ve now consolidated similar options in the menu bar and streamlined the button text for better readability and responsive screen sizing. The Copilot chat pane will now automatically provide text-based answers and summaries across all pages in a report. Previously, users had to specifically request cross-page summaries or click a "base summary on the entire report" button. With this update, cross-page summaries and answers are now the default setting, streamlining the exploration process. Power BI has a transformative new feature designed to redefine how organizations manage and consume metrics and features visuals and Copilot insights Called Metrics Hub. Metrics Hub is an innovative metric layer within Fabric, aimed at helping organizations define, discover, and reuse trusted metrics with ease. This feature allows trusted creators within an organization to develop standardized metrics that incorporate essential business logic, ensuring consistency across the organization. To learn more about these, and all the other new features this month, read the Power BI September 2024 Feature Summary. Core Announcing the availability of Trusted workspace access and Managed private endpoints in any Fabric capacity. We’d like to share an update for the Fabric network security features that were announced in general availability earlier this year. Trusted workspace access, and Managed Private endpoints enable you to secure and optimize your data access and connectivity with Fabric and protect your business-critical data from unauthorized or unwanted access. However, these features were available only in F64 or higher capacities. Based on your feedback, we are now making these features available in all F capacities. You can now use these features with any F capacity that suits your business needs. We are also making Managed Private endpoints available in Trial capacities as part of this release. Here’s a quick recap of what these features do and how they can help you: Trusted workspace access allows seamless and secure access to firewall enabled Azure storage accounts. It is designed to help you securely and easily access data stored in Storage accounts from Fabric workspaces, without compromising on performance or functionality. This feature extends the power and flexibility of OneLake shortcuts to work with data in protected storage accounts in place without compromising on security. You can also use this capability with Data pipelines and the COPY INTO feature of Fabric warehouses to ingest data securely and easily into Fabric workspaces. To get started with this feature and to learn about limitations, see Trusted workspace access in Microsoft Fabric – Microsoft Fabric | Microsoft Learn. This feature can be used in any F capacity. Managed private endpoints provide secure connectivity from Fabric to data sources that are behind a firewall or not accessible from the public internet. Managed Private Endpoints enable Fabric Data Engineering items to access data sources securely without exposing them to the public network or requiring complex network configurations. Managed private endpoints are supported for various data sources, such as Azure Storage, Azure SQL Databases, and many others – the most recent addition being Azure Event Hub and Azure IOT Hub. To learn more about Managed Private Endpoints and supported data sources see Overview of managed private endpoints for Microsoft Fabric – Microsoft Fabric | Microsoft Learn. This feature can be used in any F capacity as well as Trial. Multitenant organization (MTO) (public preview) Fabric now supports Entra Id Multitenant Organizations (MTO). Many larger organizations have multiple Entra Id tenants for various reasons such as mergers and acquisitions, compliance and security boundaries, or due to complex organizational structure. The multitenant organizations capability in Entra Id synchronizes users across multiple tenants, adding them as users of type external member. We are excited to announce public preview support for MTO. External members can now sign in to Fabric to consume and create content. MTO users can bring their own licenses from their home tenants. Users that have been assigned Power BI Pro or PPU licenses in their home tenants will not need to acquire an additional license for the other MTO tenants. Click here for more information. Announcing Git integration (generally available) Fabric Git integration is now generally available! This feature allows you to sync workspaces with Git repositories, leverage version control, and collaborate seamlessly using Azure DevOps or GitHub. Though some items are still in preview, additional items will become generally available for Git integration over time. Learn more about what's available now and what’s coming next. A new design for Deployment pipeline (preview) Deployment Pipeline Redesign is now in preview! We are thrilled to announce the launch of Deployment pipelines redesign. This major update brings many improvements and new features designed to make your deployment process more efficient and user-friendly. We’ve reimagined the deployment pipeline from the ground up, ensuring that every aspect of your deployment workflow is optimized for performance and ease of use. Homepage improvements We are excited to share the latest enhancements to our homepage that are designed to streamline your experience and boost your productivity. 1. Enhanced Workspace Focus The first improvement we've made is putting workspaces and related actions in the premium real estate on the homepage. The primary call to action is now dedicated to creating a workspace and navigating to recent ones. In the Get started section, you can now create a workspace with a predesigned template called task flow. The task flow guides you to create specific items in a workspace. The idea is to encourage you to think in terms of projects and what you are trying to build end-to-end, instead of having to think about which workloads or items you need to use. If you want to revisit any of your recent workspaces and continue where you left off, the Quick Access section is now optimized for efficient workspace navigation. 2. Collapsible Learn Section We’ve also refined our Recommended section, transforming it into the Learn section. This new area is packed with sample materials and learning resources designed to help new users get up to speed quickly. Recognizing that these resources are particularly useful for newcomers, we've made the Learn section collapsible. For our experienced users, collapsing this section provides more space for the Quick Access area, ensuring a more streamlined experience. Watch a demo for Core OneLake Access Databricks Unity Catalog tables from Fabric (public preview) You can now access Databricks Unity Catalog tables directly from Fabric. In Fabric, you can now create a new data item called “Mirrored Azure Databricks Catalog”. When creating this item, you simply provide your Azure Databricks workspace URL and select the catalog you want to make available in Fabric. Rather than making a copy of the data, Fabric creates a shortcut for every table in the selected catalog. It also keeps the Fabric data item in sync. So, if a table is added or removed from UC, the change is automatically reflected in Fabric. Once your Azure Databricks Catalog item is created, it behaves the same as any other item in Fabric. Seamlessly access tables through the SQL endpoint, utilize Spark with Fabric notebooks and take full advantage of Direct Lake mode with Power BI reports. To learn more about Databricks integration with Fabric, see our documentation here. Google Cloud Storage shortcuts and S3 Compatible shortcuts (generally available) GCS shortcuts and S3 Compatible shortcuts are now generally available. Utilize shortcuts in OneLake to quickly and easily make data accessible in Fabric. No need to set up pipelines or copy jobs, just create a shortcut and your data is immediately available in Fabric. Don’t forget to enable shortcut caching, GCS and S3 Compatible shortcuts both support caching. This can be enabled in your workspace setting. By enabling shortcut caching, you can reduce your egress costs when accessing data across clouds or service providers. GCS and S3 Compatible shortcuts also support the On-Premises Gateway. You can utilize the gateway to connect to your on-prem S3 compatible sources as well as GCS buckets that are protected by a virtual private cloud. To learn more about shortcuts see our documentation here. REST APIs for OneLake shortcuts (generally available) We recently made big improvements to the REST APIs for OneLake shortcuts, including adding support for all current shortcut types and introducing a new List operation. With these improvements, you can programmatically create and manage your OneLake shortcuts. We’re excited to announce that these APIs are now Generally Available! OneLake SAS (public preview) Support for short-lived, user-delegated OneLake SAS is now in public preview. This functionality allows applications to request a User Delegation Key backed by an Entra ID, and then use this key to construct a OneLake SAS token. This token can be handed off to provide delegated access to another tool, node, or user, ensuring secure and controlled access. Data Warehouse Copilot for Data Warehouse (public preview) Copilot for Data Warehouse in public preview! Copilot for Data Warehouse is an AI assistant that helps developers generate insights through T-SQL exploratory analysis. Copilot is contextualized to your warehouse’s schema. With this feature, data engineers and data analysts can use Copilot to: Generate T-SQL queries for data analysis. Explain and add in-line code comments for existing T-SQL queries. Fix broken T-SQL code. Receive answers regarding general data warehousing tasks and operations. Learn more about Copilot for Data Warehouse. Copilot for Data Warehouse is currently only available in the Warehouse. Make sure you have Copilot enabled in your tenant and capacity settings to take advantage of these capabilities. Copilot in the SQL analytics endpoint is coming soon. Delta column mapping in the SQL analytics endpoint (public preview) SQL analytics endpoint now supports Delta tables with column mapping enabled for public preview. Column mapping is a feature of Delta tables that allows users to include spaces, as well as any of these characters, in the table’s column names: ,;{}()\n\t=. The extra characters in the column names are shown in the Lakehouse, brought through into the SQL Analytics endpoint, the semantic model, and into Power BI reports. Enabling SQL analytics endpoint on schema enabled Lakehouse’s (public preview) We are enabling the SQL analytics endpoint on schema enabled lakehouses. This allows delta tables in schemas to be queried in the SQL analytics endpoint. New editor improvements for Data Warehouse and SQL analytics endpoint We are excited to share key improvements to our new editor in Fabric Data Warehouse and SQL analytics endpoint to improve the consistency and efficiency of SQL developers' experiences! Starting with the ribbon, the actions presented would change depending on your context in our previous editor. This lacked consistency with other Fabric tools and required extra click stops to launch a certain experience from the ribbon. To make the ribbon more intuitive, we have improved our ribbon to be unified by consolidating all dev tools in a single location split by two tabs for a streamline workflow. The Home and Reporting tab now consists of features for an end-to-end developer experience with no overlaps and no added actions depending on your context. Our new data grid within data preview and displaying query results now provides added capabilities at the column level including sort by ascending and descending order, select, and specify values to filter out per column along with many more features to come. These capabilities can help you quickly analyze and filter your data without running any T-SQL. We’ve listened to your feedback on scenarios where you need to look at multiple experiences at once when writing queries. For example, checking on the data in the tables and the relationships between tables in the BI models to decide how to better structure the query. Within the editor, multitasking between dynamic tabs is now supported between different experiences such as data preview, querying, and modeling for a more efficient data analyzing experience. The multitasking navigation between warehouses and SQL analytics endpoints has also been improved so that whether you’re on a data preview, SQL query, or modeling tab, you can smoothly transition between warehouses/SQL analytics endpoints, and it persists in your last activity. Database Migration Experience (private preview) We are excited to announce the opening of a Private Preview for a new Migration Experience. Designed to accelerate the migration of SQL Server, Synapse dedicated SQL pools, and other warehouses to the Fabric Data Warehouse, users will be able to migrate the code and data from the source database, automatically converting the source schema and code to Fabric Data Warehouse, helping with data migration, and providing AI powered assistance. Please contact your Microsoft account team if you are interested in joining the preview. TSQL Notebook (public preview) You can now use Fabric Notebooks to develop your Fabric warehouse and consume data from your warehouse or SQL analytics endpoint. The ability to create a new notebook item from the warehouse editor lets you carry over your warehouse context into the notebook and use rich capabilities of notebook to run T-SQL queries. T-SQL notebook enables you to execute complex T-SQL queries, visualize results in real-time, and document your analytical process within a single, cohesive interface. The embedded rich T-SQL IntelliSense and easy gestures like Save as table, Save as view or Run selected code provides familiar experiences in the notebook experience to increase your productivity. Learn more here. Nested Common Table Expression (public preview) Fabric Warehouse customers now can use Nested Common Table Expression (NCTE) to deconstruct ordinarily complex queries into smaller reusable blocks. NCTE simplifies complex query code, improves query readability, and query code investigation. With this addition, Fabric Warehouse now supports three types of CTE. They are standard, sequential, and nested CTE. A standard CTE doesn't reference or define another CTE in its definition. A nested CTE's definition includes defining another CTE. A sequential CTE's definition can reference an existing CTE but can't define another CTE. Watch the Data Warehouse demo Data Engineering High Concurrency mode for Notebooks in Pipelines (public preview) We are excited to announce the public preview of High Concurrency mode for Notebooks in Pipeline. This new feature in Microsoft Fabric enables users to share Spark sessions across multiple notebooks within a pipeline. Pipelines are primarily used for orchestrating data engineering tasks for production workloads and scheduled jobs. For enterprise data teams, optimizing resource utilization and achieving the best price-performance ratio is crucial for faster job start times and efficient compute usage. With High Concurrency Mode, users can trigger pipeline jobs, and these jobs are automatically packed into existing high concurrency sessions. Subsequent notebook steps benefit from a 5-second session start experience, even with custom compute configurations and custom pools, resulting in a 30x performance boost and instant session start. Note: Session sharing is always confined to a single user, workspace, and pipeline boundary. Sessions are selected based on matching compute configurations, library management dependencies, and file system dependencies. Learn more about high concurrency mode for notebooks in pipelines from our documentation. Workspace Level Setting to Reserve Maximum Cores for Jobs in Fabric Data Engineering (public preview) We're pleased to introduce a new workspace-level setting that allows you to reserve maximum cores for your active jobs for Spark workloads. By default, optimistic job admission is enabled in all workspaces, and it allows jobs to start with their minimum node configuration and scale up to multiple executors based on available capacity. In cases where there are excessive jobs running, pushing the capacity to its maximum limits, scale-up requests may be rejected. For enterprise customers requiring absolute maximum core reservations for critical jobs, you can now enable a compute reservation model. Workspace administrators can enable this option by navigating to the Data Engineering/Science section of the workspace settings and activating the "Reserve maximum cores for active Spark jobs" setting. Once enabled, the maximum auto scale size of the Spark pool is considered during job admission. Even if a job is currently running with 2 nodes, if the pool's maximum limit is 5, the other 3 nodes will be reserved for the job's potential growth throughout its lifetime. This will ensure that each job that’s submitted has the cores available for it to grow its maximum scale. Users now have the flexibility to choose between optimistic job admission and compute reserved mode, tailoring their workspace settings to meet specific workload requirements. Learn more about settings to reserve maximum cores for your Fabric data engineering jobs from our documentation. Session Expiry Control in Workspace Settings for Notebook Interactive Runs (public preview) We're pleased to introduce a new session expiry control in Data Engineering/Science workspace settings. This feature empowers administrators to set the maximum expiration time limit for notebook interactive sessions. Notebooks are a popular choice for interactive querying, and by default, sessions expire after 20 minutes. With this new setting, you can now customize the maximum expiration duration, helping to prevent unused sessions from consuming unnecessary capacity and potentially impacting the performance of new incoming job requests. If users require additional time, they can extend the session duration using the "extended session time" option available in the monitoring status view within the notebook experience. Learn more about session expiry settings for interactive notebooks sessions from our documentation. Spark Connector for Fabric DW – New Features Recently, we launched Fabric Spark connector for Fabric Data Warehouse (DW) in Fabric Runtime to empower Spark developers or data scientists to access and work on data from Fabric DW and SQL analytics endpoint of the lakehouse (either from within the same workspace or from across workspaces) with a simplified Spark API. This initial version supported reading data from a table or view only in Scala. We are happy to announce that we have released these additional capabilities: Support for custom or pass-through query Support for PySpark Support for Fabric Runtime 1.3 (Spark 3.5) To learn more about Spark Connector for Fabric Data Warehouse (DW) with its recent updates, please refer to the documentation at: Spark connector for Fabric Data Warehouse. T-SQL Notebook (public preview) T-SQL notebook is now available for public preview. This enhancement broadens our language support, extending from a Spark-centric approach to including T-SQL as well. With this update, T-SQL developers can now utilize Notebook for crafting their T-SQL queries to develop a warehouse. They can organize extensive queries into separate code cells and use Markdown cells for enhanced documentation, offering more comprehensive documentation experience. Just like we can add Lakehouse into the notebook, it's enabled to add a Data Warehouse or SQL analytics endpoint into the notebook. This allows you to run T-SQL code directly against the connected warehouse or SQL analytics endpoint. BI Analysts can also take advantage of the update by utilizing T-SQL Notebook to execute cross-database queries. This will enable them to compile business insights from various warehouses and SQL analytics endpoints. Most of the existing features are directly accessible for T-SQL notebooks. For instance, T-SQL developers can take advantage of comprehensive charting tools to visualize the results of their T-SQL queries, as well as collaborate with colleagues to jointly develop the notebook. (collaborate in a notebook) To create a T-SQL notebook from an existing data warehouse, a new entry named “New SQL query in notebook” is added under the “New SQL query” menu group. This action generates a new Notebook in the same workspace, with the data warehouse automatically added into it. You can create a notebook code cell with T-SQL as the language and run the query against the connected Data Warehouse. Fabric Spark Diagnostic Emitter: Collect Logs and Metrics (public preview) Fabric Apache Spark Diagnostic Emitter is now in public preview, a powerful new feature that allows Apache Spark users to collect logs, event logs, and metrics from their Spark applications and send them to various destinations, including Azure Event Hubs, Azure Storage, and Azure Log Analytics. This feature provides robust support for monitoring and troubleshooting Spark applications, enhancing your visibility into application performance. What Does the Diagnostic Emitter Do? The Fabric Apache Spark Diagnostic Emitter enables Apache Spark applications to emit critical logs and metrics that can be used for real-time monitoring, analysis, and troubleshooting. Whether you're sending logs to Azure Event Hubs, Azure Storage, or Azure Log Analytics, this emitter simplifies the process, allowing you to collect data seamlessly and store it in the destination that best suits your needs. Key Benefits of the Apache Spark Diagnostic Emitter Centralized Monitoring: Send logs and metrics to Azure Event Hubs, Azure Log Analytics, or Azure Storage for real-time data streaming, deep analysis and querying, as well as long-term retention. Flexible Configuration: Easily configure Spark to emit logs and metrics to one or more destinations, with support for connection strings, Azure Key Vault integration, and more. Comprehensive Metrics: Collect a wide range of logs and metrics, including driver and executor logs, event logs, and detailed Spark application metrics. To learn more, see: Monitor Apache Spark applications with Azure Log Analytics Collect your Apache Spark applications logs and metrics using Azure Event Hubs Collect your Apache Spark applications logs and metrics using Azure Storage account Environment Artifact integration with Synapse VS Code extension Microsoft Fabric environments is a consolidated item for all your hardware and software settings. In an environment, you can select different Spark runtimes, configure your compute resources, install libraries from public repositories or local directories and more. To learn more, see Create, configure, and use an environment in Fabric. By supporting the Environment item within the Synapse VS Code extension, you can explore and manage the Environment from VS Code side. Expanding the new node of the Environment within VS Code, you can see all the Environment items from the selected workspace and easily identify the workspace default one. You can switch the workspace default to some other item by hovering over the environment and selecting the Set Default Workspace Environment button. Hovering the environment and selecting the Inspect button, the environment details should be displayed in the right panel in JSON format. You can find out the association between the code item, such as Notebook, and the environment from the code item detail property panel. Notebook debug within vscode.dev (public preview) Vscode.dev is a lightweight version of VS Code running fully in the browser. We released the extension Synapse VS Code -Remote last year to enable the edit and run Fabric Notebook within vscode.dev. We are excited to announce the introduction of the debug feature for Notebook today. This update brings us closer to achieving a pro-dev experience with Notebook. Once you've opened the Notebook through the VS Code (Web) interface, you can place breakpoints and debug your Notebook code, which is helpful for troubleshooting. This update first starts with the Fabric Runtime 1.3 Adding Python support in Fabric User Data Functions Introducing support for Python functions in Fabric User Data Functions. Private preview users can now create and run functions that will run on Python version 3.11. This enables users to leverage powerful Python libraries such as pandas, numpy, seaborn and more. These functions are compatible with the existing Fabric data source integrations, so users can interact with Data Warehouses as well as Lakehouses. This feature is available for users of the Private Preview of User Data Functions. To participate in this preview, please fill out the application form. Invoke Fabric User Data Functions in Notebook You can now invoke User Defined Functions (UDFs) in your PySpark code directly from Microsoft Fabric Notebooks or Spark jobs. This integration makes it easy to call reusable functions across different workspaces, reducing the need to rewrite or refactor code. With NotebookUtils integration, invoking UDFs is as simple as writing a few lines of code. Whether you’re working in a Notebook or running batch jobs using Spark Job Definitions, the process is incredibly straightforward. Below is an example of invoking UDFs in Notebooks. Functions Hub is now available in Fabric User Data Functions Functions Hub provides a single location to view, access and manage your User Data Functions. Users can access this feature in the left navigation bar, or in the Data Engineering experience. In Functions Hub, users can see all the functions they have access to and perform actions such as manage access permissions, create a new function, view lineage and more. This is helpful to manage User Data Functions artifacts on scale. This feature is available for users of the Private Preview of User Data Functions. To participate in this preview, please fill out the application form. Support for spaces in Lakehouse Delta table names We are pleased to announce the support for spaces in Lakehouse Delta table names, enhancing usability for all users across Fabric. This feature allows you to create and query Delta tables with spaces in their names, such as "Sales by Region" or "Customer Feedback". This aligns with the expectations of Power BI and SQL customers, who are used to working with spaces in table names. Now, you can seamlessly use spaces in table names across Lakehouse Explorer and its experiences, Spark, Shortcuts, and other experiences. All Fabric Runtimes and Spark authoring experiences support table names with spaces. Code completion authoring works as expected, properly escaping the table names with the required backtick characters. Fabric Runtime 1.3 GA We are extremely excited to announce the advancement of Fabric Runtime 1.3 from Public Preview to General Availability. Our Apache Spark-based big data execution engine, optimized for both data engineering and data science workflows, has been fully updated and seamlessly integrated into the Fabric platform, along with all our components. With this announcement, all new workspaces will, by default, be based on the latest GA runtime version, which is now Fabric Runtime 1.3. Existing workspaces will remain in their current version; however, we strongly encourage you to migrate to Runtime 1.3 as soon as possible to take full advantage of the newest functionalities. The latest enhancements in Fabric Runtime 1.3 include the upgrade to Delta Lake 3.2, updates and upgrades to Python libraries, and improvements to the R language. Additionally, we've implemented query-specific optimizations to further enhance performance and efficiency. Read more here. Native Execution Engine on Runtime 1.3 (public preview) Native Execution Engine for Fabric Runtime 1.3 is now available in public preview. Previously, the Native Execution Engine supported earlier versions of Runtime 1.3, but it is now fully compatible with the latest GA runtime version, which is also based on Delta Lake 3.2. The Native Execution Engine can significantly enhance the performance of your Spark jobs and queries. This engine has been completely rewritten in C++, operates in columnar mode, and utilizes vectorized processing. It offers superior query performance across data processing, ETL, data science, and interactive queries—all directly on your data lake. Importantly, this engine is fully compatible with Apache Spark™ APIs, including the Spark SQL API. The current release of the Native Execution Engine excels particularly in the following scenarios: Working with data in Parquet and Delta formats. Handling queries that involve complex transformations and aggregations, leveraging the engine’s columnar processing and vectorization capabilities. Running computationally intensive queries, as opposed to simple or I/O-bound operations. Best of all, no code changes are required to take advantage of the Native Execution Engine. Read more here. Acceleration tab and UI enablement for the Native Execution Engine No code changes are required to speed up the execution of your Apache Spark jobs when using the Native Execution Engine. You have the flexibility to activate the Native Execution Engine either through your environment settings or selectively for an individual notebook or job. By enabling this feature within your environment settings, all subsequent jobs and notebooks associated with that environment will automatically inherit this configuration. We are also excited to share that enabling the Native Execution Engine is now even more accessible. You can activate it not only through Spark settings but also via the Environment tab, specifically under the "Acceleration" section in the UI, ensuring a seamless and straightforward enablement process. Fabric Spark Runtimes Release Notes We are committed to ensuring that our Microsoft Fabric Spark runtimes remain at the forefront of performance and security. In our ongoing efforts, we occasionally introduce updates that may affect your experience. While we understand the critical nature of these updates, we recognize the challenges in keeping up with changes, identifying affected components, and understanding the rationale behind these modifications. To enhance your experience, we are now providing detailed, automated release notes for each Apache Spark-based runtime we deliver. These notes will be readily accessible, offering complete transparency and clarity regarding the updates, including their purpose. This will empower you to seamlessly adapt your workloads and fully leverage the benefits of each update. For the most current information, including a comprehensive list of changes and specific release notes for each runtime version, we encourage you to check and subscribe to Spark Runtimes Releases and Updates. Enable/Disable Functionality in API for GraphQL The Enable/Disable feature for queries and mutations in Microsoft Fabric's GraphQL API provides administrators and developers with granular control over API access and usage. This functionality allows you to selectively activate or deactivate specific queries and mutations within your GraphQL schema, giving you the ability to manage API capabilities dynamically without altering the underlying code or deploying changes. By leveraging this feature, you can permanently or temporarily disable certain operations for maintenance, gradually roll out new functionality, or restrict access to sensitive data operations as needed. This level of control enhances security, aids in API versioning, and provides flexibility in managing your GraphQL API's behavior to align with your application's evolving requirements and operational needs. To disable a query or mutation within a GraphQL item, simply select the ellipses next to the query or mutation you want to disable. Then select the Disable option from the pop-up menu. If a query or mutation has been disabled, you will see it grayed out on the Schema Explorer. You can enable it by simply choosing Enable from the entries’ pop-up menu. Public REST API of Livy Endpoint The Fabric Livy endpoint lets users submit and execute their Spark code on the Spark compute within a designated Fabric workspace, eliminating the need to create any Notebook or Spark Job Definition artifacts. This integration with a specific Lakehouse artifact ensures straightforward access to data stored on OneLake. Additionally, Livy API offers the ability to customize the execution environment through its integration with the Environment artifact. When a request is sent to the Fabric Livy endpoint, the user-submitted code can be executed in two different modes: Session Job: A Livy session job entails establishing a Spark session that remains active throughout the interaction with the Livy API. This is particularly useful for interactive and iterative workloads. A Spark session starts when a job is submitted and lasts until the user ends it or the system terminates it after 20 minutes of inactivity. Throughout the session, multiple jobs can run, sharing state and cached data between runs. Batch Job: A Livy batch job entails submitting a Spark application for a single execution. In contrast to a Livy session job, a batch job does not sustain an ongoing Spark session. With Livy batch jobs each job initiates a new Spark session, which ends when the job finishes. This approach works well for tasks that don't rely on previous computations or require maintaining state between jobs. The endpoint of the session job API would look like: https://api.fabric.microsoft.com/v1/workspaces/ws_id/lakehouses/lakehouse_id /livyapi/versions/2023-12-01/ batches The endpoint of the session job API would look like: https://api.fabric.microsoft.com/v1/workspaces/ws_id/lakehouses/lakehouse_id /livyapi/versions/2023-12-01/sessions ws_id: this is the workspace in which the hosting Lakehouse artifact belongs to. lakehouse_id: this is the artifact id of the hosting Lakehouse. All the capacity consumption history from this batch API call will be associated with this artifact. To access the Livy API endpoint, you need to create a Lakehouse artifact. After it's set up, you'll find the Livy API endpoint in the settings panel. The “Recent Run” section of the Lakehouse will now display all submitted requests. Click on the “Application name” column to open the detailed monitoring page and view additional logs. Typically, the Livy request is directed to the workspace starter pool by default. However, users can customize the execution by including the environment artifact ID in the payload body, utilizing the settings from the specified environment artifact. In this example, the value of the id is the artifact id of the environment, the workspaceId is the id of the current workspace. In the current version, the environment and Lakehouse artifacts should belong to the same workspace. { "conf": { "spark.fabric.environmentDetails": "{\"id\":\"558bd4a3-5107-413e-ad6d-48a4b80678f6\",\"workspaceId\":\"ea0f47a3-8f30-4cf9-bfb6-0b3af37d9eb8\"}" } } Watch the Data Engineering demos Data Science Announcing Public Preview: Share Feature for Fabric AI Skill The highly anticipated feature for Fabric AI Skill, the "Share" capability is now in public preview. This powerful addition allows you to share the AI Skill with others using a variety of permission models, providing you with complete control over how your AI Skill is accessed and utilized. With this new feature, you can: Co-create: Invite others to collaborate on the development of your AI Skill, enabling joint efforts in refining and enhancing its functionality. View Configuration: Allow others to view the configuration of your AI Skill without making any changes. Query: Enable others to interact with the AI Skill to obtain answers to their queries. Additionally, we are introducing flexibility in managing versions. You can now switch between the published version and the current version you are working on. This feature facilitates performance comparison by running the same set of queries, providing valuable insights into how your changes impact the AI Skill’s effectiveness. We’ve also refined the publishing process. You can now include a description that outlines what your AI Skill does. This description will be visible to users, helping them understand the purpose and functionality of your AI Skill. We are excited for you to explore these new capabilities and look forward to hearing about your experience. Your feedback is crucial as we continue to enhance your experience with Fabric AI Skill. Data Wrangler now supports Spark DataFrames and PySpark code generation (generally available) Data Wrangler, a notebook-based tool for exploratory data analysis, now works for both pandas DataFrames and Spark DataFrames in general availability. You can use the tool to explore and transform data in your notebook with an immersive visual interface, generating either Python code or PySpark code in just a few steps. If you’ve used Data Wrangler before, the steps will be familiar. You can begin by opening any active pandas or Spark DataFrame from the “Data Wrangler” prompt in the notebook ribbon. Data Wrangler will automatically convert Spark DataFrames to pandas samples for performance reasons. However, all the generated code will ultimately be translated to PySpark when you save it back to your notebook. (You can also modify the sample size and sampling method by selecting “Choose custom sample” from the Data Wrangler dropdown.) As with a pandas DataFrame, Data Wrangler will show you descriptive insights about your data in the right-hand “Summary” panel and in each column’s header. You can use the left-hand “Operations” panel to browse and apply common data-cleaning transformations. Selecting one will prompt you to provide a target column or columns, along with any necessary parameters. As you fill in those values, a preview of the applied operation, along with the corresponding code, will be automatically generated. If you apply the previewed step, Data Wrangler’s display grid and summary statistics update to reflect the results. The code appears in a running list of committed operations. After applying a series of steps, you can copy or save the generated code using options in the toolbar above the display grid. For Spark DataFrames, all the code generated on the pandas sample is translated to PySpark before it lands back in the notebook. Data Wrangler will display a preview of the translated PySpark code and provide an option to export the pandas code as well. Announcing new usability improvements for Data Wrangler You may have noticed that Data Wrangler has a new entry point under the “Home” tab of the Fabric notebook ribbon. We’re excited to share additional usability updates designed to improve your experience. Launch Data Wrangler directly from a cell You can now open Data Wrangler directly from a notebook cell when you use the Fabric display() command to print a pandas or PySpark DataFrame. Just click on the “Data Wrangler” prompt in the interactive display output. Work faster with improved performance Thanks to new engineering updates, you’ll notice that Data Wrangler takes less time to apply an operation once you’ve loaded the preview. We’ve cut down the latency to speed up your workflow. Modify your display using the new “Views” prompt Data Wrangler works best on large monitors, but you can now use the “Views” dropdown above the display grid to minimize or hide parts of the interface based on your preferences or screen size. File editor in Notebook We are excited to share the launch of the file editor feature in Fabric Notebook. This new feature allows users to view and edit files directly within the notebook's resource folder and environment resource folder in notebook. Supported file types include CSV, TXT, HTML, YML, PY, SQL, and more. With the file editor, you can: View and edit files: Easily access and modify files within the notebook. Keyword highlighting for module files: Provide necessary language service when opening and editing code files like .py and .sql. Enhanced user experience: Integrated with the notebook pane switcher to help you easily navigate different features inside notebook. Watch the Data Science demos Real-time Intelligence Real-Time Intelligence is a powerful service that empowers everyone in your organization to extract insights and visualize their data in motion. With capabilities across ingestion, processing, transformation, analytics, visualization and action, Real-Time Intelligence transforms your data into a dynamic, actionable resource that drives value across the entire organization. Learn more here. Creating a Real time Dashboard by Copilot From the list of tables in Real-Time hub, users can click on the three dots menu and select create real-time dashboard. Copilot will review the table and automatically create a dashboard with two pages, one with insights about the data in the table and one page that contains a profile of the data with a sample, the table schema and more details about the values in each column. This can be further explored and edited to make it easy for users to find insights on their time-series data without having to write a single line of code. Insight page: Profile page: Adding a real-time dashboard to an org app Real-time dashboards can be included in the new version of org app. The org app can include PBI reports, PBI paginated reports, Notebooks and Real time dashboards. This makes it very easy to distribute RTD to large user cohorts outside a specific Workspace. An org app with a PB report and an RTD: Introducing A New Real-Time Hub User Experience We are thrilled to announce the launch of the new Real-Time Hub user experience (UX), a redesigned and enhanced experience that will help you get the most out of your real-time data. Whether you want to monitor, process, analyze, or act on your data streams, the new UX will make it easier and faster than ever. What's new? A new left navigation provides you with seamless access to all your data-in-motion. This includes both the data streams that you have access to and the ones that you have integrated into Fabric, as well as Fabric events and Azure resources. With just a few clicks, you can now effortlessly navigate through these resources in a more user-friendly way. A new page called “My Streams” where you can bring in external events from various sources and integrate them with your real-time data. You can use My Streams to create custom streams that combine data from different sources, including Microsoft sources, Database CDC, and Fabric events. “Get Events” button is now rebranded as “Add source”, which aligns with the value proposition of Real-Time hub to allow customers to not only discover, manage, and consume events, but also all the other types of data-in-motion. Four new Eventstream connectors have been introduced into the Real-Time hub. Now you can stream data from Azure SQL MI DB (CDC), SQL Server on VM DB (CDC), Apache Kafka, and Amazon MSK Kafka. These sources will enable you to enrich your real-time data with historical and analytical data from your data warehouse or database. To learn more about these four connectors, continue reading 'New Stream Sources Now Available' below. Simply sign in to Microsoft Fabric and start experiencing the new Real-Time hub UX! New Streaming Sources Now Available via Eventstream Connectors We’re excited to announce the expansion of our Eventstream Connectors with four powerful new streaming sources, enabling even more seamless data streaming across your ecosystem: Azure SQL MI CDC: Capture data changes in your Azure SQL Managed Instance (MI) database and stream them into Eventstream for real-time processing, analysis, and monitoring. SQL Server on VM CDC: Capture data changes in your SQL Server on Virtual Machines and stream them directly into Eventstream for processing, analysis, and monitoring. Apache Kafka: Easily stream data from Apache Kafka cluster into your Eventstream, enabling unified processing and analysis across platforms. Amazon MSK Kafka: Seamlessly stream data from Amazon Managed Streaming for Apache Kafka (MSK) into Eventstream and perform real-time processing. These new sources empower you to build richer, more dynamic Eventstreams, ensuring that your real-time streaming applications can fully leverage the reporting and analysis capabilities in Fabric. Introducing Eventhouse as a new Destination in Eventstream As part of our ongoing efforts to enhance Eventstream’s capabilities, we are excited to introduce Eventhouse as a new destination for your data streams. Eventhouses, equipped with KQL Databases, are designed to handle and analyze large volumes of data, particularly in scenarios that demand real-time analytics and exploration. With the Eventhouse destination in Eventstream, you can efficiently process and route data streams into an Eventhouse and analyze the data in near real-time using KQL. Eventstream’s Integration with Managed Private Endpoint We’re excited to introduce the Private Network feature for Fabric Eventstream! With Fabric’s Managed Private Endpoint, you can now establish a private connection between your Azure services, such as Azure Event Hub, and Fabric Eventstream. This integration ensures your data is securely transmitted within a private network, allowing you to explore the full power of real-time streaming and high-performance data processing that Eventstream offers. The diagram below shows how Eventstream pulls data from your Azure Event Hub within a virtual network using Fabric’s Managed Private Endpoint. To learn more about Managed Private Endpoint, visit here: Overview of managed private endpoints for Microsoft Fabric - Microsoft Fabric | Microsoft Learn Introducing the new look and feel of KQL Database We’re excited to unveil the newly redesigned KQL DB experience in Microsoft Fabric Real Time Intelligence! This update brings a clean, modern interface packed with powerful features to help you work smarter, whether you’re managing several databases, dealing with streaming data, or exploring massive datasets. A UI Tailored to Your Workflow The new KQL DB interface is built to accommodate the diverse needs of data professionals. Whether you're exploring a simple database or navigating one with hundreds of tables, the design remains intuitive and adaptable. An Enhanced Database Page Experience The new KQL database page has undergone a significant transformation. When you open the Tables tab of the database, you’re greeted with visually rich cards (tiles) that present key metrics briefly.4 These tiles display ingestion trends, data size, recent active users, and the availability status in OneLake. For those who prefer more detail, you can switch to the list view, offering a comprehensive table-focused perspective. Exploring your data is now more interactive with the updated Data Preview tab. Here, you can instantly see the most recent records across all tables in a convenient JSON view, allowing for quick insights without needing to dive too deep. If you’re looking to drill down further, the updated histograms and time filters let you see the overall data ingestion trend in more depth. Diving Deeper into Tables The individual table view has seen improvements, too. The Data Preview of a table lets you explore records with easy-to-use interface as the database-level preview. Also, the Schema tab gives you a complete picture of the data structure, including distribution stats and column details. Histograms in the table view bring your data to life, allowing you to explore different time ranges and customize binning to suit your analysis needs. All this is wrapped in a design that’s both accessible and easy to navigate. Flexible Views for Any Data Scope The new look and feel of KQL DB in Microsoft Fabric offers a more intuitive, powerful, and visually engaging way to work with your data. We can’t wait for you to try it out and see how these enhancements elevate your experience! To read more about the new KQL DB look and feel, check out the documentation. Set alerts on KQL Querysets with Data Activator triggers Now you can set up Data Activator alerts directly on your KQL queries in KQL querysets. Whether you’re tracking real-time metrics or keeping an eye on important logs, this feature makes sure you get notified the moment something important happens. Set Alerts Based on KQL Query Results or Specific Conditions With this new feature, you can set alerts to trigger based on specific results or conditions from a scheduled KQL query. For example, if your KQL DB tracks application logs, you can configure an alert to notify you if the query, scheduled at a frequency of your choice (e.g., every 5 minutes), returns any logs where the message field contains the string "error". This feature also lets you monitor live data trends by setting conditions on visualizations, similar to how you can set alerts on visuals within Real-Time Dashboards. For instance, if you visualize sales data distribution across product categories in a pie chart, you can set an alert to notify you if the share of any category drops below a certain threshold. This helps you quickly identify and address potential issues with that product line. You can choose whether to receive alerts via email or Teams messages when the condition is met. To read more about setting alerts for KQL querysets, check out the documentation. Data Exploration made easier with Top values feature The "Top Values" feature in our no-code data exploration UI is available for Real-Time Dashboards’ tiles! This new component makes it easier than ever to explore and understand your data without writing a single line of code. Instantly view the top values for each column in your query result set, allowing you to see value distributions and gain insights into the nature of your data. Plus, seamlessly add new filters directly from this component to refine your analysis. Real-Time Dashboard lower than ever refresh rate We are pleased to share an enhancement to our dashboard auto refresh feature, now supporting continuous and 10 seconds refresh rates, in addition to the existing options. This upgrade, addressing a popular customer request, allows both editors and viewers to set near real-time and real-time data updates, ensuring your dashboards display the most current information with minimal delay. Experience faster with data refresh and make more timely decisions with our improved dashboard capabilities. As the dashboard author you can enable the Auto refresh setting and set a minimum time interval, to prevent users from setting an auto refresh interval smaller than the provided value. Note that the Continuous option should be used with caution. The data is refreshed every second or after the previous refresh is completed if it takes more than 1 second. Multivariate anomaly detection Now you can use the power of Eventhouse, Spark, and OneLake to perform real-time monitoring and multivariate anomaly detection of time series data. We added a workflow that is based on the algorithm that is used in the AI Anomaly Detector service (which is being retired as a standalone service). Multivariate anomaly detection is a method of detecting anomalies in the joint distribution of multiple variables over time. This method is useful when the variables are correlated, thus the combination of their values at specific times might be anomalous, while the value each variable by itself is normal. Multivariate anomaly detection can be used in various applications, such as monitoring the health of complex IoT systems, detecting fraud in financial transactions, identifying unusual patterns in network traffic and more. Multivariate anomaly detection in Fabric takes advantage of the powerful Spark and Eventhouse engines on top of a shared persistent storage layer. The initial data can be ingested into an Eventhouse and exposed in OneLake. The anomaly detection model can then be trained using the Spark engine. The trained model is stored in Fabric MLflow models registry. Predictions of anomalies on new streaming data can be done in real time using the Eventhouse engine. The interconnection of these engines that can process the same data in the shared storage allows for a seamless flow of data from ingestion, via model training, to anomalies prediction. For more details see an overview and tutorial. Enhanced Time Series Analysis: More Control, Better Insights Whether you are using Real-Time Dashboards or KQL Querysets for time series analysis, you’ll find this new way of interacting with your data visualization valuable. You can now interact with the data by selecting specific items from the legend using the mouse, using Ctrl to add or remove selections, or holding Shift to select a range. The Search button helps you quickly filter items, while the Invert button allows you to reverse your selections. Navigate through your selections with ease using the Up and Down arrows to refine your data view. Real-Time Intelligence Copilot conversational mode We'd like to share an upgrade to our Copilot assistant, which translates natural language into KQL. Now, the assistant supports a conversational mode, allowing you to ask follow-up questions that build on previous queries within the chat. This enhancement enables a more intuitive and seamless data exploration experience, making it easier to refine your queries and dive deeper into your data, all within a natural, conversational flow. New Rule Creation Experience We’re excited to announce the launch of a new rule creation experience within Data Activator. This innovative experience unlocks new capabilities to monitor your events with more fine-tuned controls at greater scale, ensuring you solve and stay on top of your operational business challenges with ease. The new unified experience allows you to easily bring in your data, identify the business object you want to monitor, quickly and easily create business rules on top of them, and automatically carry out actions when the rule is met. This feature is not only a new and improved experience with the same functionality as is available today, but also lays the groundwork for new modeling capabilities in the future. With this update you’ll be able to: Stay on top of your critical metrics by monitoring your business objects. You can track and analyze key business objects such as individual packages, households, refrigerators, and more in real-time, ensuring you have the insight needed to make informed decisions. Whether it’s understanding how individual instances of your business objects impact sales figures, inventory levels, or customer interactions, our monitoring system provides detailed insights, helping you stay proactive and responsive to changes in your business environment at a fine-tuned level of granularity. Unlock the full potential of creating business rules on your data with advanced data filtering and monitoring capabilities. This update offers a wide array of options for filtering, summarizing, and scoping your data, allowing you to tailor your analysis to your specific needs. You can set up complex conditions to track when data values change, exceed certain thresholds, or when no new data has arrived within a specified timeframe. Ensure your communications are perfect before hitting send by seeing a preview of your Email and Teams messages. This will allow you to see a preview of your message exactly as it will appear to the recipient. Review your content, check formatting, and make any necessary adjustments to ensure clarity. With this feature, you can confidently have Data Activator send messages on your behalf knowing they look just the way you intended. Set up rules that trigger automatically with every new event that comes in on your stream of data. Whether you need to send notifications or initiate workflows, this feature ensures that your processes are always up-to-date and responsive. These are just some of the highlights the new Data Activator rule creation experience provides, but there is much more to explore. To learn more about how this update can help you create and manage your business rules more easily and efficiently, check out our detailed blog post that covers all the features and benefits in depth. We’ve made it easier to alert your teammates in Power BI We’ve streamlined the process of creating Data Activator alerts for your teammates in Power BI. Previously, you had to create your alert in Power BI, then open it in Data Activator to add your teammates as recipients. Now, you can add recipients directly within Power BI. We’re rolling this out gradually during the coming few weeks, so look out for it during the month of September when you click “Set Alert” in Power BI. For the latest updates and what's coming in Real-Time Intelligence check our out documentation and roadmap. Watch the Real-Time Intelligence demos Data Factory Dataflow Gen2 Copilot in Dataflow Gen2 (generally available) Earlier this year, we released a Public Preview of the Copilot for Data Factory capabilities in Dataflows Gen2. Today, we are excited to announce that these capabilities are now Generally Available. Copilot in Fabric enhances productivity, unlocks profound insights, and facilitates the creation of custom AI experiences tailored to your data. As a component of the Copilot in Fabric experience, Copilot in Data Factory empowers customers to use natural language to articulate their requirements for creating data integration solutions using Dataflow Gen2. Essentially, Copilot in Data Factory operates like a subject-matter expert (SME) collaborating with you to design your dataflows. Copilot for Data Factory is an AI-enhanced toolset that supports both citizens and professional data wranglers in streamlining their workflow. It provides intelligent Mashup code generation to transform data using natural language input and generates code explanations to help you better understand earlier generated complex queries and tasks. Learn more about how to get started with Copilot for Data Factory: Copilot for Data Factory overview - Microsoft Fabric | Microsoft Learn. Fast Copy in Dataflow Gen 2 (general available) The Fast Copy feature is now generally available. This functionality enables efficient ingestion of large amounts of data, utilizing the same backend as the Copy Activity in data pipelines. It significantly reduces data processing time and enhances cost efficiency. Learn how to enhance performance and cut costs with Fast Copy in Dataflows Gen2. Fast Copy supports numerous source connectors, including ADLS Gen2, Blob storage, Azure SQL DB, On-Premises SQL Server, Oracle, Fabric Lakehouse, Fabric Warehouse, PostgreSQL, and more. An on-premises gateway is also supported, allowing for high-performance data transfer from on-premises sources. For details, see Fast Copy with On-premises Data Gateway Support in Dataflow Gen2. Resources: Docs: Fast copy in Dataflows Gen2 - Microsoft Fabric | Microsoft Learn Incremental refresh for Dataflow Gen2 We are happy to share the introduction of incremental refresh for Dataflows Gen2. This significant enhancement in Microsoft Fabric’s Data Factory is designed to optimize data ingestion and transformation, particularly as your data continues to expand. About Incremental Refresh Incremental refresh enables you to refresh only new or updated data, thus reducing refresh times, enhancing reliability by avoiding long-running operations, and minimizing resource usage. This feature is particularly valuable in scenarios where data volume is substantial, and efficiency is required. Prerequisites To leverage incremental refresh in Dataflows Gen2, ensure the following prerequisites are met: You must have a Fabric capacity. Your data source should support query folding and must include a Date/DateTime/DateTimeZone column for data filtering. Your data destination must support incremental refresh. Supported destinations include Fabric Warehouse, Azure SQL Database, and Azure Synapse Analytics. How to Use Incremental Refresh Create a new Dataflow Gen2 or open an existing one. In the dataflow editor, create a new query that retrieves the data you need to refresh incrementally. Ensure your query returns data with a Date/DateTime/DateTimeZone column for filtering. Verify that the query fully folds to the source system. Right-click the query and select Incremental refresh. Provide the required settings, including the DateTime column, data extraction parameters, and bucket size. Configure any advanced settings if necessary. Publish Dataflow Gen2. Resources: Incremental Refresh Docs Incremental Refresh Blog Certified connector updates Check out the new and updated connectors in this release: New connectors: Kyvos Clickhouse Updated connectors: InformationGrid Are you interested in creating your own connector and publishing it for your customers? Learn more about the Power Query SDK and the Connector Certification program. Watch the Data Factory demo Data pipeline Fabric Pipeline Integration in On-premises Data Gateway (generally available) On-premises connectivity for Data pipelines in Microsoft Fabric is now Generally Available. Using the on-premises Data Gateway, customers can connect to on-premises data sources using data pipelines with Data Factory in Microsoft Fabric. This enhancement significantly broadens the scope of data integration capabilities. In essence, by using an on-premises Data Gateway, organizations can keep databases and other data sources on their on-premises networks while securely integrating them into Microsoft Fabric (cloud). Learn more: How to access on-premises data sources in Data Factory - Microsoft Fabric | Microsoft Learn Invoke remote pipeline in Data pipeline We’ve been working diligently to make the very popular data pipeline activity known as “Invoke Pipeline” better and more powerful. Based on customer feedback, we continue to iterate on the possibilities and have now added the exciting ability to call pipelines from Azure Data Factory (ADF) or Synapse Analytics pipelines as a public preview! This opens tremendous possibilities to utilize your existing ADF or Synapse pipelines inside of a Fabric pipeline by calling it inline through this new Invoke Pipeline activity. Using cases that include calling Mapping Data Flows or SSIS pipelines from your Fabric data pipeline will now be possible. We will continue to support and include the previous Invoke Pipeline activity as “legacy”, without the support for ADF or Synapse remote pipeline invocation and without child pipeline monitoring in Fabric. But for the latest features like remote invocation and child pipeline monitoring, you can use the new Invoke Pipeline. Spark Job environment parameters One of the most popular use cases in Fabric Data Factory today is automating and orchestrating Fabric Spark Notebook executions from your data pipelines. A very common request has been to reuse existing Spark sessions to avoid any session cold-start delays. And now we’ve delivered on that requirement by enabling “Session tags” as an optional parameter under “Advanced settings” in the Fabric Spark Notebook activity! Now you can tag your Spark session and then reuse the existing session using that same tag. Watch the Data pipeline demos Mirroring Mirroring Azure SQL Database Mirroring Azure SQL Database in Fabric now extends our capabilities to support mirrored tables within additional SQL’s Dynamic Definition Language (DDL). Now, operations such as Drop Table, Rename Table, and Rename Column can be seamlessly executed while tables are in the process of mirroring. Click here to watch a demo. New Azure Data Factory Item Bring your existing Azure Data Factory (ADF) to your Fabric workspace! We are introducing a new preview capability that allows you to connect to your existing ADF factories from your Fabric workspace. By clicking “Create Azure Data Factory” inside of your Fabric Data Factory workspace, you will now be able to fully manage your ADF factories directly from the Fabric workspace UI! Once your ADF is linked to your Fabric workspace, you’ll be able to trigger, execute, and monitor your pipelines as you do in ADF but directly inside of Fabric. Copy Job (public preview) We'd like to introduce Copy Job, elevating the data ingestion experience to a more streamlined and user-friendly process from any source to any destination. Now, copying your data is easier than ever before. Moreover, Copy job supports various data delivery styles, including both batch copy and incremental copy, offering flexibility to meet your specific needs. With Copy Job, you can enjoy the following benefits: Intuitive Experience: Seamless experience data copying with no compromises, making it easier than ever. Efficiency: Enable incremental copying effortlessly, reducing manual intervention. This efficiency translates to less resource utilization and faster copy times. Flexibility: Empower yourself to customize your data copying behavior. From selecting tables and columns to data mapping and read/write behavior, you have the flexibility to tailor the process to your specific needs. Additionally, enjoy the freedom to set flexible schedules, whether it's a one-time copy or at a regular cadence. Click here to learn more about Copy Job. Watch the Mirroring demosFabric November 2025 Feature Summary
The November 2025 Fabric release introduces several major updates, including the general availability of SQL database, Cosmos DB, and enhanced mirroring support for key data sources such as SQL Server, Cosmos DB, and PostgreSQL. This month also brings new AI-driven features like Copilot sidecar chat tools and real-time data exploration, as well as crucial platform enhancements such as Azure DevOps cross-tenant support, improved security permissions in OneLake, and expanded connectivity through new connectors and developer tooling. These updates are designed to empower users with greater flexibility, intelligence, and control across the Fabric platform. Contents Events and Announcements Get Fabric certified for FREE during Fabric Data Days Your favorite Fabric Community Conference is back – with a twist! Fabric Platform Govern in OneLake for Tenant admins (Preview) Right-click tab menu for easier multitasking Azure DevOps Service Principal & Cross Tenant Support (Generally Available) OneLake OneLake diagnostics (Generally Available) OneLake security ReadWrite permission (Preview) Databases SQL database in Fabric (Generally Available) SQL Auditing (Preview) Customizable PITR backup retention (Generally Available) New Tools in Copilot Sidecar Chat (Generally Available) Data Virtualization support for Fabric SQL database (Preview) Microsoft Python Driver for SQL database - mssql-python (Generally Available) Cosmos DB in Fabric (Generally Available) Data Engineering Spark connector for SQL databases (Preview) ArcGIS GeoAnalytics for Microsoft Fabric Spark (Generally Available) Faster Notebook Loading with Progressive Rendering Fabric Data Engineering VS Code improvement New features in Fabric User Data Functions, Ignite edition Azure Artifact Feed in Fabric Environment (Preview) Data Science Connect Data Agents to your Azure Search Index in Microsoft Foundry Updated Example Query and Instruction Data Agent Limits Fabric AI Functions Enhancements (Generally Available) MCP Server support in Fabric data agents Fabric data agents now integrate with Microsoft 365 Copilot Fabric Data Agents to Support SQL Databases Fabric Data Agent Entry Points in Data Warehouse and Lakehouse Upgrade your machine learning tracking system Supporting Internal Python Packages in ML Model Endpoints Data Warehouse IDENTITY columns (Preview) Data Clustering (Preview) Warehouse Snapshots in Microsoft Fabric Data Warehouse (Generally Available) Varchar(max) support Introducing the HTTP and MongoDB CDC Connectors for Eventstream Introducing Cribil Source in Real-Time Intelligence Eventstream (Preview) Eventstream Activator destination (Generally Available) Capacity Overview events (Preview) Entity Diagram in Eventhouse KQL Database (Preview) Automate advanced actions with Fabric Activator Operations agent (Preview) Imagery file support in Maps (Preview) Data labeling in Maps (Preview) Copilot assisted real time data exploration (Preview) Data Factory Enterprise readiness Snowflake Connector Supports to Use Key Pair Authentication Method Manual Update for On-premises Data Gateway (Preview) Certificate & Proxy Support for Virtual Network Data Gateway (Preview) Pipelines Error Insights summary Copilot for Pipeline Natural Language to Generate and Explain Pipeline Expressions with Copilot (Preview) Hierarchical view for pipelines in Monitoring Hub Connectivity Spark & Impala 2.0 Connectors (Generally Available) Dataflows Modern Get Data in Excel Desktop New importing data flow supported in Modern Get Data Copilot Fabric AI Functions integration with Dataflow Gen2 (Preview) Power Query Language Service IntelliSense in Dataflow Gen2 (Preview) Airflow job New API for Apache Airflow Jobs for Files & Requirements Now you can upload files to your Apache Airflow project from the UI New Apache Airflow Job File Management APIs Mirroring Mirroring for Snowflake Iceberg Tables Support Mirroring for SAP (Preview) Mirroring for SQL Server (Generally Available) Mirroring for Cosmos DB (Generally Available) Mirroring for PostgreSQL (Generally Available) Mirroring for Azure SQL Database – support for UAMI (Preview) Copy job Expanded CDC Support for More Sources & Destinations Full & Incremental Copy Subsets of data with Database Queries Truncate Destination before Full Copy Copy Multiple Folders in one Copy job Connection Parameterization with Variable library for CI/CD (Generally Available) Developer tooling Fabric VS Code extension is now open source https://youtu.be/Ym2ADQv1P7Y?si=Y78E9zHJRD9pH0Y9 Events and Announcements Get Fabric certified for FREE during Fabric Data Days Supercharge your career with 50+ days of data & AI. Join us for 2 months of learning, contests, live sessions, discount exam vouchers and community connection. Through December 5, 2025, you can get your Fabric certification for free with a 100% discount voucher for exams DP-600 and DP-700. Exams must be taken by December 31, 2025. Request your voucher now! Your favorite Fabric Community Conference is back – with a twist! SQLCon Joins FabCon from March 16-20, 2026, in Atlanta, GA! For the first time ever, SQLCon will be co-located with FabCon, bringing together the entire data community in one place. Two Conferences. One pass. One Epic Week. Join us for the ultimate Microsoft Fabric, SQL, Power BI, Real-Time Intelligence, AI, and Databases community-led event. Master SQL Server 2025 internals in the morning, dive into Fabric innovations in the afternoon, attend Power Hour before dinner and network with peers from both communities. The sessions you choose are totally up to you. Register with code FABCOMM to save $200. Fabric Platform Govern in OneLake for Tenant admins (Preview) In today’s data-driven world, effective data governance is crucial for ensuring the integrity, security, and usability of data. The governance experience for Fabric admins within the OneLake catalog is being extended. With this experience we’re empowering Fabric admins with the tools and insights they need to govern and secure their organizational data estate within Fabric. Admins can now review and manage their tenant more efficiently by viewing all monitoring reports in one place with cross-filtering and drill-through features. With copilot, simply select the 'Copilot' icon in the ‘view more’ report to chat with your data, uncover trends, drill into details, and get quick summaries. Learn more about OneLake catalog, Governance in OneLake catalog, Governance and compliance in Fabric in our documentation. Right-click tab menu for easier multitasking A new right-click menu has been implemented for horizontal tabs, enhancing the efficiency of tab management. You can now: Open in new browser tab: Opens the current item into a separate browser tab. Pin tab: Pin important tabs to keep them always visible and easily accessible. Refer to the multitasking improvements documentation to learn more. Azure DevOps Service Principal & Cross Tenant Support (Generally Available) This highly anticipated feature empowers Fabric customers to achieve a comprehensive set of automation processes. Users can develop end-to-end automation flows, from Fabric workspace creation, to seamlessly connect it to their Azure DevOps repository which now can be even reside in a different tenant than their Fabric Home Tenant, using Fabric CLI or leverage Infrastructure as Code (IaC) using Fabric Terraform module, all powered by secure, scalable service principal authentication. To learn more, refer to the Automate Git integration by using APIs documentation. OneLake OneLake diagnostics (Generally Available) OneLake diagnostics, makes it simple to answer ‘who accessed what, when, and how’ across your Fabric workspaces. This enables federated data governance, operational transparency, and compliance reporting on a scale. Because events are stored in your Lakehouse as open JSON files, you can analyze them with the tools you already use—Spark, SQL, Eventhouse, Power BI, or any solution that ingests JSON logs. Because OneLake implements the ADLS and Azure Blob Storage APIs, all that data is accessible outside of Fabric too! OneLake diagnostics can be enabled in workspace settings For more in-depth information, refer to the Gain End-to-End Visibility into Data Activity Using OneLake diagnostics (Generally Available) blog post. OneLake security ReadWrite permission (Preview) OneLake security now supports ReadWrite access controls, giving data owners the ability to define precise permissions for how users can write to data in lakehouses. This enhancement allows teams to assign ReadWrite access to workspace Viewers or users with only Read access. This allows users to write data to tables and folders without having elevated permissions in the workspace to create and manage Fabric items. It’s a critical step toward enabling secure, collaborative workflows without compromising governance. With ReadWrite access, all OneLake write operations can be performed through Spark notebooks, the OneLake File Explorer, or OneLake APIs. This allows teams the flexibility of ensuring the principal of least privilege is followed while also enabling key workflows involving uploading pdfs or excel files for further analysis. To learn more about how ReadWrite access works in OneLake security, check out OneLake security access control model (preview) documentation. Databases SQL database in Fabric (Generally Available) Built on the trusted SQL Server and Azure SQL Database engine, this is the first fully SaaS-native operational database experience within Microsoft Fabric. It empowers developers, data engineers, and IT professionals to build scalable, secure, and intelligent applications faster than ever. To learn more, refer to the SQL database in Fabric documentation. SQL Auditing (Preview) Auditing is needed mainly for Security and Compliance reasons, to ensure transparency about everything that happened in the database. Its logs can later be used to: Compliance auditing (HIPAA, SOX), support investigation and threat analysis Monitoring database activities Tracking permission changes To learn more, refer to the SQL Auditing documentation. Customizable PITR backup retention (Generally Available) Currently Point-in-time restore backups default to 7 days. We are now enhancing the ability to customize this retention to anywhere from 1 to 35 days, depending on your business needs for data retention. This setting can be managed in the database settings in the Backup retention policy option below. To learn more, refer to the SQL database backups documentation. Customer-managed keys in SQL Database (Preview) Microsoft Fabric already encrypts all data-at-rest using Microsoft-managed keys. But for organizations with strict data governance policies or regulatory requirements, Customer-managed keys (CMK) offer an additional layer of control and flexibility. With CMK, you can use your own Azure Key Vault keys to encrypt SQL database data in Fabric workspaces, giving you: Compliance with industry-specific encryption standards Key ownership and rotation control Granular access management To learn more about customer-managed keys in SQL Database, check out the full blog post. New Tools in Copilot Sidecar Chat (Generally Available) A major upgrade to Copilot for SQL database in Fabric introduces new interactive toolsets in the Sidecar Chat for diagnosing performance, optimizing design & authoring SQL code. Try asking Copilot: ‘Which queries are consuming the most CPU in my database right now?’ ‘List tables without a primary key or clustered index.’ ‘Find missing index recommendations for my database.’ Refer to the documentation for more information about the Copilot features in SQL database in Fabric. Data Virtualization support for Fabric SQL database (Preview) Data virtualization enables you to leverage all the power of Transact-SQL (T-SQL) and seamlessly query external data from OneLake, eliminating the need for data duplication, or ETL processes, allowing for faster analysis and insights. Integrate external data, such as CSV, and Parquet, with your relational database while maintaining the original data format and avoiding unnecessary data movement. To learn more, refer to the Data virtualization with Azure SQL Database (Preview) documentation. Microsoft Python Driver for SQL database - mssql-python (Generally Available) This milestone is an important advancement in offering Python developers a modern, performant, and user-friendly experience when working with SQL Server, Azure SQL Database, or SQL databases within Microsoft Fabric. To learn more, refer to the Microsoft Python Driver for SQL Server - mssql-python (preview) documentation. Cosmos DB in Fabric (Generally Available) You can now analyze live Cosmos DB data directly in Fabric—no complex or costly ETL required. Data stays in sync with OneLake, providing a single source of truth for real-time and historical insights. As a distributed NoSQL database, Cosmos DB in Fabric brings support for semi or unstructured data to analytics and ML workloads and a host of new capabilities to Fabric, including vector indexing and search using DiskANN and reverse ETL capabilities allowing customers to serve analytics to users with incredible speed at massive scale. Whether you’re building dashboards, running analytics, or training ML models, you can work on fresh operational data for faster, AI-ready insights. This GA release delivers production-grade performance, enterprise security, and the scale of Cosmos DB’s low-latency architecture within the unified Fabric experience. To learn more, check out the Getting Started with Cosmos DB in Microsoft Fabric Demo, refer to the What is Cosmos DB in Microsoft Fabric (preview)? documentation and visit our samples gallery. Data Engineering Spark connector for SQL databases (Preview) The Spark connector for SQL databases is a high-performance library that makes the read/write of SQL databases in Fabric easier and seamless. This connector offers following capabilities: This is preinstalled in the Fabric runtime, so you don't need to install it separately. Use Spark to run large write and read operations on Azure SQL Database, Azure SQL Managed Instance, SQL Server on Azure VM, and Fabric SQL databases. When you use a table or a view, the connector supports security models set at the SQL engine level. These models include object-level security (OLS), row-level security (RLS), and column-level security (CLS). Support multiple authentication methods and multiple write modes when writing data to the database. To learn more, refer to the Spark connector for SQL databases documentation. ArcGIS GeoAnalytics for Microsoft Fabric Spark (Generally Available) Microsoft and Esri have partnered together to bring spatial analytics into Microsoft Fabric for production scenarios. Our collaboration with Esri introduces cutting-edge spatial analytics integrated within Microsoft Fabric Spark notebooks and Spark job definitions (across both Data Engineering and Data Science experiences). The following is an example of gaining insight into overall patterns of features and their associated values for different distributions or across different time periods for the Total Insured Value: This integrated product experience empowers Spark developers and data scientists to natively use Esri capabilities and run GeoAnalytics functions and tools within Fabric Spark for transformation, enrichment, and pattern / trend analysis of data across different use cases without any need for separate installation and configuration. Here is an example of Total Insured Value by probability of hurricane force winds for the given geographical area: ArcGIS GeoAnalytics offers a comprehensive suite of geospatial capabilities that cater to a wide range of applications. Esri is integrating components of the ArcGIS suite of products into Microsoft Fabric. Specifically, ArcGIS GeoAnalytics for Microsoft Fabric product brings a set of geospatial functions and tools functions and tools directly into the Fabric Spark environment to facilitate analysis of events, visualize relationships between places, and derive valuable insights from your data. To learn more, refer to the ArcGIS GeoAnalytics for Microsoft Fabric (Generally Available) documentation. Faster Notebook Loading with Progressive Rendering Opening notebooks with large table outputs can be frustrating when performance slows down. Progressive rendering changes to that experience by loading display() outputs incrementally instead of waiting for everything to finish before you can interact. This means you can start editing, running cells, and exploring your notebook right away while the remaining display() outputs continue to render in the background. For data-heavy notebooks or those with complex tables and visualizations, this improvement makes a big difference. Progressive rendering keeps the interface responsive, reduces wait times, and helps you stay productive without interruptions. It’s a simple yet powerful enhancement designed to make working with large notebooks smoother and faster. Note that it currently only works with Spark notebook. Check out our video blog to see progressive rendering in action and experience the difference! Optimal Refresh for Materialized Lake Views (Preview) This feature enhances refresh performance by determining the most effective refresh strategy (incremental, full, or no refresh) for your Materialized lake views. The 'optimal refresh' feature is enabled by default, simply enable the delta CDF property for the source so you can immediately benefit from this capability. For additional information, refer to the Optimal refresh for materialized lake views in a lakehouse documentation. Fabric Data Engineering VS Code improvement In response to recent feedback from the community, we are pleased to introduce a series of quality improvements to the Fabric Data Engineering VS Code extension. A common request is to be able to open several Fabric notebooks in a single VS Code window. With the VS Code workspace feature, users can open multiple Fabric Notebooks from different Fabric workspaces within a single VS Code instance. By selecting the ‘Add to Workspace’ option, the notebook will be open within the same VS code windows with all existing notebook. Another frequently requested improvement is the removal of the strict reliance on Conda. With the introduction of the ‘Microsoft Fabric Runtime’, users can now execute complete notebook code within a remote workspace, enforcing local desktop checks for Conda availability is unnecessary now. Consequently, this validation has been removed when the extension is activated. For ISVs and partners, it is common to work with multiple tenants or accounts within the same tenant. During the sign-in process, users now have the option to select a different Fabric account, rather than taking the account already signed in to the Fabric portal as default. All these changes are available in VS code marketplace now with the release of version 1.15.3 To learn more, refer to the Fabric Data Engineering VS Code experience documentation. New features in Fabric User Data Functions, Ignite edition The User Data Functions has been hard at work to bring you the functionality you need to tie together your Fabric architectures and create robust data solutions. This section will cover the latest features in Fabric User Data Functions for the Microsoft Ignite 2025 event. You can find all the updates in the Functions Ignite 2025 blog post. 1. Fabric Activator support (preview) You can now set up User Data Functions as actions from your Activator rules. To do this, create a new Rule for one of your event categories and select the new ‘Run Function’ action from the list. You can pass parameters from your events and set conditions to run your functions based on your events properties. You can leverage this feature to create efficient real-time event processing experiences where every event is processed by an individual function run. To learn more, refer to the Activator integration documentation. 2. Variable Library integration You can now connect to your Variable Libraries from User Data Functions. You can do this by using the Manage connections experience and creating a connection to your Variable Library items. You can leverage this integration to use different value sets inside your functions without making any code changes. This feature works especially well with Fabric CI/CD where you can leverage different value sets for each of the environments you are working with. Learn more by reading the Variable Library integration article in the User Data Functions documentation. 3. Azure Key Vault support Using a Key Vault is the best practice way to use secrets, such as API keys, passwords, and certificates. With this method, you can access any Azure Key Vault that your user account in Fabric has access to. Make sure to assign Reader permissions to your account for your secrets. This feature is helpful for writing functions to consume external APIs securely. To learn more, refer to the Azure Key Vault connection documentation. 4. Cosmos DB support You can now use a native-programming approach to connect to your Cosmos DB databases hosted on Fabric or Azure. Cosmos DB allows you to quickly set up the data tier of your architectures since you don’t need to create a schema. You can store JSON documents with any structure: properties, arrays, nested objects, etc. You can get started by retrieving your endpoint from your Fabric Cosmos DB item and using any of the samples included in the Portal editor. To learn more, refer to the Cosmos DB documentation. And that’s it! Make sure to visit the Functions Ignite 2025 blog post to read the full updates. To learn more about this function, refer to the Fabric User Data Functions documentation. Azure Artifact Feed in Fabric Environment (Preview) Fabric Environment now supports installing Python libraries directly from Azure Artifact Feed! This new capability makes it easier and more secure for teams to manage and deploy custom libraries on scale. To install the packages from your Azure Artifact Feed, you need to set up the connection in Azure admin portal and then specify it in an Environment. Set-up connection in Fabric - In Fabric, the connections need to be set-up through Connections component. When creating the new connection, choose 'cloud' type and 'Azure Artifact Feed (Preview)' connection type. Ensure you select the checkbox ‘Allow Code-First Artifacts to access this connection’. Record the connection ID after successfully creation, this is needed for specifying the connection in Environment. Installing the packages in Environment from your Azure Artifact Feed - Along with the availability of the Azure Artifact Feed supportability, Fabric Environment has introduced a brand new UX experience to better support the private repositories management. You can now copy, paste, and edit your YAML configuration directly in the Fabric UI, making it simple to manage libraries from both public and private sources. You can now use the new YAML editor in Fabric Environment to list your dependencies and reference your Azure Artifact Feed connection. Note that the Azure Artifact Feed URL needs to be replaced by the Connection ID to be correctly recognized by Environment. To learn more about this feature, refer to the library management in Fabric environments documentation. Data Science Connect Data Agents to your Azure Search Index in Microsoft Foundry Data agent creators can now connect their agents directly to Azure AI Search indexes built in Microsoft Foundry, unlocking powerful unstructured data scenarios. Using the resource URL, you can securely connect to your index—data agents fully respect the permissions of your Azure AI resources. In Microsoft Foundry, you can craft rich AI Search indexes with custom enrichments, preprocessing logic, and tailored schemas for PDFs, text files, and more. Once connected, Data Agents can reason over that unstructured content and even join insights from your AI Search index with your structured data sources, giving you a unified, intelligent view across all your data. _November_2025_Feature_Summary To learn more about how you can connect your AI Search index from Microsoft Foundry, refer to the Configure your data agent documentation. Updated Example Query and Instruction Data Agent Limits We’ve increased several key limits to give creators more room to guide and shape Data Agent behavior. Data Source Instructions for Eventhouse KQL databases now support up to 15,000 characters (previously 5,000), giving you far more space to describe schemas, business logic, and edge cases. Example Queries have expanded from 1,000 to 5,000 characters, allowing richer examples for complex question patterns. These changes give you greater flexibility and control when tuning how your Data Agent interprets and answers user prompts. To learn more about the data agent configurations, refer to the Data agent configurations documentation. Fabric AI Functions Enhancements (Generally Available) Apply powerful functions such as ai.extract(), ai.classify(), ai.generate_response(), and more, to turn your data into insights with just a single line of code. This release also introduces new parameters for greater flexibility and control, such as response_format in ai.generate_resoponse() to define your output structure, and instructions in ai.summarize() to provide additional context to the LLM. We’ve also added support for advanced configurations when using gpt-5, such as verbosity and reasoning_effort. Finally, we’ve increased default concurrency for faster execution and expanded Microsoft Foundry model integration to PySpark, allowing you to run AI Functions on models beyond OpenAI. These updates will be generally available across all geographies in the coming weeks. To learn more, refer to the Transform and enrich data with AI functions documentation. MCP Server support in Fabric data agents Support for managed MCP server endpoints is now available in Fabric data agents. This enables seamless interoperability across the AI ecosystem, allowing external systems and services to tap into the rich domain expertise of data agents and access curated knowledge stored in OneLake. This enhancement makes it easier for organizations to connect intelligent agents with enterprise-grade data—unlocking more powerful, context-aware insights from both structured and unstructured sources. To learn more about how to consume your Fabric data agent as MCP server in VS Code refer to Data agent MCP Server documentation. Fabric data agents now integrate with Microsoft 365 Copilot Data agents in Fabric now deliver enhanced integration with Microsoft 365, enabling seamless access to enterprise data within M365 Copilot. Through this integration, users can query and reason over curated data in OneLake directly from M365 applications, bridging productivity and analytics workflows. This capability extends M365 beyond document-centric interactions, allowing Copilot to leverage governed, enterprise-grade data models for more precise, context-aware responses and insights. To learn more about to consume your Fabric data agent in M365 Copilot refer to the What’s New for Fabric Data Agents at Ignite 2025 blog. Leveraging Prep your data for AI for semantic models in Fabric data agent Fabric data agent now fully supports Prep for AI customizations in Power BI semantic models. When you add a semantic model to the Fabric data agent, any customization you make in Prep for AI is automatically respected. This includes AI instructions, verified answers, and data schemas that you’ve defined in the semantic model. Using Prep for AI helps you guide the model to focus on the right tables, use preferred terminology, and rely on verified information. As a best practice, review and refine these settings before connecting your semantic model to the data agent. This ensures more accurate and context-aware responses when users query the data agent. Learn more about how to leverage Prep for AI when adding a semantic model to your Fabric data agent. Fabric Data Agents to Support SQL Databases Data Agent in Microsoft Fabric has expanded its capabilities to include direct support for SQL Database Artifacts, eliminating the need for intermediary lakehouses. This enhancement allows users to connect Fabric SQL databases in addition to other mirrored SQL databases directly to Data Agent. Once connected, Data Agent leverages its NL2SQL engine to translate natural language queries into SQL, enabling instant insights through the SQL Analytics Endpoint. This integration simplifies workflows and accelerates decision-making by making structured data in operational databases accessible through conversational AI. Fabric Data Agent Entry Points in Data Warehouse and Lakehouse Connecting your Lakehouses and Warehouses to the Data Agent is now dramatically simpler. A new entry point in the ribbon of both Lakehouse and Warehouse interfaces allows you to instantly create a Data Agent or add your data source to an existing one. This streamlined experience automates both Data Agent creation and Data Source addition, significantly reducing the time and effort required to get started with building and integrating data agents. Upgrade your machine learning tracking system We’ve made a foundational upgrade to the machine learning tracking system in Microsoft Fabric. This update prepares your workspace for upcoming enhancements, such as advanced tracking capabilities and streamlined cross-workspace model logging, while keeping your day-to-day experience unchanged. The upgrade is available as an option and can be initiated at your convenience. You can start upgrading from either an ML artifact or through your workspace settings. Detailed instructions are available in the Upgrade your machine learning tracking system documentation. Stay tuned for more improvements as we continue making your data science journey smoother and more rewarding! Supporting Internal Python Packages in ML Model Endpoints We’ve made an important improvement to machine learning model endpoints. Previously, real-time endpoints couldn’t use internal libraries—a limitation many users faced. Now, you can activate machine learning model endpoints built using AutoML and FLAML, making it easier to deploy and manage your models with these powerful tools. Data Warehouse IDENTITY columns (Preview) IDENTITY columns (Preview) in Fabric Data Warehouse, is a long-awaited feature that simplifies surrogate key generation during data ingestion. IDENTITY columns automatically produce unique values for each new row, eliminating the need for manual key assignments and eliminating the risk of key duplication and key integrity issues. Creating a table with an IDENTITY column, inserting a row, and querying its values This system-managed approach ensures uniqueness across the Fabric Warehouse distributed engine, even when separate data ingestion jobs start in parallel. For more information about IDENTITY columns in Fabric Data Warehouse. For more information, refer to the IDENTITY documentation. Data Clustering (Preview) Data Clustering (Preview) unlocks significant performance gains and reduced consumption use for queries. By organizing rows with similar values together during ingestion, Data Clustering enables aggressive file pruning, only scanning files with data that match query predicates. Comparing a query that uses a regular table with one that uses Data Clustering This optimization is powered by a sophisticated algorithm that preserves data locality across multiple dimensions, outperforming traditional techniques like lexicographical indexes. For more information about Data Clustering in Fabric Data Warehouse. For more information, refer to the Data Clustering documentation. Warehouse Snapshots in Microsoft Fabric Data Warehouse (Generally Available) Managing data consistency during ETL has always been a challenge for our customers. Dashboards break, KPIs fluctuate, and compliance audits become painful when reporting hits ‘half-loaded’ data. With Warehouse Snapshots, Microsoft Fabric solves this by giving you a stable, read-only view of your warehouse at a specific point in time. Think of this as a true time travel database, an industry-first capability that sets us apart. For more information on refer to the full blog post on Warehouse Snapshots in Microsoft Fabric (Generally Available). Varchar(max) support Fabric Data Warehouse and SQL analytics endpoints for mirrored artifacts now support large string and binary data using VARCHAR(MAX) and VARBINARY(MAX) types. Data Warehouse lets you ingest, store, process, and analyze large descriptive text, logs, JSON, or spatial data, with up to 16MB per cell, without hitting the size limits for most of the data that is common in the warehouse scenarios. The SQL endpoint for mirrored artifacts ensures large values from source systems are read without the previous 8KB truncation. For new tables, string and binary delta types are mapped to varchar(max) and varbinary(max) SQL types in SQL analytics endpoint. Existing tables with columns already storing large objects can be recreated to adopt the new data type or will be automatically upgraded to VARCHAR(MAX) on the next schema change. This is critical for preventing JSON corruption in mirrored Cosmos DB artifacts, where truncation could break queries due to malformed JSON. Stay tuned to the blog for updates on Varchar(max) support in Lakehouses. Real-time Intelligence Introducing the HTTP and MongoDB CDC Connectors for Eventstream Two new connectors for Eventstream— HTTP and MongoDB CDC — that make it easier than ever to bring diverse, real-time data into Fabric Real-time Intelligence (RTI) for real-time analytics. You can find both connectors in the Real-Time Hub starting today. HTTP Connector for Eventstream The HTTP connector provides a no-code, configurable way to stream data from any REST API directly into Eventstream for real-time processing. With just a few clicks, you can: Continuously pull data from SaaS platforms and public data feeds (e.g., CoinGecko, OpenWeather). Automatically parses JSON responses into clean and structured events. Get started quickly by selecting a predefined public API, entering your API key, and letting Eventstream prefill the required headers and parameters. The MongoDB CDC Connector for Eventstream The MongoDB CDC connector streams Change Data Capture (CDC) events from any MongoDB deployment— on-premises, cloud-hosted, or MongoDB Atlas —into Eventstream. It allows you to capture real-time database changes and stream them directly into Eventstream for immediate processing and analytics. For more information, refer to the documentation on Add MongoDB CDC source to an eventstream and Add HTTP source to an eventstream. Please note: We have begun rolling out this feature, it will be available in all regions by mid-December. Start exploring the new connectors today and happy streaming! Introducing Cribil Source in Real-Time Intelligence Eventstream (Preview) The exchange of real-time data across different data platforms is becoming increasingly popular. We are pleased to announce that the Cribil source is now available in Real-Time Intelligence, allowing real-time data to flow into Fabric RTI Eventstream through our partnership with Cribil. You can now add the Crbil source to Eventstream to create the Kafka endpoint. Then, use this Kafka endpoint information in Cribil to establish the connection. In the Cribil portal, select ‘Fabric Real-Time Intelligence’ as the destination and configure it with the Kafka details. By integrating Cribil, you can utilize Cribil data sources to access real-time data from various platforms such as Splunk, SQS, etc. and then bring the data to Fabric, thereby broadening the range of data sources available to Real-Time Intelligence. Partnering with third-party data platforms improves flexibility and interoperability, enabling organizations to easily unify streaming data within a single analytics environment. As a result, customers can take advantage of Fabric RTI’s capabilities for thorough analysis and insights, regardless of where their data comes from, and teams can quickly adapt to new business opportunities by integrating additional sources from third-party providers. To learn more, and guidance on getting started, refer to the Cribils Source documentation. Eventstream Activator destination (Generally Available) In today’s data-driven world, speed matters. Real-time signals are everywhere—customer clicks, IoT telemetry, operational metrics—but they only create value when they lead to action. That’s why we’re excited to share Eventstream Activator destination, now generally available (GA) in Microsoft Fabric Real-Time Intelligence. With Eventstream Activator destination, you can detect important patterns in your live data and trigger the right action automatically—no code required. Ingest and transform events in Eventstream, route them to Activator, and define simple rules for alerts, notifications, or workflows. It’s the fastest path from streaming signal to business outcome. How it Works Ingest & Transform in Eventstream - Connect diverse sources (telemetry data, apps, CDC, IoT, Fabric events etc.), then filter, enrich, or aggregate in Eventstream. Route to Activator - Add Activator as a destination in your Eventstream topology. Choose the transformed stream you want Activator to monitor. Detect & Act - In Activator, create rules for the patterns or thresholds you care about and configure actions (alerts, Teams notifications, workflows, and more). Please note: We have begun rolling out this feature, it will be available in all regions by mid-December. To learn more about setting up Activator destination in Eventstream, refer to the Add a Fabric Activator destination to an eventstream documentation. Capacity Overview events (Preview) This new capability provides administrators with real-time insights into the health and utilization of their Microsoft Fabric capacities. Capacity Overview Events include two event types: Capacity Summary – Delivers a point-in-time snapshot of capacity usage, based on a smoothed utilization metric. Capacity State – Captures changes in capacity state, such as transitions to Paused or Overloaded. With these events, organizations can proactively monitor capacity behavior, trigger automated workflows using Activator, or route events to Eventhouse or Lakehouse through Eventstreams for deeper analysis and long-term retention. To learn more, check out the blog post and follow the step-by-step tutorial to get started. You can also explore the Fabric Capacity Events Accelerator, which includes prebuilt dashboards, templates, and best practices. Entity Diagram in Eventhouse KQL Database (Preview) As your KQL database grows, tables gather data from several Eventstreams, functions connect different tables, update policies move and transform data, and materialized views quietly keep aggregated data up to date - all working together behind the scenes It’s powerful, but it can also be hard to see the full picture. That’s exactly why we built the Entity Diagram - to give you a simple, visual way to explore how everything in your database connects. No more guessing where data comes from or where it goes, no more wondering what depends on what - just a clear view that helps you understand, troubleshoot, and design with confidence. What is the Entity Diagram? The Entity Diagram gives you a visual map of your database. It shows the relationships between your entities: tables, functions, materialized views, update policies, shortcuts, and continuous exports. It also shows cross-database relationships and Eventstream items that serve as data sources for tables, so you can instantly understand how data flows through your system. You can view details, follow connections, and see what depends on what - all in one place. View Ingestion Details You can now see the number of records ingested for each table or materialized view. If the ingestion comes from an Eventstream, you can also see a node for the Eventstream item. If you click on the Eventstream, it will take you directly to it. You can track how data flows through update policies and how it is aggregated into materialized views, giving you a complete view of your data flow. Spot Schema Violations The Entity Diagram also flags schema violations between entities, such as broken references from functions to tables or columns, or update policies referencing functions or source tables that no longer exist. This helps you quickly identify and fix issues that might disrupt your data flow. What’s in it for you Whether you are a developer, data engineer, or analyst, the Entity Diagram helps you understand your KQL database clearly. You can explore how tables, functions, materialized views, update policies, and other entities are connected, track data flow including the number of records processed through tables and passed along update policies or materialized views, identify schema violations, and make confident changes with a complete understanding of your database. To learn more, check out the View an entity diagram in KQL database (preview) documentation. Automate advanced actions with Fabric Activator Fabric Activator enables you to automatically take actions or send alerts whenever certain data conditions are met. What’s new? You can now automate business logic by running User data functions (Preview) and Spark job definition as an action when your data changes. Access advanced actions for automation in Real-Time Hub (coming soon in Real-Time dashboard, Eventhouse, and KQL queryset), including: Pass parameter values to Functions, Spark job definitions, Pipeline, and Notebook. Create custom action (Power Automate workflow) that you can reuse across Activator rules. Send Teams message to group chats and channels. Customize email and Teams recipients and messages. How it works You can access this feature by creating an Activator item or access it in embedded experience like in Real-Time Hub (coming soon in Real-Time dashboard, Eventhouse, and KQL queryset). In Real-Time Hub for example, you can see ‘Set alert’ button when browsing data sources like Azure events, Fabric events, or Eventstream. Selecting ‘Set alert’ will open a side pane where you can set up conditions and actions, including notifications, Fabric activities, and custom action. By creating the rule, you can automate your business process leveraging Activator. Try it out and share your feedback! To try this feature now, head over to Fabric. We look forward to hearing from you, if you have any feedback or ideas, join the discussion in the Activator community. Operations agent (Preview) With operations agent, users can create autonomous agents that monitor data, infer goals, and recommend actions. These agents dynamically construct plans based on business objectives, data sources, and available actions, keeping a human in the loop while enabling automation when desired. To get started with operations agent, you give it access to specific Eventhouse sources, define business goals and instructions, and specify actions integrated through Power Automate. The agent then builds a plan to achieve those goals. It sets up monitoring rules, always grounded with data from the Eventhouse, and then watches for events that match those rules behind the scenes. When those conditions are met, the agent wakes up and starts to reason over the data. It looks at the actions it’s been configured with and makes recommendations back to the user based on what it deems is most appropriate at the time, along with context about what caused the alert to fire. These are presented through Teams to notify the users and keep a human-in-the-loop. You can try the operations agent out now! You need to enable both the Copilot/AI and operations agent tenant level settings, and have a workspace backed by a Fabric capacity (not Trial). More information is available in our documentation. Learn about our other Real-Time Intelligence announcements. Imagery file support in Maps (Preview) Maps now support imagery files such as Cloud Optimized GeoTIFF (COG) and Raster PMTiles, in addition to vector spatial formats, enabling richer geospatial analysis with imagery references. Upload your imagery file to Lakehouse and simply right-click ‘Show on map’ to display it in the map view, offering the same seamless experience as existing static spatial formats like GeoJSON. vember_2025_Feature_Summary (Source: European Space Agency, Copernicus Services. (2025). Sentinel-2 Level 2A. Retrieved from Sentinel-2 Level-2A | Planetary Computer) (Source: National Oceanic and Atmospheric Administration. (2024). NOAA Chart Display Service- ncds-20c. Retrieved from NCDS MBTiles Download) To learn more, refer to the Create a map (preview) documentation. Data labeling in Maps (Preview) Data labeling settings are now available for all geometry types—points, lines, and polygons—reflecting your feedback. These new customization options make it easier than ever to surface critical attributes directly on the map, so you can highlight key details with just a few clicks and keep your spatial insights front and center. (Source: Department of Education. (2024). Public School Locations 2021-22. Retrieved from Public School Locations 2021-22 - Catalog) (Source: Department of Agriculture, U.S. Forest Service. (2017). National Forest System Trails (Feature Layer). Retrieved from National Forest System Trails (Feature Layer) - Catalog) To learn more, refer to the Customize a map (preview) documentation. Copilot assisted real time data exploration (Preview) Copilot-assisted real-time data exploration enables users to analyze live data using natural language. This new feature integrates Copilot into Real-Time Dashboards and Real-Time Hub, allowing you to explore the data behind dashboard tiles and tables simply by asking questions. You can instantly filter, break down the data, compare timeframes, and uncover insights without writing any query language. With advanced no-code tools, you can also manually adjust and fine-tune the visuals generated by Copilot. Once you are done exploring the data and are pleased with the visual representing the derived insight, you can save it as a new tile on dashboards. To begin, simply open a Real-Time Dashboard in Fabric and use the Copilot pane or the inline Copilot prompt on any tile to start chatting with your data. To learn more, refer to Explore real-time dashboard data using Copilot documentation. Data Factory Enterprise readiness Snowflake Connector Supports to Use Key Pair Authentication Method Key pair authentication is now available for connecting with the Snowflake connector. When you create a new Snowflake connection or edit an existing one in ‘Manage connections and gateways’, you’ll find the ‘KeyPair’ authentication option. After setting up the Snowflake connection with Key Pair authentication, you can easily use this connection in Pipeline, Copy job, Dataflow gen2, and Mirroring. To learn more, refer to the Snowflake connector documentation. Manual Update for On-premises Data Gateway (Preview) This new capability simplifies gateway maintenance and helps ensure your environment remains secure and up to date. With this preview, you can now initiate gateway updates manually—either directly from the gateway UI or programmatically through API or script—giving you full control over when and how updates are applied. The November release serves as the baseline version for this feature, and customers can start performing manual updates beginning in December. This enhancement also paves the way for future support of fully automatic updates. To learn more, refer to the Update an on-premises data gateway documentation. Certificate & Proxy Support for Virtual Network Data Gateway (Preview) Virtual Network (VNET) Data Gateway, available in preview, enables secure and flexible enterprise connectivity by allowing certificate-based authentication for compliant gateway communication and proxy configurations for environments where direct internet access is restricted—empowering organizations to confidently use the VNET Data Gateway in highly controlled, security-focused network infrastructures. To learn more about this feature, refer to the Manage virtual network (VNet) data gateways documentation. Pipelines Error Insights summary Copilot for Pipeline The new Error Insights Summary Copilot in Pipeline is designed to make error handling smarter and faster. When dealing with pipelines that fail with dozens or even hundreds of errors, it can be overwhelming to investigate each issue manually. With this Copilot capability, you now get a concise, intelligent summary of all activity errors, complete with categorized insights, root cause analysis, and actionable recommendations. Whether you're in the Pipeline Monitoring or Authoring page, simply click on the ‘Error Insights’ button for any failed run to activate Copilot. Instead of clicking through each error individually, you’ll instantly see an insight into the categorized errors to help you understand what went wrong and how to fix it. For instance, when a pipeline failed with more than 102 errors, instead of reviewing each error individually, Copilot grouped them into three categories. Each category included an issues summary, root cause analysis, and recommended actions, making the process more efficient and saving time. This feature greatly improves the intuitiveness and productivity of pipeline troubleshooting. To learn more, refer to the Get started with Copilot for Pipelines documentation. Natural Language to Generate and Explain Pipeline Expressions with Copilot (Preview) Building Pipeline expressions can be complex -- but it doesn’t have to be. We are introducing a new Copilot capability (currently in preview) that transforms how you create and understand Pipeline expressions in Fabric Data Factory! What’s new with the Pipeline Expression Builder Copilot? You’ll find this Copilot built inline in the Pipeline Expression Builder, where you chat with Copilot, just like you would with our other Data Factory Copilot offerings. You can now generate expressions using natural language. Simply describe what you need in your expression – Copilot will translate your intent into accurate pipeline expressions for you. You can also use Copilot to explain existing expressions in plain language. No more decoding syntax. Copilot provides clear, contextual explanations, so that you understand what your Pipeline expression is doing. Pipeline expressions are powerful but often intimidating. This feature helps boost your productivity by reducing manual coding, minimizing errors, and empowering everyone to build robust pipelines confidently. To learn more, refer to the Copilot Pipeline Expression documentation. Hierarchical view for pipelines in Monitoring Hub Managing your complex orchestration workflows just got easier with Hierarchical view for Pipelines in Monitoring Hub! Jobs are often triggered automatically, and pipelines are one of the most common examples of this. With Hierarchical view, you can now: Navigate across layers of jobs: Seamlessly explore upstream and downstream jobs within a Pipeline. Trace dependencies: Quickly locate related jobs and understand how they connect, giving you full visibility into your workflow. To utilize Hierarchical view in Pipelines, navigate to the Column options in the Monitoring Hub page. Toggle on ‘Upstream run’ and’ Downstream runs’. You should now be able to see hierarchical views of your Pipeline runs. This feature empowers you to monitor and troubleshoot with confidence, ensuring smooth operations across all your automated processes. To learn more, check out our documentation on How to monitor pipeline runs in Monitoring hub. Connectivity Spark & Impala 2.0 Connectors (Generally Available) This release brings enhanced performance and security to Fabric workloads, particularly when working with large datasets in Dataflow Gen 2. Why does this matter for Dataflow Gen 2? Implementation 2.0 for the Spark and Impala Connectors is now generally available in Microsoft Fabric , built on the open-source Arrow Database Connectivity (ADBC) driver . This release brings enhanced performance and security to Fabric workloads, particularly when working with large datasets in Dataflow Gen 2 . Faster data access: The ADBC-based implementation eliminates serialization and copying, reducing overhead. Secure by design: Memory safety and garbage collection align with modern secure development lifecycle (SDL) standards. Optimized for scale: Perfect for complex pipelines and large-scale transformations in Dataflow Gen 2. Beginning November 2025, specify [Implementation="2.0"] in your connection settings to utilize these improvements: For Spark: ApacheSpark.Tables("http://server.cloudapp.azure.com:10001/cliservice", 2, [BatchSize=null, HierarchicalNavigation=true, Implementation="2.0"]) For Impala: Impala.Database("server.cloudapp.azure.com", [Implementation="2.0"]) For further details, please refer to the Spark Connector and Impala Connector documentation. Dataflows Modern Get Data in Excel Desktop Modern Get Data experience in Excel Desktop is designed to make finding and connecting to your data sources easier than ever. Instead of navigating multiple menus or guessing where to start, you can now access all available data sources in one central place. Simply go to the Data tab in Excel and click Get Data to explore a wide range of connectors. Whether you prefer scrolling through the list or searching directly, finding the right source is now fast and intuitive. This new experience also includes powerful search capabilities. You just need to type in keywords to quickly locate the data source you need. For example, connecting to your Fabric Lakehouse is now just a few clicks away, and you can immediately load your data into the Power Query editor for transformation. With this streamlined workflow, Excel has become even stronger for your analytics, helping you save time and focus on insights rather than set-up. To learn more about Get Data experience, refer to the documentation. New importing data flow supported in Modern Get Data Copilot Copilot in Modern Get Data within Dataflow Gen2 allows you to ingest data from both the OneLake catalog and OneDrive for further analysis. These features make it easier to bring in data for your next transformation using natural language. When browsing data from the OneLake catalog, you can select your OneLake data from different workspaces by either choosing the data directly in OneLake or searching for your specific OneLake artifact. Once you choose data from OneDrive or OneLake, you can quickly load it into the MGD Copilot for further transformation. To learn more, refer to the Copilot in Modern Get Data documentation. Fabric AI Functions integration with Dataflow Gen2 (Preview) These functions bring generative AI capabilities directly into Microsoft Fabric, making it possible to perform advanced AI tasks without needing machine learning expertise. With an intuitive user experience, users can invoke large language models through Fabric’s built-in AI endpoint. The experience is simple and integrated—you can add AI-powered columns using the AI Prompt option in the Add Column tab, and prompts can automatically include the full row as context. Once you’re inside the AI Prompt dialog you can provide the prompt of your choice and select what columns from your table you wish to pass as added context for the prompt. This feature is designed to make AI accessible across Fabric experiences, including Dataflow Gen2 and notebooks. The AI Prompt feature will begin rolling out globally the first week of December. To learn more, refer to the Fabric AI Functions in Dataflow Gen2 documentation. Power Query Language Service IntelliSense in Dataflow Gen2 (Preview) IntelliSense powered by the Power Query Language Service is now integrated with Power Query Online and available inside Dataflow Gen2 as a preview feature. What’s New? IntelliSense Support: Enjoy syntax highlighting, auto-completion, and inline suggestions when writing M scripts in Power Query Online. Improved Authoring Experience: Make editing and creating queries faster and less error-prone with real-time guidance. Consistent Experience: Aligns with the familiar capabilities from Power Query Desktop, now in the cloud. Why It Matters This feature helps you: Reduce errors when writing M code. Speed up development with smart suggestions. Work confidently in Dataflow Gen2 without switching tools. Use Dataflows today and try out this feature! Airflow job New API for Apache Airflow Jobs for Files & Requirements Job management APIs are now available for Fabric Apache Airflow jobs! These APIs let you directly manage files and requirements within your Airflow projects—upload, update, and organize resources with ease. By streamlining these operations, you reduce manual steps and simplify automation. Built for flexibility, these APIs help teams automate environment setup and maintain consistent deployments, making it easier to keep Airflow jobs organized and up to date. For more information, refer to the API capabilities for Fabric Data Factory's Apache Airflow Job documentation. Now you can upload files to your Apache Airflow project from the UI Managing your Apache Airflow workflows is now more convenient with the new UI-based file upload feature. This enhancement allows you to add files directly to your Airflow project through an intuitive interface, streamlining the process of updating configuration files, requirements, or other essential resources. By eliminating the need for manual uploads via command line or external tools, this feature helps ensure your projects stay organized and up to date with minimal effort. Whether you're onboarding new data or updating dependencies, the UI makes it easy to keep your Airflow environment running smoothly. New Apache Airflow Job File Management APIs The Apache Airflow Job File Management APIs have been released, representing significant progress toward enhancing the efficiency, security, and developer experience of workflow orchestration. These APIs are designed to give you full control over job files in your Apache Airflow environments, enabling seamless automation and integration across your data workflows. The File Management APIs allow you to: Upload and manage DAG files: Easily add new DAGs or update existing ones. List and retrieve files: Get a complete view of your job files for auditing. Secure file operations: Built-in support for role-based access ensures enterprise-grade security. To learn more about how to use these APIs, check out our documentation on API capabilities for Fabric Data Factory's Apache Airflow Job. Mirroring Mirroring for Snowflake Iceberg Tables Support Microsoft Fabric now supports Iceberg tables for Mirroring with Snowflake. Users can seamlessly bring both managed and Iceberg tables from their Snowflake environment into Fabric, enabling unified analytics and data management. During setup, Fabric automatically detects and distinguishes between managed and Iceberg tables, giving users the flexibility to mirror future tables as they’re created. For Iceberg tables, which reside in customer-owned storage, Fabric allows you to select your preferred storage provider, ensuring secure and direct connectivity to your data. Once configured, managed tables replicate (with row counts visible as they sync) while Iceberg tables are surfaced via shortcuts in the same mirrored DB. Analysts can preview, query via the SQL endpoint, and join Iceberg and managed tables together just like any other tables in Fabric—no special steps or rewrites. This update streamlines cross‑platform analytics and accelerates time‑to‑insight for Snowflake customers adopting Fabric. For more information about Iceberg Support in Mirroring for Snowflake, please refer to Snowflake Mirroring Iceberg Support documentation. Mirroring for SAP (Preview) With Mirroring for SAP via SAP Datasphere, Microsoft Fabric provides seamless integration with SAP utilizing SAP Datasphere’s data extraction capabilities to mirror SAP data directly into Fabric for unified analytics and enhanced business insights. The integration delivers near real-time access to data across the entire SAP application landscape, including SAP S/4HANA (both on-premises and cloud editions), SAP BW and BW/4HANA, as well as cloud solutions such as SAP SuccessFactors and SAP Ariba. As a result, organizations benefit from up-to-date, reliable data for reporting and advanced analytics, seamlessly combine SAP data with other enterprise sources in Fabric, expedite decision-making processes, and fully leverage Fabric’s comprehensive analytical capabilities in conjunction with their SAP investments. To learn more about mirroring for SAP, refer to the Mirrored database from SAP documentation. Mirroring for SQL Server (Generally Available) This milestone marks a significant step forward in our mission to provide seamless, near-real-time data replication capabilities, empowering you to derive maximum value from your SQL data with Microsoft’s unified data platform. Mirroring for SQL Server 2016-2022 and the newest version SQL Server 2025 offers continuous data replication from these sources into OneLake, ensuring that your data remains current and readily accessible for advanced analytics and reporting needs without complex ETL processes. To learn more and to get started, refer to Microsoft Fabric Mirrored Databases from SQL Server documentation. Mirroring for Cosmos DB (Generally Available) Azure Cosmos DB Mirroring in Fabric allows you to seamlessly integrate your existing operational workloads with the analytical capabilities of Microsoft Fabric. Take advantage of the innovations in Fabric with SQL queries over JSON data and build real-time intelligence over your existing transactional data. Integrate with the Microsoft Fabric ecosystem of services, as well as Copilot-powered Microsoft Power BI, — all without having to stitch together separate services. Azure Cosmos DB Mirroring in Fabric allows you to select which containers to mirror into Fabric, giving you total workload isolation and complete control over what data you build analytics on. Get the best of both worlds leveraging the SLA-backed latency and availability you have come to expect from Azure Cosmos DB with Microsoft Fabric’s array of analytical services and features, making it even easier now to bring your existing operational data and make accessible across the Fabric ecosystem. To learn more about Mirroring Azure Cosmos DB (Preview), refer to the documentation. Mirroring for PostgreSQL (Generally Available) This release features enhancements such as support for PostgreSQL flexible servers hosted behind VNETs or Private Endpoints, as well as EntraID authentication for source database connections. Additionally, support for high-availability enabled servers has been introduced, ensuring business continuity for mirroring sessions through seamless failover—an advancement particularly valuable for enterprise-level scenarios. With Azure Database for PostgreSQL Mirroring in Fabric, you can run all Fabric analytical workloads and capabilities on near-real time replicated data from your transactional sources without impacting on the performance of your production databases. This enables teams to unlock deeper insights and drive faster decision-making using the freshest data, all while maintaining the stability and responsiveness of operational workloads. To learn more, please refer to the Mirroring Azure Database for PostgreSQL flexible server documentation. Mirroring for Azure SQL Database – support for UAMI (Preview) Fabric Mirroring of Azure SQL Database with UAMI (User Assigned Managed Identity) is now in preview mode. To use UAMI, customers need to provide additional parameters to Fabric and ensure the primary identity on SQL has permissions to publish to the mirrored database artifact in Fabric on primary identity changes. To learn more, and guidance getting started, refer to the documentation. To learn more, refer to the Tutorial on Azure SQL Database. Copy job Expanded CDC Support for More Sources & Destinations Copy job now supports CDC (Change Data Capture) for even more sources, including SAP via Datasphere, Snowflake, and Google BigQuery. With this enhancement, you can automatically capture inserts, updates, and deletions from these sources and replicate them to supported destinations—no watermark columns required, no manual refreshes, and no extra effort. This makes your data ingestion faster, more efficient, and reliable. In addition, Copy job now supports merging CDC data into more destinations, including Fabric Lakehouse. You can seamlessly merge inserts, updates, and deletions from supported sources into Fabric Lakehouse tables, ensuring your data is always up to date. What’s more, the monitoring experience of Copy job has been enhanced: you can now access more detailed statistics for each run, including watermark values, load type, and row counts for inserts, updates, and deletions, giving you full visibility over your Copy job. To learn more, refer to the Change data capture (CDC) in Copy Job (Preview) documentation. Full & Incremental Copy Subsets of data with Database Queries You can now copy subsets of data from your tables using database queries, unlocking a wide range of data ingestion scenarios. For example: Copy only data for a specific region from a table with a region column to ensure compliance in data ingestion. Copy only the top N rows for testing or sampling. More importantly, this feature supports both full and incremental copies on table subsets based on your custom queries, allowing flexible data selection and filtering before loading. This makes your data ingestion more efficient, precise, and tailored to your needs. Please start by trying it with Azure SQL DB — support for more connectors that will be added soon. To learn more, refer to What is Copy job in Data Factory. Truncate Destination before Full Copy You can now optionally truncate destination data before the full load, ensuring your source and destination are fully synchronized without duplicates. By default, Copy job does not delete any data in your destination. When you enable this option: The first run of incremental copy will truncate all data in the destination before loading the full dataset. Subsequent incremental copies will continue to append or merge data without affecting existing records. If you later reset incremental copy to full copy, enabling this option will again clear the destination before loading. This approach not only ensures that your destination remains clean, fully synchronized, and free of duplicates, but also delivers significant performance improvements during full loads, providing a reliable foundation for your data ingestion solution. To learn more, refer to the What is Copy job in Data Factory documentation. Copy Multiple Folders in one Copy job You can now copy multiple file objects, including multiple folders or a combination of folders and individual files, in a single Copy job. This makes your development work more efficient, eliminating the need to create multiple Copy jobs to achieve the same result. To learn more, refer to the What is Copy job in Data Factory documentation. Connection Parameterization with Variable library for CI/CD (Generally Available) From Simplifying Data Ingestion with Copy Job – Connection Parameterization, Expanded CDC and Connectors, we introduced the ability to deploy the same Copy job across multiple environments using the Variable library to inject the correct connection for each stage, enabling automated CI/CD processes by externalizing connection values. This feature is now fully available in Copy job, allowing you to use it confidently in any production setting. You can easily connect to different data stores for development, testing, and production without having to change your Copy job each time. To learn more, refer to the CI/CD for Copy Job in Data CI/CD for Copy job in Data Factory in Microsoft Fabric Factory documentation. Developer tooling Fabric VS Code extension is now open source The Microsoft Fabric extension for Visual Studio Code has been officially released as open source, demonstrating Microsoft's commitment to community collaboration and developer empowerment. You can find the source code for the extension on VS Code-fabric repository. This extension offers essential functionalities for managing Microsoft Fabric workloads directly from Visual Studio Code. It handles authentication and tenant management, workspace operations, CRUD actions on Fabric items, Git integration, and exposes APIs for satellite extensions to add specialized functions. Figure: Microsoft Fabric extension for VS Code in extension marketplace As an open-source project, contributions are welcomed on GitHub through submitting bug reports, requesting features, and making pull requests, following Microsoft's open-source code of conduct. Closing Thank you for exploring these updates with us. To see the features in action, check out the November Monthly update video, that's packed with demos!Unlocking Geospatial Intelligence in Microsoft Fabric with Esri’s ArcGIS Maps Workload (Preview)
Introducing ArcGIS Maps for Fabric ArcGIS Maps for Microsoft Fabric is now available, bringing Esri’s industry-leading geospatial analytics along with dynamic, interactive mapping and visualization capabilities directly into Fabric. The new ArcGIS Maps workload empowers data professionals to visualize, explore, analyze, share location-based insights and integrate Esri’s authoritative data assets with their ArcGIS online service all seamlessly integrated with your organization’s data in OneLake. What are ArcGIS Maps for Fabric? ArcGIS Maps is a flexible, interactive mapping tool designed for visual data exploration and insight generation. It is an amalgamation of art and science. This is a tool that not only gives you the ability to work with data but also offers a rich graphical interface that allows users to tell a story. Unlike rigid workflows, ArcGIS Maps lets users adapt their analysis as new questions arise, making it easy to uncover patterns, trends, and outliers in their data. End-to-End Integration in Microsoft Fabric OneLake Integration: With the data in OneLake and access to the OneLake data Catalog users can select the most relevant dataset from across their organization’s data assets to use for their analysis. Data Engineering Workflows: Can perform advanced geospatial analysis in Fabric notebooks leveraging of the scale of Spark’s parallel processing to prepare massive volumes of data, that get written back to OneLake and available for visualization and reporting using ArcGIS Maps solution. Power BI Integration: Users can embed interactive maps in Power BI reports and organizational apps, making geospatial insights accessible to everyone. Getting Started Launch ArcGIS for Maps from the Fabric workload hub. With a single click you can create your first web map item which behaves just like any Fabric artifact. Key Features Effortless Data Onboarding Load data into your web map item from OneLake or use Fabric’s built in data-connectivity experience to bring data in from hundreds of different sources. ArcGIS for Maps automatically detects geospatial attributes in your dataset or prompts for coordinates, ensuring your data is ready for mapping. Rich Visualization & Smart Mapping Instantly plot geospatial data and visualize distributions—add cartographic polish, like visual filters that creates a vignette that offers qualitative context to your quantitative analysis. Use smart mapping features like heat maps, color ramps, clustering, and binning to highlight differences, trends, and hotspots. Switch between bubbles, pins, and heat maps with intuitive UI controls to make your data pop. Advanced Analysis & Attribute Table Ability to inspect data columns, view summary statistics, and rename fields for clarity before visualizing on a map. Aggregate large volumes of data (e.g., Customer Sales across a geographic region), by using clustering that dynamically redraws itself as you add and remove data points. It creates and adjusts a histogram of the Sales data by region and as you adjust the cluster size, offering full interactively to explore details by clicking on map features. Integration with ArcGIS Living Atlas & External Data Enhance your maps by adding authoritative data and imagery from ArcGIS Living Atlas of the World or your own ArcGIS Online content to enrich your webmap. Access population density, world terrestrial ecosystems, and many more third-party datasets to enrich your analysis. AI-Powered Arcade Assistant This is a scripting engine that augments the Smart mapping features to provide a natural language prompt interface to conduct additional analyses directly on the active web map data powered by a fine-tuned AI model. Create HTML tables, summaries, and advanced calculations without deep scripting knowledge, thanks to the Arcade scripting language and AI assistant. Scalable Performance & Security ArcGIS for Maps can handle hundreds of thousands of data points in browsers; from very large datasets, pre-processed with GeoAnalytics libraries in Spark Notebooks for optimal performance. Share maps securely with users across your organization, securely with granular access control and consent management. Ready to unlock the power of Geospatial Intelligence in Fabric? Try ArcGIS Maps for Microsoft Fabric today, Microsoft Fabric Workload Hub Access the documentation for a step-by-step guide on using the product.
