Fabric August 2026 Feature Summary
Welcome to the August 2026 Fabric update! Microsoft Fabric continues to evolve with new capabilities that help organizations build, manage, and scale their data and AI solutions more efficiently. This month's updates introduce enhancements across Fabric Platform, OneLake, Data Engineering, Conversational Analytics, Data Warehouse, Real-Time Intelligence, and Data Factory. Whether you're strengthening governance, improving performance, streamlining development workflows, or expanding AI-powered experiences, these updates are designed to help you get more value from your data while simplifying day-to-day operations. Explore the following highlights to see what's new in Microsoft Fabric this month. Events and Announcements Join us for FABCON and SQLCON in Barcelona, September 28 – October 1, 2026 Explore what’s possible with Microsoft Fabric and get up to speed on the latest in SQL, analytics, and AI. From 130 sessions and 4 keynotes to workshops, the expo, community spaces, and the Power BI DataViz World Championships, this is where the data community comes together. Learn directly from Microsoft and community experts shaping the future of Fabric and SQL. Register now and save €200 with code FABCMTY200. Fabric Platform KQL-Dashboard Embed in Fabric (Preview) A new capability that lets you add interactive KQL-Dashboard content directly into your own browser-based web applications is available. Now, you can bring Fabric analytics into the apps, portals, and workflows your users already use. Analytics are most useful when they are available where decisions happen. With Fabric Embed, you can place Fabric content inside a custom application or internal portal instead of requiring users to switch to the Fabric portal. Fabric Embed can help you: Bring interactive analytics into an existing business workflow. Explore Fabric analytics without switching between your application and the Fabric portal. Keep Fabric workspace permissions and Microsoft Entra ID identities at the center of access control. Build user-based embedded experiences for people who already have access to the underlying Fabric item. The embedded experience complements the Fabric portal. Content owners can continue to create and manage analytics in Fabric while application developers present that content in the context most useful to their users. To learn more, refer to the Microsoft Fabric Embed documentation. Git Integration – Workspace Relation API (Preview) Following our recent announcement of branch workspaces and the relationship that’s automatically created when a user performs a branch-out operation, we’re introducing the new Workspace Relations API. These endpoints let you create, query, update, and delete relationships between Fabric workspaces programmatically — and for anyone building automation around Git integration, branched workspaces, and CI/CD, it’s a big deal! The workflow is straightforward. A developer creates a feature branch. An ADO pipeline or GitHub Action then provisions a new feature workspace, configures Git, applies the right settings and permissions, and synchronizes the workspace. Finally, the Workspace Relations API links that feature workspace back to its parent — closing the loop and giving your automation a first-class, queryable connection between parent and branched workspaces. Figure: Branch workspace relation. To learn more, refer to Development process using Branch-Out experience. Git Integration and Deployment Pipeline – Item level permission restriction Starting December 1, 2026, users without read-write permissions on workspace items can't use Git integration and won’t be able to deploy a workspace or assign workspace to a stage via Deployment Pipeline. This restriction can result in loss of access to certain items because of sensitivity labels and protection policies applied to those items. To learn more, refer to Information Protection in Microsoft Fabric. OneLake Resource instance rules for OneLake (Generally Available) Resource instance rules for OneLake are ready for production workloads across enterprise analytics environments. They give workspace admins a precise way to allow access from trusted Azure resource instances while continuing to enforce network and data-level protections. Use resource instance rules when you need to enable secure service-to-service access to OneLake without relying solely on IP allowlists or requiring private networking for every integration. Combined with Private Link, IP firewall rules, and identity-based permissions, they help organizations apply layered security based on the needs of each workspace and workload. Resource instance rules support a broad set of Azure services that can present a verifiable Azure resource identity, including Azure Databricks, Azure SQL Server, Azure Data Factory, Azure Event Grid, Azure Machine Learning, and more. Configuration requires only the Azure resource ID, simplifying setup while maintaining control over which Azure resources can access OneLake. To learn more, refer to Manage inbound access to OneLake with Resource Instance Rules. Data Engineering Native Execution Engine performance improvements This month, we continued to improve the Native Execution Engine (NEE) with a new set of query execution optimizations designed to accelerate Spark workloads while reducing compute consumption. Key enhancements include broadcast joining reuse across queries, native acceleration for ranking window functions such as RANK and DENSE_RANK, and automatic materialization of reused Common Table Expressions (CTEs). Together, these optimizations eliminate redundant computation, keep more processing within the engine's vectorized execution path, and improve performance for common data engineering patterns including large joins, analytical reporting, ranking workloads, and complex transformation pipelines. Because these improvements are enabled automatically when NEE is available, customers benefit from faster execution times, lower Fabric capacity consumption, and improved price-performance without requiring code changes to existing notebooks, Spark Job Definitions, or pipelines. These optimizations are enabled by default once the Native Execution Engine is turned on, allowing customers to realize performance gains immediately without additional configuration or tuning. Customers can enable NEE at the workspace environment level by navigating to Environment > Acceleration and turning on Native Execution Engine, ensuring it is available for all Spark sessions using that environment. It can also be enabled at the session level by setting spark.native.enabled=true in Spark configuration. With NEE enabled, customers can seamlessly take advantage of the latest runtime innovations to process data faster, improve resource efficiency, and maximize the value of their Fabric capacity investments. To learn more about Native Execution Engine explore our documentation Native execution engine for Fabric Data Engineering. Fabric Runtime 2.0 (Generally Available) As the execution foundation for Microsoft Fabric's Data Engineering and Data Science experiences, Runtime 2.0 delivers a modern, high-performance platform built on Apache Spark and deeply integrated across the Fabric ecosystem. Purpose-built for large-scale data processing and analytics workloads, Runtime 2.0 represents a major advancement in performance, reliability, security, and future readiness. Built on the latest open-source innovations, it enables customers to accelerate data processing, simplify operations, and take advantage of the newest capabilities across Microsoft Fabric. This release includes significant platform upgrades, including Apache Spark 4.1, Delta Lake 4.2, Python 3.13, Java 21, Scala 2.13, and Azure Linux 3.0, providing a modern and enterprise-ready foundation for the next generation of data engineering, data science, and analytics workloads. These enhancements enable customers to take advantage of the latest open-source innovations while continuing to benefit from a fully managed, enterprise-grade experience in Microsoft Fabric. Whether you're building data pipelines, developing AI and machine learning solutions, processing streaming workloads, or powering enterprise analytics, Runtime 2.0 provides a more capable, scalable, and performant platform for your workloads. With improved performance, updated open-source foundations, and continued investment in capabilities such as the Native Execution Engine, Runtime 2.0 provides a modern platform for data engineering, data science, and analytics workloads in Microsoft Fabric. Explore the full documentation and start using Runtime 2.0 in production Runtime 2.0 in Fabric. Enhanced Spark Properties Support in Notebook and Spark Job Definition Notebook and Spark Job Definition (SJD) activities now enable users to specify Spark properties directly within the data integration pipeline. This enhancement allows Spark properties to be set inside the activity panel, ensuring that the values provided are used for activity execution. If an Environment item is linked to the Notebook or SJD and both the activity panel and Environment item define the same property, the value specified in the activity panel will take precedence and overwrite the Environment value. In the case of Notebooks, if the %%configure command is used within the notebook code to set a Spark property, the value set using %%configure will be applied for execution. This update offers users maximum flexibility, allowing Spark properties to be defined at different layers based on their specific use cases. By supporting property configuration in the activity panel, Environment item, and notebook code, users can tailor property values to meet the unique requirements of each execution of Notebook/SJD. To learn more on Transform data by running notebook and Transform data by running a Spark Job Definition activity. Conversational Analytics Enhanced Data Agent Visualizations with Fabric Visuals The data agent now uses Fabric visuals to render the charts it returns, bringing higher-quality, more consistent visualizations into your conversations with your data. When you ask a question like "Generate a bar chart of revenue by region" or "Show me my top 10 customers by sales," the data agent responds with an interactive, polished visual alongside its text and table answers, so you can spot trends, comparisons, and outliers. Because the data agent now shares the same visual foundation as Fabric Apps, charts look and behave consistently with AI-generated visuals in Fabric, with refinements to formatting, legends, tooltips, and axis scaling. Supported chart types include line, bar, stacked bar, pie, scatter, and area charts. To learn more, refer to the Get visual responses from a Fabric data agent documentation. Advanced DAX Generation for Semantic Models in Data Agents Advanced DAX generation for Power BI semantic models is now available in Fabric data agents when you use the Preview runtime. Instead of generating a DAX query in a single pass, the new system works iteratively as a specialized sub-agent that can use tools, inspect results, and refine its approach across multiple steps, providing significant improvements in response accuracy. It also uses instance value indexing to resolve values from the semantic model before generating a query, resulting in more accurate and reliable filters. This update is built on the same semantic-model query engine used across Fabric Skills, Power BI, and M365 Copilot, providing more consistent answers across Microsoft experiences. To use the new experience, open the Runtime dropdown in the data agent ribbon and switch from Standard to Preview. More improvements for semantic models in data agents are coming soon, including data source description and instructions, granular schema selection, and example queries. In the Preview runtime, the data agent uses advanced DAX generation to answer a question over a connected semantic model, enabling more accurate DAX generation and responses. To learn more, refer to the Semantic model best practices for data agent documentation. Data Agent orchestrator upgraded to GPT 5.1 The data agent orchestrator now runs on GPT-5.1, across both the standard and preview runtimes. The orchestrator handles how questions are rephrased, how work is planned across your data sources, and how the final answer is composed — so this upgrade changes behavior in all three. Most of what you see should be an improvement in answer quality and planning, but the change is not behavior-neutral: prompts tuned against the previous model may produce different results. We recommend re-running your evaluations, reviewing the results against your saved baselines, and updating your agent instructions and prompts where the new behavior doesn't match what your scripts expect. To learn more, refer to the data agent runtimes documentation. Example Query Usability Improvements in Data Agent We've made several usability improvements, for example queries. Errors now surface inline, directly alongside the query, so you can identify and correct issues without leaving the editor. The editor also resizes automatically based on the length of your query, removing the need to adjust the pane manually as you write. To learn more, refer to the example queries in data agent documentation. Add Schema Descriptions for SQL Sources in Data Agent Users can now provide tailored schema descriptions through the new schema description editor, available for SQL sources on the Preview runtime. Schema descriptions improve query generation and accuracy by giving the agent more context about what everything in your data means — use them to resolve ambiguous columns, or to give a table or field a more precise meaning than its name conveys. Instead of inferring intent from column names alone, the agent works from what your data team documented, so it selects the right tables and interprets fields the way you intended. To learn more, refer to the schema descriptions documentation. Data Agent is migrating from Assistants API to Responses API The OpenAI Assistants API that powers the orchestration layer for the Microsoft Fabric data agent, is currently scheduled to be shut down by OpenAI on August 26, 2026. After that date, direct calls to the Assistants API will stop working. If you connect to a Fabric data agent programmatically through the Assistants API, you need to migrate to the data agent Model Context Protocol (MCP) endpoint. SDK and Fabric portal users require little or no action because Microsoft will migrate those experiences internally, although conversation history may reset once. Existing agent data sources, instructions, and tools remain unchanged. To learn more, refer to Prepare your Fabric Data Agent integrations for Assistants API retirement. Fabric Data Agents in Microsoft Copilot Studio (Generally Available) Now, you can bring governed business data from Microsoft Fabric into Copilot Studio agents, so those agents can answer questions and support business processes using trusted enterprise data. Since preview, the integration has moved to the new tool-based experience: select Add a tool, search for Fabric, and add Fabric IQ Data MCP, and your agent can call the Fabric data agent like any other tool. The Fabric data agent still runs in Fabric and respects permissions on the underlying data sources. You can also publish your agent to Microsoft Teams and Microsoft 365 Copilot, so business users get data-grounded answers where they already work. To learn more, refer to the Fabric Data Agent MCS GitHub documentation for setup steps and join the community discussion to share feedback. Fabric data agents in Microsoft Foundry: Easier to connect, easier to trust Fabric data agents in Microsoft Foundry are now easier to connect and easier to monitor. The integration moves to Model Context Protocol, so your Fabric data agents appear as tools that Foundry agents can invoke when they need enterprise data in OneLake. Connecting them no longer means hunting for workspace and artifact IDs. You add the Fabric IQ (OneLake Catalog) tool, filter for data agents, and pick the ones you want by name. You can also connect more than one Fabric data agent to a single Foundry agent, so an agent can draw on a sales agent, a supply chain agent, and a customer support agent and choose the right one for the question. On the operations side, you can now view logs and traces for Fabric data agents through Foundry Observability. Traces show which tools were invoked, how long each step took, and what came back, which makes it much easier to troubleshoot an answer that looks wrong or a workflow that runs slow. This is the visibility teams need to move agents from experiments to production. The update is rolling out to all regions over the coming days. Add Fabric data agent as part of Fabric IQ to your Foundry agent. To learn more, refer to the Observability for Fabric data agents in Microsoft Foundry documentation. Add co-publishers for data agents in Microsoft 365 Copilot When you publish a Fabric data agent to Microsoft 365 Copilot, the Microsoft 365 agent platform registers the agent and records you as its only owner. That created a problem for teams. Your co-creators could still edit the data agent in Fabric, but when they tried to republish it, the operation failed, because Microsoft 365 only lets registered owners publish. Fabric access and Microsoft 365 ownership are two separate lists, so giving someone edit rights in Fabric was never enough. Now, you can avoid this with co-publishers. After you publish the data agent, open Settings, go to the Publishing pane, and add your Fabric co-creators under Microsoft 365 Copilot co-publishers. Each person you add is registered as a co-owner on the Microsoft 365 agent platform, so anyone on that list can republish the agent. Add co-publishers right after your first publish so no one hits a failure in the meantime. Publishing pane in the Fabric data agent settings, showing where you add Microsoft 365 co-publishers. To learn more, refer to the Consume a data agent from Microsoft 365 Copilot (preview) documentation. Data Warehouse Identity columns with identity insert (Generally Available) Since preview, thousands of customers have adopted IDENTITY to auto-generate surrogate keys and streamline migrations from SQL Server, Azure SQL Database, and Azure Synapse. Now we're introducing support for IDENTITY_INSERT and reseed operations - two highly needed additions - so you can insert explicit key values, migrate data in bulk with COPY INTO, and safely realign identity ranges with DBCC CHECKIDENT. Figure: Using identity insert on Fabric Data Warehouse. IDENTITY columns are available now in every Fabric Data Warehouse. To learn more, check our updated tutorial and documentation. CI/CD 2.0 with DacFx (Preview) Microsoft Fabric Data Warehouse is introducing a major update to the DacFx engine that powers schema comparison, Git integration, and deployment pipelines. DacFx builds a declarative model of your warehouse and determines the schema changes required to move safely between development, test, and production environments. Git-integrated CI/CD workflow for Microsoft Fabric, showing feature workspace synchronization, branch merging, and deployment pipeline promotion across development, test, and production workspaces. With this update, Git integration uses DacFx-based incremental extraction to produce cleaner, more focused commits. Deployment pipelines also use the updated model to generate more accurate comparisons and smarter deployment plans, with settings tuned for schema evolution. The new warehouse item definition version 2.0 updates the SQL project SDK, moves shared queries into a .sharedqueries folder, adds project-level Git configuration, and re-extracts object definitions to support constraints, identity columns, clustering, and consistent formatting. These changes make future commits easier to review and reduce noisy diffs. For more information, refer to the Upgrade Fabric Data Warehouse System File Version in a Git Integrated Fabric workspace documentation. Microsoft Fabric source control notification prompting users to apply the latest Warehouse system update, with a warning that the update will introduce differences between the workspace and its connected Git repository. The update also improves comparison accuracy. Git-connected workspaces can adopt the update when ready through the System update available experience, giving teams control over upgrade timing. Review and commit the generated changes before continuing normal development and deployment workflows. To learn more, refer to the Development and Deployment Overview documentation. Simplify Fabric Warehouse deployments with Schema Compare in VS Code Database deployments should not feel like a guessing game. With Schema Compare in Visual Studio Code, developers can see exactly what changed before those changes reach a Fabric Warehouse—bringing clarity and control to every release. Compare a Fabric Warehouse with another warehouse or a SQL database project, then review differences across tables, views, stored procedures, functions, and other database objects in a clear, object-by-object view. Choose the changes you want, update the project from the warehouse, or deploy selected changes to the target—without manually assembling and reviewing every deployment script. By keeping database projects synchronized in Git, teams gain a reliable source of truth and can bring schema changes into familiar pull-request and CI/CD workflows. The result is a safer, more intentional path from development to production, with fewer surprises at deployment time. Before applying changes, review the generated script for unsupported operations and potential data loss. To learn more, refer to the Develop warehouse projects in Visual Studio Code and Schema Compare in the MSSQL extension documentation. GPU Query Acceleration (Preview) Query Acceleration brings GPU-powered performance directly to Fabric Data Warehouse, enabling eligible analytical queries to run faster without query rewrites, special syntax, or additional systems to manage. Query Acceleration in Fabric Data Warehouse uses GPUs to accelerate the most compute-intensive portions of analytical queries, helping overcome the limits of CPU-only execution. It works transparently with existing T-SQL, Direct Query reports, applications, and tools, automatically offloading eligible operations such as scans, filters, joins, and aggregations to GPUs while the CPU continues to manage the rest of the execution pipeline. Customers can use Query Insights, Data Warehouse Monitoring, and SQL Server Management Studio (SSMS) for query execution plans to identify accelerated queries and understand how Query Acceleration is applied during query execution. Designed for analytical and high-concurrency workloads, Query Acceleration can improve throughput, reduce query latency, and deliver more consistent performance for dashboards and interactive analytics. Acceleration is applied selectively, enabling performance gains even when only part of a query is eligible for GPU execution. The capability is built with reliability in mind. Unsupported operations or runtime constraints can seamlessly fall back to CPU execution without affecting query correctness. Performance improvements depend on workload characteristics, but Microsoft benchmarks have demonstrated gains of up to 7× across reporting, application, and AI-driven analytics scenarios. Query Acceleration builds on Microsoft's Tensor Query Processor research, described in CoddSpeed: Hardware Accelerated Query Processing in Microsoft Fabric, which was selected as the SIGMOD Companion 2026 Best Industry Paper. To sign up for the Preview, please fill out the form. Metadata Sync supports Delta Checkpoint V2 (Generally Available) Metadata Sync (MD Sync) now supports Delta Checkpoint V2, enabling synchronization of modern Delta tables across both MD Sync (Legacy) and MD Sync (New). Delta Checkpoint V2 is a Delta Lake enhancement designed to improve scalability for large tables through a more efficient checkpoint structure. Previously, tables using Checkpoint V2 couldn't be synchronized and were reported as unsupported. With this release, MD Sync can discover and synchronize Delta tables that use the Checkpoint V2 format. This enhancement helps customers: Synchronize Delta tables that use Checkpoint V2. Improve interoperability with Spark, Databricks, and other Delta-based platforms. Support metadata synchronization for large-scale Delta tables more efficiently. Continue using existing checkpoint formats without any changes. MD Sync support for Delta Checkpoint V2 is available in both MD Sync (Legacy) and MD Sync (New), helping ensure consistent access to Delta tables across Fabric experiences. Secure data ingestion with COPY INTO and Workspace Identity (Generally Available) COPY INTO in Fabric Data Warehouse now supports Workspace Identity, enabling users to load approved data from OneLake or ADLS Gen2 without requiring direct access to the source files. Previously, ingestion users often needed permissions to both the target warehouse and the source storage location, or teams relied on SAS tokens, account keys, or service principals. With this release, source access can be centrally assigned to the workspace identity, while users retain only the SQL permissions required to load data into the target table. Key Capabilities: Load approved data without granting users direct access to raw storage. Use managed identity-based authentication for OneLake and ADLS Gen2 sources. Reduce reliance on SAS tokens, shared keys, and service principal secrets. Maintain separate authorization boundaries for source access and target-table permissions. Support least-privilege ingestion and separation of duties between storage and warehouse administrators. Workspace Identity support for COPY INTO is generally available in Fabric Data Warehouse, providing a simpler and more governed approach to secure data ingestion. To learn more, refer to the Ingest Data into Your Warehouse Using the COPY Statement and COPY INTO (Transact-SQL) documentation. SQL Audit Logs: More Signal, Less Noise with Predicate Filtering (Generally Available) SQL Audit Logs in Fabric Data Warehouse and SQL Analytics Endpoint now support identity-based predicate exclusion filtering, enabling administrators to reduce repetitive audit events generated by selected users and service principals. Previously, expected activity from automation identities, scheduled processes, metadata synchronization jobs, and other operational actors could create significant audit noise. With this release, administrators can configure exclusions through the API or SQL Audit Logs user experience, while activity from identities that do not match the exclusion predicate continues to be audited normally. Key Capabilities: Reduce repetitive audit events from known users and service principals. Focus investigations on higher-value and unexpected activity. Lower the storage, processing, export, and query burden associated with low-value events. Manage identity exclusions through either automated APIs or the user experience. Apply a governed audit policy aligned with organizational monitoring and compliance requirements. Identity-based predicate exclusion filtering is generally available for SQL Audit Logs in Fabric Data Warehouse and SQL Analytics Endpoint, providing a cleaner audit stream, less operational overhead, and more focused investigations. To learn more, refer to the SQL Audit Logs in Fabric Data Warehouse documentation. OneLake security improvements for SQL analytics endpoints (Generally Available) OneLake Security for SQL analytics endpoints now includes improvements for nested groups, shortcut-backed tables, column-level security, and service principals, enabling more consistent enforcement of OneLake security policies across enterprise Fabric environments. Previously, limitations with group expansion, shortcut scenarios, and service principal ownership could make centralized security difficult to apply at scale. With these improvements, customers can define security at the source lakehouse and rely on the SQL analytics endpoint to honor those policies across producer and consumer workspaces. Key Capabilities: Manage access through nested Microsoft Entra group hierarchies. Honor source-side OneLake Security policies for shortcut-backed tables in hub-and-spoke architectures. Apply column-level security consistently when users receive access through groups. Use service principals for automated deployments, pipelines, and application-owned data products, including service principal-owned lakehouses. Define security once in OneLake and reduce the need to duplicate permissions across consumer workspaces and Fabric engines. These OneLake Security improvements help make security synchronization more practical for enterprise architectures while providing consistent access control across lakehouses and SQL analytics endpoints. Microsoft is also continuing to improve security sync notifications, error handling, and permission propagation across Fabric experiences. To learn more, refer to the OneLake Security for SQL analytics endpoints documentation. Real-Time Intelligence Set Alerts Directly from Anomaly Detector (Generally Available) Detecting anomalies becomes more valuable if you can act on them. Previously, after publishing an anomaly detector configuration, you had to leave Anomaly Detector and navigate to Real-Time Hub to create an alert. This added extra steps and interrupted your workflow right after completing your configuration. With this update, you can now create alerts directly from Anomaly Detector. Once you publish a configuration, use the Set alert button in the ribbon to launch the alert creation pane without leaving Anomaly detector. If your configuration hasn't been published yet, you'll be guided through publishing first and then taken directly to the alert setup experience. This helps you move seamlessly from configuring anomaly detection to monitoring it in production. The integrated experience allows you to monitor your anomalies on each event, helping you get notified as soon as anomalies are detected. If you have more complex business logic, select on each event when to add in additional logic to your conditions. Whether you're monitoring operational metrics, business KPIs, or real-time telemetry, you can now complete the entire workflow in one place and start acting on detected anomalies faster with fewer clicks. Create alerts directly from your anomaly detector configuration and continue your workflow without navigating to another experience. Configure notifications for anomaly detector events directly within Anomaly Detector and start monitoring your published configuration immediately. Anomaly detector supports Eventhouse shortcut tables Anomaly Detector now supports Eventhouse shortcut tables, making it possible to analyze data without first copying or moving it into a dedicated Eventhouse table. You can create anomaly detectors directly on supported shortcut tables and use the same analysis, model recommendations, and continuous monitoring experiences available for native Eventhouse data sources. This expands anomaly detection to a broader range of data already connected through Eventhouse shortcuts, helping teams monitor external and federated data sources with less setup and duplication. By enabling anomaly detection directly on shortcut tables, you can move more quickly from connecting data to detecting issues, while continuing to work within a unified Real-Time Intelligence experience. To learn more, refer to the Anomaly Detection in Real-Time Intelligence documentation. Operations Agent Activity Log Understanding what your agent is doing and why is key to building trust and improving outcomes. The activity log is designed to provide that transparency. It gives you a clear view into the agent’s behavior, including the conditions it evaluated, the recommendations it generated, and how those recommendations were handled. Whether you are validating results, troubleshooting unexpected behavior, or refining your configuration, the activity log helps you better understand how decisions are being made. You can access the activity log from the Activity log section in the side navigation. It presents a chronological timeline of events with timestamps and relevant context for each entry. Selecting any event allows you to explore additional details and understand what happened at each step. In the Operation details page, you can view the operation details and status. To learn more, refer to the Create and Configure Operations Agents documentation. Eventstream MQTT connector (Generally Available) It is now easier than ever to ingest real-time data from MQTT brokers directly into Microsoft Fabric Real-Time Intelligence. MQTT is one of the most widely adopted messaging protocols for lightweight, low-bandwidth event driven messaging scenarios. Eventstream MQTT connector simplifies the ingestion of operational and IoT data into Microsoft Fabric, helping organizations turn real-time device events into actionable insights. Key Benefits: Connect to any MQTT broker and ingest messages directly into Fabric Eventstream. Production-ready reliability and support with General Availability readiness. Enterprise-grade security with support for TLS, mutual TLS (mTLS), and custom certificate authorities managed through Azure Key Vault. Private network connectivity through Eventstream's streaming connector virtual network capabilities, enabling secure access to brokers hosted in private and on-premises environments. To learn more, refer to the Add MQTT source to an eventstream documentation. Reference data enrichment in Eventstream (Preview) Eventstream now enables you to enrich real-time event streams with contextual business data using Reference Data Join. Simply add a Reference Data node to your Eventstream, select a Delta table from a Fabric Lakehouse, and use it to enrich streaming events with lookup, metadata, or reference information. You can also leverage Lakehouse shortcuts to access Delta tables across OneLake, making it easy to bring contextual data from anywhere in your Fabric environment into your real-time processing pipelines. Reference Data Join supports both no-code and SQL-based enrichment experiences. Use the built-in Join operator to visually configure INNER and LEFT OUTER joins or use the SQL operator for advanced scenarios. Select only the columns you need from the reference dataset and configure optional refresh intervals to keep slowly changing reference data up to date. This enables Eventstream to continuously use the latest lookup information for real-time enrichment, without requiring additional data movement or downstream processing pipelines. You can easily add multiple reference data sources to a single Eventstream and combine them with streaming data to create richer, more contextual event pipelines. Developers and data engineers can test and validate join conditions, preview join results, and verify SQL-based enrichment queries before deploying them into production, helping ensure accuracy and confidence in real-time data processing workflows. Reference Data Join unlocks powerful real-time enrichment scenarios in Eventstream. Users can enrich IoT telemetry with device metadata, correlate operational events with customer and product information, perform lookups against business reference datasets, and add contextual information to streaming data in flight. By bringing reference data and event processing together in a single experience, Eventstream enables customers to transform raw events into actionable business insights in real time. To learn more, refer to the Reference data join in Eventstream using Lakehouse documentation. Eventstream observability in Workspace Monitoring re-enabled with per-Eventstream control (Preview) Eventstream observability in Workspace Monitoring is back — now with granular control over which Eventstreams emit monitoring data. A new ‘Log Eventstream activity’ toggle in Eventstream Settings lets you enable or disable observability per Eventstream, so you can balance monitoring coverage with capacity consumption. When enabled, your Eventstream emits performance metrics, error counts, and health status to three tables in your Workspace Monitoring Eventhouse: EventStreamMetrics: throughput, backlog, and watermark delay EventStreamErrorMetrics: deserialization, conversion, and runtime error counts EventStreamNodeStatus: node health (Running / Failed) The toggle defaults to OFF for all Eventstreams. To get started, open any Eventstream, go to Settings, and turn on Log Eventstream activity. Your monitoring data will appear in the Workspace Monitoring database within minutes. Eventstream activity" within Monitoring. The panel highlights an active toggle switch, a description explaining that enabling this feature emits performance and error metrics to a monitoring database. To learn more, refer to the Monitor Eventstream data flows in Workspace Monitoring documentation. Eventstream UI editor improvements (Preview) We've redesigned key parts of the Eventstream editor to make building and troubleshooting faster and more intuitive. Always Publish: No more blocked publish buttons. Publish your work at any stage, the editor gives you clear, contextual guidance on what still needs attention instead of preventing you from moving forward. Inline error indicators: Errors now appear directly on the node that needs fixing, with actionable guidance on click. No more hunting through a detached error list to find what's broken. Operator and destination descriptions: Each option now includes an inline description explaining what it does, so you can build confidently without switching to docs. These changes reduce friction during authoring and make it easier to go from idea to running your pipelines. Secure Azure Event Hubs Connections in Eventstream with Workspace Identity (Preview) Bringing real time event data into Microsoft Fabric is now simpler and more secure with Azure Event Hubs integration for Eventstream. Organizations can connect Event Hubs directly to Eventstream and start routing event data to destinations such as Eventhouse for analytics and operational insights. A key capability is Workspace Identity, which removes the need to manage shared access keys. Instead, Eventstream can authenticate to Azure Event Hubs using the Fabric workspace identity. Administrators simply grant the workspace the Azure Event Hubs Data Receiver role, enabling secure access through Microsoft Entra based permissions. This approach improves security, simplifies credential management, and aligns with enterprise governance requirements. The integration supports both public and private network deployments. For Event Hubs hosted in private networks, organizations can connect through a streaming virtual network gateway while continuing to use Workspace Identity for authentication. For advanced event processing scenarios, users can enable schema support, associate schemas from the event schema registry, and route structured events to destinations such as Eventhouse. Combined with Workspace Identity, this provides a secure and scalable foundation for building real time data pipelines without the operational overhead of managing secrets or credentials. Schema Registry and Event SchemaSet Region Availability Previously, Schema Registry feature and the Event SchemaSet artifact was available for preview in 31 regions. Expanded to 10 additional regions: Geography Region Americas Central US Americas Mexico Central Americas West US 3 Europe Italy North Europe Poland Central Europe Spain Central Europe West Europe Asia Pacific Australia Southeast Asia Pacific Israel Central Asia Pacific Japan West If you were previously blocked from trying SchemaSets and schema-based Eventstream data ingestion, you can now do so in these regions. For more information on Event SchemaSets and how you can create and manage them, visit Schema Registry Overview. To learn more about configuring Eventstreams with schema-based sources, visit Use schemas in Eventstreams. Data-driven styling and UX improvements for Maps (Generally Available) Maps become most valuable when they help users understand not just where things are, but what the data means. Now, Data-Driven Styling, along with Markers Rotate by Data, Traffic Flow visualization, additional Map View options, and an improved Layer Settings experience. Together, these enhancements help organizations transform raw geospatial data into intuitive, actionable business insights. Let Your Data Tell the Story Understanding patterns hidden within geographic data can be challenging when every feature on a map looks the same. With Data-Driven Styling, Fabric Maps enables map builders to visually represent business data directly on the map, helping viewers quickly identify trends, hotspots, and outliers without inspecting individual records. The new Color by Value Range capability allows authors to style map layers using numeric measures such as revenue, utilization, sensor readings, environmental measurements, or operational KPIs. Instead of applying a single color to an entire layer, Fabric Maps visualizes value distributions chromatically, making important differences immediately visible. Organizations can choose between two visualization approaches: Gradient Styling uses continuous color transitions to reveal magnitude, trends, and geographic variation across a dataset. Step-Based Styling allows users to define custom value ranges with distinct colors, making it easy to visualize business thresholds, risk levels, performance bands, or service categories. To make these visualizations easier to interpret, Fabric Maps automatically generates corresponding data legends that explain how colors map to underlying values. This helps viewers understand the meaning behind the visualization and make decisions with greater confidence. Fabric Maps also includes thoughtfully designed color palettes, including options that support colorblind-friendly visualization scenarios, helping more users accurately interpret map-based insights. Combined with clear, automatically generated legends, these capabilities improve accessibility and make data-driven maps easier to understand across a wider range of audiences. per street and road and high-value exposure in flood-prone areas. Visualize Direction and Movement with Markers Rotate by Data Many operational scenarios involve not only location but also direction. Fabric Maps now supports Markers Rotate by Data, allowing marker symbols to automatically rotate based on values stored in a data column. This capability is available for Marker layers, enabling builders to visualize directional information directly on the map. Whether visualizing aircraft headings, vehicle movements, equipment orientation, or other operational workflows, map authors can represent direction without requiring custom visualization development. By transforming static points into directional indicators, organizations can add valuable operational context and communicate movement patterns more effectively. Add Real-World Context with Traffic Flow Visualization Location data alone doesn't always tell the full story. Fabric Maps now supports Traffic Flow overlays, allowing map builders to bring current traffic conditions into their existing map experiences. By combining business data with real-world traffic information, organizations can gain additional situational awareness for logistics operations, field service planning, transportation monitoring, and operational decision-making. The added context helps users better understand the environment surrounding their assets and activities without leaving the map experience. Configure the Right Map View for Your Audience Organizations often create maps for users across different regions and business contexts. Fabric Maps introduces additional Map View settings that allow map builders to configure how geographic information is presented, helping ensure maps are displayed in a way that aligns with organizational needs and audience expectations. This flexibility gives authors greater control over creating a consistent and intuitive viewing experience across a variety of business scenarios. A More Discoverable Authoring Experience UX research and user feedback showed that some key layer settings were difficult to discover. We updated the experience by moving geometry and visualization options—including visual type, Data-Driven Styling, and marker rotation—higher in the configuration pane. We also renamed General to Visibility, making the settings clearer and map authoring more intuitive. Turn Location Data into Business Insight Data-Driven Styling, built-in data legends, Markers Rotate by Data, Traffic Flow overlays, enhanced Map View options, and the improved Layer Settings experience help organizations transform location data into meaningful business insight. Together, these capabilities make it easier to uncover patterns, understand operational context, and communicate geospatial insights across teams. Start building with these capabilities today: explore the customization options, apply them to your own geospatial data, and create map experiences that turn location into action. To learn more, refer to the Customize a map in Microsoft Fabric documentation. Workspace Outbound Access Protection (OAP) for Operations Agent (Preview) Workspace Outbound Access Protection (OAP) in Microsoft Fabric helps admins secure outbound connections from workspace items to external resources. Admins can control outbound access by blocking unwanted connections by default and allowing only approved connections through configured rules. As organizations adopt AI-powered operations at scale, governance and security remain critical requirements. With this preview release, Microsoft Fabric introduces Outbound Access Protection (OAP) for Operations Agent, enabling workspace administrators to control the outbound actions an agent can perform. OAP applies workspace-level policies to actions such as Teams notifications, workflow triggers, and cross-workspace operations, helping organizations enforce security and compliance requirements while continuing to benefit from AI-driven automation. When OAP is enabled, Operations Agent continues to perform core functions including reasoning, recommendation generation, rule evaluation, and telemetry collection. However, outbound actions are governed by the workspace's configured access policies. Administrators gain greater visibility through in-product notifications, Teams messaging experiences, and the Operations Agent Activity Log, making it easier to identify and troubleshoot blocked actions What's new with Operations Agent and OAP? Govern outbound agent actions through workspace-level OAP policies. Control whether Operations Agent can send Teams notifications based on allowed connections. Prevent unauthorized cross-workspace actions when OAP policies restrict outbound access. Receive clear visibility when actions are blocked through in-product notifications and Teams messaging experiences. Monitor agent activity and OAP-related outcomes through the Operations Agent Activity Log. During the preview, some limitations apply. Cross-workspace actions are blocked when OAP is enabled. For example, Power Automate actions are not yet supported when OAP is applied, and only connectors that explicitly support OAP policies can be permitted. By extending Fabric's outbound governance framework to Operations Agent, organizations can adopt AI-powered operational automation with greater confidence while maintaining control over how and where agent-initiated actions are executed. Resources Workspace outbound access protection for operations agent (preview) Workspace Outbound Access Protection (OAP) Workspace Outbound Access Protection for Operations Agents Configure and manage Activator rules directly in Eventstream (Generally Available) You can now create and manage rules directly in Eventstream. Previously, setting up an alert required switching from Eventstream to Activator. While powerful, this meant moving between experiences to complete a single workflow. Now, alert creation is embedded directly into Eventstream. Capabilities for building or editing your Eventstream: Select the stream you want to monitor. Choose Set Alert. Define your condition (thresholds, aggregations, patterns). Configure the action. Create the rule. Capabilities for Activator destination created on Eventstream: Select Activator node Select Rule icon Create the rule Once you have the rule(s) created on your Eventstream, you can manage them by editing, deleting or opening in Activator. To learn more, refer to the Add a Fabric activator destination to an eventstream documentation. Data Factory Introducing Hierarchical Navigation in Monitoring Hub for Fabric Pipelines Modern data estates rarely consist of a single job running in isolation. Pipelines trigger notebooks, notebooks invoke other workloads, and business processes span multiple interconnected executions. When troubleshooting a failure or understanding lineage, customers often need visibility into how these executions relate to one another. Hierarchical Navigation in Monitoring Hub, is a new capability that helps you understand the relationships between runs and quickly navigate across upstream and downstream executions. With Hierarchical Navigation enabled, Monitoring Hub can display: Upstream runs that initiated a workload Downstream runs triggered by a workload Execution relationships across supported Fabric artifacts This provides a richer observability experience by helping you move beyond individual run monitoring and understand how your workloads operate together. This enhancement is another step toward a richer observability experience in Fabric, helping customers gain deeper insight into workload execution and dependencies at scale. To learn more, refer to the Hierarchical Navigation for Pipelines in Monitoring Hub documentation. Explore the modern Fabric Pipeline canvas (Preview) The new Fabric Pipeline canvas experience is designed to make pipeline authoring easier than ever. Key capabilities with modern canvas: Better visibility when navigating large pipeline graphs Cleaner, more structured layouts for complex orchestration logic Improved responsiveness when working with enterprise-scale workflows A more intuitive experience for pipelines with many activities and branches The modern Fabric Pipeline canvas, showing the updated node experience and option to disable the preview if needed. If you haven't tried it yet, now's the perfect time. The new experience is rolled out automatically and can be disabled at any time. Whether you're building your first pipeline or managing hundreds of activities across complex workflows, the new canvas is designed to help you stay productive and focused on what matters most. To learn more, refer to the Modern Pipeline Node Experience documentation. Upgrade Dataflow Gen1 to Dataflow Gen2 (CI/CD) using the Upgrade Wizard (Preview) The Dataflows Upgrade Wizard is a guided, end-to-end experience that upgrades your existing Power BI Dataflow Gen1 items to Dataflow Gen2 (CI/CD) in Microsoft Fabric with minimal effort. You can upgrade a single dataflow, or several dataflows from a workspace in a single flow. Previously, bringing a Gen1 dataflow into Fabric meant recreating it and repointing everything that depended on it. The Upgrade Wizard upgrades in place instead. Each dataflow keeps its ID, name, schedule, and connections, so the reports and semantic models that connect to it keep working without any changes. Before anything changes, the wizard assesses every dataflow in the workspace and tells you which ones need attention and why, such as incremental refresh settings to reconfigure or a linked entity to update, so you know what to expect before you upgrade. Why this matters Upgrade in place, with nothing to rebuild and nothing to repoint. Upgrade a whole workspace at once instead of one dataflow at a time. See what needs attention before you upgrade. Unlock Dataflow Gen2 innovations: improved performance, deeper Fabric integration, CI/CD and Git support, data destinations of your choice, richer diagnostics, Copilot-assisted authoring, and a modern data transformation foundation. The wizard is available for Dataflow Gen1 items in Premium or Fabric workspaces and requires Fabric to be enabled. To learn more, refer to the Upgrade Dataflow Gen1 to Dataflow Gen2 (CI/CD) using the Upgrade Wizard documentation. Extended watermark support in Copy job Watermark-based incremental load support enables Copy Job to efficiently ingest only new or changed data from key enterprise and SaaS data sources, avoiding costly full reloads. This reduces source-system impact, network traffic, and runtime while improving scalability for production analytics workloads. Customers can now use incremental loading across more of their critical data sources, including Salesforce, Informix, Cassandra, Greenplum, Presto, and Databricks, by leveraging Copy job’s built-in watermark mechanism and without building custom ingestion logic. To learn more, refer to the Incremental copy in Copy job documentation. Enable Change data feed during Lakehouse table creation in Copy job Copy job can now create Lakehouse tables with Change Data Feed (CDF) enabled automatically. There’s no longer a need to pre-create destination tables or manually configure Delta table properties. Simply select Enable CDF on destination and Copy job takes care of the setup for you. This ensures the table is immediately ready for incremental processing and downstream CDC scenarios, helping reduce data movement and improve replication efficiency. By eliminating manual configuration steps, it makes advanced data integration patterns much easier to adopt and operate at scale. To learn more, refer to the Automatic table creation and truncation on destination documentation. Amazon Redshift as new source in Copy job As part of our mission to enable multi-cloud data movement at petabyte scale with Copy job, we are bringing Amazon Redshift support as a source. This enables customers to seamlessly ingest data from one of AWS's most widely adopted data warehouse platforms directly into Fabric. Redshift support further strengthens Fabric's vision of delivering an open, connected, and multi-cloud data platform. To learn more, refer to the Connectors for Copy Job documentation. Copy job supports timestamps without time zone in Lakehouse Support for timestamps without time zones (timestamp_ntz) allows Fabric Lakehouse tables to preserve date and time values exactly as stored, without applying time zone conversions. Copy job can now automatically map timezone-independent datetime values to Delta Lake timestamp_ntz, ensuring greater compatibility with source systems that allow storing date and time values without time zone information. Migration Assistant for SQL database in Fabric (Generally Available) The guided, Fabric-native wizard takes you from a source SQL Server schema to a running SQL database in Fabric: upload a DACPAC, review compatibility results, deploy the schema with Copilot-assisted fix suggestions, and copy your data using built-in Fabric Copy Jobs. We've added the capability preview customers asked for most: Validate. You can now check out a DACPAC for compatibility before creating a SQL database in Fabric. Upload the file and the assistant reports which schema objects will deploy cleanly, which ones use features that aren't supported, the reason behind each failure, and the dependencies between objects — with nothing provisioned and no capacity consumed. That means you can scope migration effort, plan remediation, and get change-approval sign-off before you commit to a target database. To get started, select Migrate in your Fabric workspace and choose Migrate to SQL database in Fabric. To learn more, refer to the Fabric Migration Assistant documentation. Until next month That's a wrap for the August 2026 Microsoft Fabric Monthly Update. As always, we'll continue sharing new capabilities, enhancements, and improvements across Microsoft Fabric in future monthly updates. Thank you for being part of the Fabric community!A new analytics frontier: GPU-accelerated Fabric Data Warehouse (Early Access Preview)
As data volumes grow, concurrency rises, and analytics workloads become more dynamic and AI-driven, performance becomes harder to predict and harder to scale. Every query sits in the critical path, adding pressure to the warehouse, and every second counts. This is the core tension in analytics today. The expectations have changed, but the underlying technology has not, leaving agents, applications, and AI systems waiting on data. To meet this moment, analytics needs a new kind of execution engine.
Microsoft Fabric September 2023 Update
Welcome to the September 2023 update. We have lots of features this month including updates to the monitoring hub, Fabric Metrics app, VS code integration for Data Engineering, Real-time data sharing and many more. Continue reading for more details on our new features! Contents Core Monitoring hub – column options OneLake OneLake file explorer (v.1.0.10) - Menu Option to View Workspaces and Items Online Power BI General Upgrade Power BI Desktop .NET Framework to 4.7.2 or newer Reporting Mobile layout interactive canvas Smart Narrative improvements Modeling Edit your data model in the Power BI Service - Updates Edit linguistic relationships in Q&A setup DAX function changes to MINX and MAXX Edit relationships in the properties pane (GA) Data Connectivity SAP HANA (Connector Update) Emplifi Metrics (New Connector) Service Row-level security test as role improvements Mobile Supporting AAD Shared device mode (preview) Developers Git integration Visualizations New visuals in AppSource Change Chart Beeswarm Chart Explore time-based data down to each millisecond with Drill Down TimeSeries PRO New Updates for accoPLANNING (Release 57) Zebra BI Charts on-visual settings Word Cloud by Powerviz Innofalls Charts: A Versatile and Interactive Visual Hierarchy Chart by MAQ Software Power BI JSON Report Theme Generator by BIBB Synapse Data Warehouse Data Warehouse Utilization Reporting in Fabric Metrics app Column-Level security is now available on Data Warehouse and SQL Endpoint Data Warehouse Row Level Security SQL Projects support for Fabric Data Warehouse Deployment Pipelines Default Dataset Improvements Data Engineering VS Code Integration Introducing Notebook File System support in Synapse VS Code extension for Data Engineering and Data Science workload in Microsoft Fabric Import Notebook in Workspace view Mssparkutils new API for fast data copy Support diff versions when saving notebook Notebook Sharing with Execute-only mode Notebook resources .whl file support Real-time Analytics OneLake shortcut to delta tables from KQL DB Real-Time Data Sharing: Introducing database shortcuts in Real-Time Analytics Sample Gallery – Explore what you can achieve in Real-Time analytics Model and Query data as graphs using Kusto Query Language (KQL) Easily connect to KQL Database from Power BI desktop Eventstream now supports AMQP format connection string for data ingestion Eventstream supports data ingestion from Azure IoT Hub Improved Eventstream Creation Speed Data Factory Data Pipeline Dataflow Gen2 New Service Principal authentication kind Bug fixes and reliability improvements Community New learning path: Implement a Lakehouse with Microsoft Fabric Core Monitoring hub – column options We have released a new feature inside the monitoring hub to better users customize their experience. Column options gives users more room to operate. Users can select and reorder the columns that meet the scene according to their customized needs. Regardless of whether the user switches to any other tool or scene in Fabric, these columns options are persisted along with the filter until the user returns again. You can now easily reorder the columns with drag and drop inside the option menu. You do not need to worry about the ones at the bottom. All the selected column options will be bubbled up to the top of the menu when you open it next time. Try it out yourself! OneLake OneLake file explorer (v.1.0.10) - Menu Option to View Workspaces and Items Online We are excited to announce the new release of OneLake file explorer for Windows! Now you can seamlessly transition between using the OneLake file explorer app and the Fabric web portal. When browsing data in OneLake file explorer, right click on a workspace and select “OneLake->View Workspace Online.” Selecting this will open the workspace browser on the Fabric web portal. Similarly for items within your workspace. In addition, you can now easily find your client-side logs, which you may need to troubleshoot issues. Right-click on the OneLake icon in the Windows notification area, select Diagnostic Operations, then Open logs directory. Get started by downloading the latest OneLake file explorer. Power BI General Upgrade Power BI Desktop .NET Framework to 4.7.2 or newer Power BI Desktop with .NET Framework 4.5 will no longer be supported after 9/30/2023. Please upgrade to .NET Framework 4.7.2 or newer to avoid any support issues. Reporting Mobile layout interactive canvas With this latest update we are happy to announce that we have made the mobile canvas interactive. This new capability provides the ability to test how buttons, slicers, and visuals will behavior on the app before publishing the report. But that's not all. With this canvas interactivity, users can now interact with visuals directly and adjust Table and Matrix column headers to align perfectly with mobile screens. Smart Narrative improvements The Smart Narrative visual can now display either text or numeric values when showing data for a column. Previously this was limited to only numeric values. Learn more about the Smart Narrative’s summarization capabilities at Create smart narrative summaries - Power BI | Microsoft Learn. Modeling Edit your data model in the Power BI Service - Updates The new data model editing in the Service feature was released to preview in April. We’ve been busy reacting to your feedback and enhancing the experience. Below are the improvements coming later this month: Improve layouts limitations Changes you make to layouts will now persist between Desktop and the Service, including: Changes made to data model layouts in Desktop will now be incorporated into the Service upon uploading the .PBIX file. Similarly, changes made to layouts in the data model within the Service will now be incorporated in the Desktop upon downloading the .PBIX file. Please continue to submit your feedback directly in the comments of this blog post or in our feedback forum. Edit linguistic relationships in Q&A setup The Q&A visual is an effective way to help users further understand their data by asking questions and receiving answers in the form of visuals. It offers users a way to explore their data in ways not covered by the rest of the report without requiring deeper knowledge of their data model or report authoring. However, while the Q&A engine is good at answering precise questions about data, it may not be able to associate every word or phrase a user inputs with data in the model. For example, answering “what are our best consoles this year?” may require connecting the term consoles to the name products in the model, and understanding that the term best corresponds to the highest sales values. These terms are contextual, however – users could mean something completely different asking for console and best in other industries, organizations, or even datasets. To help authors ensure that the Q&A visual provides consistent and accurate answers based on the unique language their report consumers actually use, we introduced Q&A setup tools with an emphasis on providing Q&A with synonyms for column and table names in the model. This way, authors can explicitly define console as referring to products, and users will always receive the correct answers when they ask similar questions in the future. However, synonyms (nouns) are only half of the picture. The other half of the terms (adjectives, verbs, prepositions, adverbs) can’t be defined with such straightforward mappings because they must be understood as a part of a phrase – they qualify other terms or relate other terms together. Best in the previous example is one; asking “who sold the most books” requires us to know that stores sell books, connecting stores to books. There are many types of these linguistic relationships, so we built a new tab entirely to help you create and manage linguistic relationships for your data. You can get into the Q&A setup menu using the gear icon on the Q&A visual or the Q&A setup option in the Modeling tab of the ribbon, then selecting the new Relationships tab. There, you’ll be able to define a variety of relationships, including verb, adjective, noun, preposition, and more. Choose a type which fits the term you’re trying to define (for example, “best” is an adjective), then follow the prompts to define what it means in the context of your data. Our investment into Q&A does not stop here. Even in a world where natural language capabilities are increasingly driven by large language models, there is value in the precision, consistency, and customizability of our sophisticated Q&A engine. In the other direction, defining synonyms and relationships can be a lengthy process just asking to be streamlined with the power of AI-generated suggestions. Keep an eye out in the future for the ways we’re bridging the two to bring out the best of both worlds! DAX function changes to MINX and MAXX We have added an optional variant parameter to MINX and MAXX DAX functions. These functions ignore text and Boolean values when there are variants or mixed data types, such as text and numeric. Now with the new optional variant parameter set to TRUE, the functions will consider the text values. Here is an example of a variant measure. In the table below the variant measure is ordered in ascending order: MINX and MAXX without the variant parameter set, or set to FALSE, will ignore text and Boolean data types. MINX and MAXX with the variant parameter set to TRUE will now include text values. Boolean values are still ignored. The default of the MINX and MAXX optional variant parameter is FALSE, so all existing DAX expressions using MINX and MAXX will not be impacted by this change. To allow text values to be included, you can update the DAX expression to include TRUE as the third parameter. Read more about these changes at Microsoft Learn: MINX function (DAX) - DAX | Microsoft Learn MAXX function (DAX) - DAX | Microsoft Learn Edit relationships in the properties pane (GA) We are excited to announce the general availability of the edit relationships in the properties pane! The edit relationships in the properties pane has been available for public preview since the October 2022 release. Now you can click on any relationship line in the modeling to edit your relationship in the properties pane. This is available in the Model view of Power BI Desktop and in web modeling of the Power BI service. Edit relationships without running queries to preview the data and only validating the relationship when you click apply changes. A welcome relief for those using DirectQuery storage mode, as these queries can take time and impact your data sources. You can learn more about how to utilize this feature at Create and manage relationships in Power BI Desktop - Power BI | Microsoft Learn. Data Connectivity SAP HANA (Connector Update) The update enhances the SAP HANA connector with the capability to consume HANA Calculation Views deployed in SAP Datasphere by taking into account SAP Datasphere’s additional security concepts. This enables consumption of Calculation Views in Datasphere and allows customers to connect to HANA Cloud views without the need for additional privileges on the _SYS_BI schema. Emplifi Metrics (New Connector) We are happy to announce the release of the new Emplifi Metrics connector. Please find release notes from the Emplifi team below: “Integrating social media insights alongside the rest of your marketing or business intelligence data gives you a holistic understanding of your entire digital strategy, all in one place. With Emplifi Power BI Connector, you’ll be able to include social media data from the Emplifi Platform in your charts and graphs and combine them with other data you own. The Power BI Connector is a layer between Emplifi Public API and Power BI itself. It helps you work with your data intuitively, directly in the Power BI tool. The majority of data and metrics available in the Emplifi Public API are also available in the Connector. Please visit the official documentation for more information about Emplifi Public API and a list of available metrics. You’ll find it here: https://api.emplifi.io/.” Service Row-level security test as role improvements We have listened to your feedback about the row-level security test as role experience in the Service and have made several improvements this month including: We have added a new dropdown, allowing you to choose and test any report connected to the dataset. You can now easily see important permissions details pertaining to a specific individual while role testing for that person. We have improved our warning and error messages. To learn more about validating row-level security roles in the Service read our documentation. Mobile Supporting AAD Shared device mode (preview) We, in Power BI continue our investments in empowering frontline workers with data and insights in their work. And in this month release, the Power BI mobile apps add support in Azure active directory shared device mode. Shared Device Mode enables you to configure devices to be shared by a group of employees. This is very common for frontline workers, who do not use their personal device for work related tasks but are getting a work-related mobile device from a shared pool of devices for these tasks. The Shared Device Mode enables single sign-on (SSO) and device-wide sign out for Microsoft Power BI and all other apps that support Shared Device Mode. Once a user signs in into a supported app, the user will be logged into any application compatible with Shared Device Mode as well. Once a user signs out of a supported app, all other applications integrated with shared device mode sign out, to to prevent unauthorized or unintended access by the next user. Back to the frontline worker scenario: when an employee picks a device from a pool at the start of their shift, they need only a single sign-in to one supported app (for example Power BI), and they’re automatically signed in to all other apps that support shared device mode on the device, so the device and the apps are ready for work with the user’s account. At the end of their shift, when they sign out of one app, they're signed out globally from all other apps that support shared device mode, having the device is ready for the next employee and can be safely handed off. Developers Git integration Paginated reports now supported with git integration Since Git integration has been launched few months ago, we supported only 2 items in Power BI- Power BI reports and Power BI datasets. We are now excited to add a new item- Paginated reports! After connecting your workspace to Azure DevOps, you can commit your paginated reports and have them versioned in your repository. After that, you can open the .rdl files directly from git in Power BI Report Builder, edit and push the changes into git. The workspace will identify the changes and will prompt developers to update the workspace with the changes to the paginated report. With this feature, we are adding paginated reports developers to enjoy the collaboration, versioning and modern developer workflows offered in Power BI and Fabric. Learn more about using git with paginated reports. Checkout and Switch Branches in a Workspace A workspace can connect to a single branch at a time. Once you’ve setup your separate workspace to work with git, there might be scenarios where you want to change just the connection of a branch. We have now added 2 new features to help you achieve this much faster: Checkout branch- checkout allows you to easily create a new branch, based on the current state of the workspace. After choosing to checkout, you can create a new branch that will be connected to the WS, while the uncommitted changes are retained. This is useful in cases of conflicts, that allows you to commit your changes to a backup branch, and then manage the merge conflict in the git repo. Switch branch- Workspace admins can decide to change just the connection of the branch very easily. It’s helpful in cases you want to start working on a new branch and wipe clean your WS content for that, or when you are moving from your regular project to small bug fixes (assuming they are on the same content), that should happen on a different branch. New E2E CI/CD tutorial now available Many users have been asking, since the release of git integration, how to work with deployment pipelines and git as part of a CI/CD process. These two features can help build a compelling e2e flow, when connected properly. Follow this step-by-step guide to learn how to make the most out of both tools together. Visualizations New visuals in AppSource Change Chart Pro Circle Card by Devlup Funnels Apex Milestone Trend Dot Chart Activity Gauge by Powerviz Bridger Visual by BI Samurai Number lines by BI-Champ Funnel Chart by Powerviz Date Picker by Powerviz Bar Chart with Export Data Feature Wordcloud by Powerviz PackedBubbleChart Box Ploty by Devlup Funnels LeapLytics - Markdown Viewer Aimplan Data Input Table Beeswarm Chart Overview of Beeswarm Chart Beeswarm chart shows the distribution of data along one axis or both axes while also showing individual points. It is like a one-dimensional or two-dimensional scatter plot but with closely-packed, non-overlapping points. How to use this Visual The visual is intuitive and easy to follow. Only 1 field is mandatory i.e. Bee Category. Tip: If you don’t have any category, you can simply add a calculated column with any name and use it in the category field Bee Size field is optional and is represented in the visual by bubble area or size. If this field is missing then all bubbles will be of same size Bee Label field is also optional. It is used to add labels to each individual bubbles. This chart can be used in 3 ways Y – Axis only X – Axis only Scatter Plot (Both Y and X Axes) 1) Y-Axis Only As shown in picture below, if you only use a column/field in “Beeswarm Y-Axis” section while keeping “Beeswarm X-Axis” section empty, your data will be plotted on Y-Axis categorized according to the “Bee Category” section 2) X-Axis Only Similarly, if you only use a column/field in “Beeswarm X-Axis” section while keeping “Beeswarm Y-Axis” section empty, your data will be plotted on X-Axis categorized according to the “Bee Category” section 3) Scatter Plot (Both X and Y Axes) If you use columns/fields in both sections i.e “Beeswarm X-Axis” section and “Beeswarm Y-Axis” section, the visual will work like a scatter plot with data points plotted on X-Axis and Y-axis . This is depicted below You can show images inside bubbles as well using the “Images” field. For example, the report below shows top millionaires of the world with their net worth on Y-Axis and their Age on X-Axis. Its available in demo file. Formatting Options These options are available in the format pane of the visual under “Settings”. 1) Bubble size As the name implies, this setting simple increases / decrease the bubble size 2) Lower Upper and Left Space Beeswarm chart produces non colliding bubbles or in other words circles don’t overlap. This can often cause bubbles do go outside the chart area especially when there is a concentration of data around minimum and maximum data values. This is where these settings are very useful to bring data inside the chart area. These settings increase or decrease the Y-Axis and X-Axis range to achieve this objective. Try adjusting these settings on the demo file’s sheet “ScatterPlot Beeswarm” to see how they work. Purchase of Premium Features To use the visual without Watermark in Power BI Online, please purchase the Visual’s license for a 1 year period. The license is for unlimited number of users and viewers. If you have any questions, please send me email at [email protected] Download Download the demo file here. Download the custom visual from APPSOURCE Explore time-based data down to each millisecond with Drill Down TimeSeries PRO Drill Down TimeSeries PRO lets you create timeseries charts that are easy to explore on any device. Using its intuitive interactions, users can drill down to months, days or hours by clicking on the chart. Combine up to 12 series and choose between multiple charts - line, column, area. Learn more about Drill Down TimeSeries PRO! MAIN FEATURES: On-chart interactions – pan, zoom and drill down Full customization – set intervals, min/max values, colors, fonts, stacking and clustering Static and dynamic thresholds – set up to 4 thresholds to demonstrate KPIs Cross-chart filtering – select data points on multiple charts instead of using slicers Touch device friendly – explore your data anywhere POPULAR USE CASES: Finance – stock exchange indices, capital ratios, transaction volumes Sales – web traffic, audience reach, lead volume IT – network traffic, response times, syslog and error trends Logistics – inventory movement and turnover, loading time, fleet maintenance costs Get Drill Down TimeSeries PRO now! This visual comes with 30 days free access to paid features. ZoomCharts Drill Down Visuals are known for interactive drilldowns, smooth animations and rich customization options. They are mobile friendly and support: interactions, selections, custom and native tooltips, filtering, bookmarks, and context menu. New Updates for accoPLANNING (Release 57) accoPLANNING for Power BI empowers business users with writeback capabilities, advanced planning, forecasting, budgeting, project management and analysis solutions. We have some new and exciting features for you in this release: • Custom calculated rows and columns configuration. It allows you to customize your own client-side calculations in your already established accoPLANNING table. • End user Lock cells. This gives the flexibility for the end user to lock cells and make sure these will not be affected by splashing or use as an approval indicator. • Hide rows and columns. This Allows users to hide rows and columns - adding flexibility in making asymmetric column and row selections in the grid for better reporting or just for ad-hoc purpose to get a better overview. The latest update also includes a range of new features and improvements designed to increase your productivity: • Our navigation has undergone a major overhaul, with the most notable change being the revamped toolbar. • Better API error message handling in the grid. • Support for automatically expanding all rows/columns/both, eliminating the need to manually adjust the size of each cell. With the accoPLANNING visual, you combine the planning and reporting process in Power BI. For more information, visit our website. https://www.accotool.com/ https://appsource.microsoft.com/en-us/product/power-bi-visuals/WA200002600?tab=Overview https://youtu.be/kNIVC6rBJWA Zebra BI Charts on-visual settings Zebra BI visuals for Power BI are taking another step further to make the user experience as seamless and interactive as possible. Zebra BI Charts has received some important updates so that you save time while customizing your visuals. Thanks to the on-visual settings, you don’t need to go to the visualizations pane every time you want to make a change. COMMENTS SETTINGS can now be adjusted directly on the visual by simply clicking on the settings icon next to the comment box. You can adjust the title, which variances you display, the icon, gap between the comments, and padding. CATEGORY SETTINGS let you adjust several settings by just clicking on the category area. Customize the axis font (family and color), trim /rotate long labels, adjust the label density, and set the gap between the columns. LEGEND SETTINGS come in handy when you want to rename the entries and don’t want to search for this option in the formatting pane. Adjust margins, use aliases in tooltips and switch comparisons. STACKED CHARTS SETTINGS on the visual let you adjust the Top N feature, set color of the chart, and display labels as %. Try it on your data for free. Word Cloud by Powerviz We are excited to announce the new Word Cloud by Powerviz, it's an advanced visual which empowers you to create some of the most high-quality and creative word art in the Power BI. Key Features: Word Styling: Make your word clouds pop with personalized text styles. It offers font styling, direction & text editing features. Color Options: Choose from 30+ color palettes, including color-blind safe options. Shapes: Want to make a statement? Create captivating word clouds by choosing shapes from icons and images, or upload your own image. Exclude: No more hassles anymore! Easily remove unwanted words, symbols from the text to create a clean and focused word cloud. Ranking: Filter out Top/Bottom N Words. Conditional Formatting: Easily spot words with dynamic rules. Many other features included lasso/reverse lasso, grid view, show condition, and accessibility support. Business Use Cases: Marketing: Sentiment analysis & SEO keywords. Education: Brainstorming, Improve engagement. Market Research: Opinion poll, open-ended survey. Presentation: Capture attention & easy communication. Try Word Cloud Visual for FREE from AppSource Check out the visual features in demo file Step by Step instructions and documentation To learn more, visit Powerviz website. Introducing Word Cloud by Powerviz – A Powerful Power BI Custom Visual on YouTube Innofalls Charts: A Versatile and Interactive Visual Innofalls Charts is a powerful visual that offers a wide range of interactive features. Our bar and waterfalls charts come with a drill-down history, enabling orientation and facilitating comparisons. All charts can be stacked or grouped into columns. Waterfalls: Our special attention to waterfalls includes running totals and delta waterfalls, enabling effective comparisons of measures or members. The waterfalls are expandable, drillable, stackable, and offer intermediate sums. Drill Down History: Keep track of your drill downs with visual feedback. Change your drill downs retroactively and combine multiple drill downs for in-depth exploration. New Comparison Features: Effortlessly make comparisons while viewing your report and incorporate deviation charts with a single click. Interactivity: Enjoy various interactive features such as drill, expand, undo/redo, dynamic comparisons, and zoom for enhanced data exploration. IBCS Formatting: Our IBCS theme provides scenario formatting, equal scaling, highlights, deviations and more for consistent and standardized reports. Animations: Enhance data comprehension and reduce change blindness with animations. Promptly detect resorted or added data elements. Discover the full potential of Innofalls Charts and get started today. Innofalls Charts is free for Power BI Desktop! Get started today [ https://www.innofalls.com/blog-get-now ] Visit our website [ https://www.innofalls.com/ ] Hierarchy Chart by MAQ Software Struggling to visualize complex hierarchical relationships? Improve understanding and decision-making with the Hierarchy Chart by MAQ software. In today's data-driven environment, visualizing hierarchical relationships is key to business insights and decision-making. From sales structures to budget allocations, our Power BI-verified visual offers a customizable way to represent these structures easily, catering to various business needs. Figure 1: Sample visual showing a color-coded organization chart (with a tooltip). Key business uses: Organization/HR: Organize human resources by department and hierarchy. Operations: Illustrate manufacturing processes, breaking down components or ingredients hierarchically. Finance: Show budget allocations by division or project, with color-coded subcategories to highlight differences. Sales: Visualize sales structure, territory responsibilities, and targets with group field indicators showing performance status. IT: Visualize IT assets by category and availability with detail. Key features: Color-code cards using a legend. Adjust card (i.e. box) size, borders, and corners. Customize the appearance of the links connecting each card. Interact with cards to control levels and cross-filter visuals. Zoom and reposition the visual with ease. Try out the features of the Hierarchy Chart by MAQ Software today on the visual’s sample report. Learn more about our Power BI custom visuals on our website. Power BI JSON Report Theme Generator by BIBB https://powerbithemegenerator.bibb.pro/ BIBB recently released its take on the Power BI Theme Generator, aiming to streamline the theming process for users. This new tool offers a user-friendly interface, allowing for an easy selection of colours and generation of JSON themes. Within BIBB's generator, users can choose colours in various ways, from manual selection to trending combinations and even importing from images or external sources like Coolors. Synapse Data Warehouse Data Warehouse Utilization Reporting in Fabric Metrics app We are excited to announce that Compute utilization reporting for Data warehouse and SQL Endpoint are now available in Public Preview for Microsoft Fabric across all regions! In the capacity-based SaaS world of Microsoft Fabric, customers can purchase a Fabric SKU and get entitled to a set of Capacity Units (CU). Various workloads, including data warehouse, consume capacity units based on usage. Fabric metrics app provides visibility into capacity usage for all Fabric workloads, including data warehouse in one place. It is used by Capacity or SQL warehouse admins to identify CU usage trends across warehouse items within a capacity, monitor overload information, and understand the cost of running operations which leads to informed capacity sizing decisions. For more information, see the detailed blog Data Warehouse Utilization Reporting in Fabric Capacity Metrics App Column-Level security is now available on Data Warehouse and SQL Endpoint We are excited to announce that Column-Level security is now available on Data Warehouse and SQL Endpoint. Column-level security simplifies the design and coding of security in your application, allowing you to restrict column access to protect sensitive data. For example, ensuring that specific users can access only certain columns of a table pertinent to their department. The access restriction logic is located in the database tier rather than away from the data in another application tier. The database applies the access restrictions every time data access is attempted from any tier. This restriction makes your security more reliable and robust by reducing the surface area of your overall security system. In addition, column-level security also eliminates the need for introducing views to filter out columns for imposing access restrictions on the users. You can implement column-level security with the GRANT T-SQL statement. Only Azure Active Directory authentication is supported. Data Warehouse Row Level Security We are thrilled to announce that Row-Level security is now available in Fabric Warehouse and SQL Endpoint. Row-Level Security enables you to use group membership or execution context to control access to rows in a database table. Row-Level Security (RLS) simplifies the design and coding of security in your application. RLS helps you implement restrictions on data row access. For example, you can ensure that workers access only those data rows that are pertinent to their department. Another example is to restrict customers' data access to only the data relevant to their company. The access restriction logic is located in the database tier rather than away from the data in another application tier. The database system applies the access restrictions every time that data access is attempted from any tier. This makes your security system more reliable and robust by reducing the surface area of your security system. Implement RLS by using the CREATE SECURITY POLICY Transact-SQL statement, and predicates created as inline table-valued functions. SQL Projects support for Fabric Data Warehouse We are excited to announce the Microsoft Fabric Data Warehouse as a supported target platform in the SQL Database Projects extension available inside of Azure Data Studio! SQL Database Projects for Azure Data Studio provides a way to design, edit, and publish schemas for SQL databases from a source controlled project. For a complete development workflow, build and deploy your database projects in CI/CD pipelines, such as GitHub Actions or Azure DevOps. A SQL project is a local representation of SQL objects that comprise the schema for a single database, such as tables, stored procedures, or functions. Use the SQL Database Projects to extract and publish warehouse schemas directly from and to Fabric Data Warehouse. Other compatible databases include SQL Server, Azure SQL Database, Azure SQL Managed Instance, and Azure Synapse SQL (serverless and dedicated). Get started today with the insiders build of ADS. Deployment Pipelines In today’s world, analytics is a vital part of decision making in almost every organization. Fabric's deployment pipelines tool provides BI creators with a production environment where they can collaborate to manage the lifecycle of organizational content. Deployment pipelines enable creators to develop and test content in the service before it reaches the users. Supported content types include reports, paginated reports, dashboards, datasets, dataflows, and now warehouses! The deployment process lets you clone content from one stage in the deployment pipeline to another, typically from development to test, and from test to production. The connections between the copied items are kept during the copy process as well. In addition, Fabric applies the configured deployment rules to the updated content in the target stage. You can also deploy content programmatically, using the deployment pipelines REST APIs. You can learn more about this process in Automate your deployment pipeline using APIs and DevOps. Default Dataset Improvements We have released some new capabilities to enhance the default dataset experience: Turn off automatically adding new objects to the default dataset – navigate to the Warehouse artifact settings to toggle this on/off Updates to the manage default dataset dialog in the model view Filter by schema Filter by object (table/view) We’ve heard your feedback about the ability to turn off the default dataset. Thank you for the feedback, we’re planning on delivering this feature in the coming months! Data Engineering VS Code Integration Introducing Notebook File System support in Synapse VS Code extension for Data Engineering and Data Science workload in Microsoft Fabric The Synapse VS Code extension empowers users to develop their notebook artifacts directly within the VS Code environment. Once users download the .IPYNB file to their local desktop, they gain the flexibility to edit, debug, and execute their notebook code locally. Furthermore, they have the option to select the kernel provided by the extension to execute and debug their notebook code on the remote Fabric Spark compute. Python developers commonly structure reusable functions as modules in the form of .py files, a well-established software engineering best practice. Now, with the incorporation of the Notebook File System within the VS Code Synapse extension, developers can apply this principle to their Fabric notebook development. In addition to running and debugging notebook code, developers can also execute and debug code from imported modules. This integration enhances code modularity and facilitates efficient development workflows. When a user clicks "Open Notebook Folder," the VS Code Synapse extension not only opens the selected .IPYNB file but also downloads all associated files, including .PY modules, from the notebook file system to the local VS Code environment. This feature ensures that the entire set of necessary files and modules is readily available for seamless development and collaboration within the VS Code interface. In this example, there is a .PY module named US2Europe.py which contains a function to covert the datetime format from US fashion to Europe’s one. To invoke this function from the notebook side and execute it on the worker node, you need to import the module containing the function and apply into the data frame To synchronize changes between the local environment and the remote workspace in the VS Code Synapse extension, users can take advantage of two key actions: Publish Resource Folder: Clicking the "Publish Resource Folder" button enables users to upload their local changes to the remote workspace. This ensures that any modifications made locally are reflected in the remote environment. Update Resource Folder: The "Update Resource Folder" button allows users to pull changes from the remote workspace into their local VS Code environment. This ensures that any updates made by collaborators or on the remote side are mirrored locally. In the event that a file has been deleted in the remote workspace but still exists in the local environment, the Synapse extension will automatically handle this situation as follows: The deleted file will be moved to a backup folder named "__backup" under the root directory of the building folder. This approach preserves a copy of the deleted file for reference, ensuring that no data is lost during the synchronization process. These actions provide users with an efficient and robust means to manage the synchronization of their files and code between local and remote environments while maintaining data integrity. Import Notebook in Workspace view We are excited to announce that we now have the “Import Notebook” entry on the Workspace -> New -> Import item! You can easily import one or more files (support .ipynb, .py, .sql, .scala, .r file format) as Fabric Notebook items in the target workspace. Mssparkutils new API for fast data copy We now support a new method in mssparkutils that can enable large volume of data move/copy much faster, which is: Mssparkutils.fs.fastcp() You can use mssparkutils.fs.help("fastcp") to check the detailed usage. According to our benchmark testing, we achieved ~7x to ~180x speed up (varies as different source type, data size and regions) compares to the traditional file system copy method mssparkutils.fs.cp(). As the below example shows, it takes ~5s to copy a 10G file from ADLS Gen2 account to Fabric Lakehouse. Support diff versions when saving notebook We now support viewing and comparing the differences between 2 versions of the same notebook when there are saving conflicts. This is particularly useful when more than one users are working on a same notebook instance via different clients, like: Publish notebook from Fabric VSCode extension Some users are using Manual save mode while others are collaborating Collaborating on notebook with unstable network With the diff function the users in collaboration session can get notified and compare the changes from other user’s edits, and they can choose a version to keep, or save a copy of any version, to easily handle the save conflicts without losing the code accidently. Notebook Sharing with Execute-only mode Fabric Notebook now supports checking the “Run” operation separately when sharing a notebook, if you just selected the “Run” operation, the recipient would see a “Execution-only” notebook after opening the item. With the “Execution-only” mode you can easily spread your notebook without worrying about someone messing up the contents, while the recipients can still run the cells. Notebook resources .whl file support We now support uploading the .whl files in the Notebook Resources explorer, you can put your own compiled libs here, use Drag & Drop code snippet to install them in the session and import the libraries in code conveniently. Real-time Analytics OneLake shortcut to delta tables from KQL DB Now you can create a shortcut from KQL DB to delta tables in OneLake, allowing in-place data queries. With this enhancement, you can effortlessly query delta tables in your Lakehouse or Warehouse directly from KQL DB. Learn more. Real-Time Data Sharing: Introducing database shortcuts in Real-Time Analytics A database shortcut in Real-Time Analytics is an embedded reference within a KQL database to a source database in Azure Data Explorer allowing in-place data sharing. The behaviour exhibited by the database shortcut is similar to that of a follower database. If you are a data provider hosting data in Azure Data Explorer and want to share this data with consumers in Microsoft Fabric, you can create a database shortcut. Database shortcut enables sharing in real-time, with no need to build or maintain a data pipeline. All database changes, including schema and data, on the provider side are instantly available on the consumer side. You can then consume data in Microsoft Fabric by using a KQL Queryset, PBI, Notebooks etc. easily . While today, you can share data from an Azure Data Explorer database with a KQL Database, soon you will also be able to share data from a KQL Database with other KQL Database(s) in Microsoft Fabric. Check out the announcement blog to learn more: Real-Time Data Sharing in Microsoft Fabric blog Sample Gallery – Explore what you can achieve in Real-Time analytics Real-Time analytics now offers a comprehensive sample gallery with multiple datasets allowing you to explore, learn and get started quickly. Selecting a sample in Real-Time Analytics automatically creates a KQL Database with data and prewritten KQL Queryset from any of the 6 sample datasets. This will let users to get started with KQL Database and KQL Queryset without the hassle of creating database, finding the right real time dataset, ingesting data, and writing queries. Each sample lets you explore different streaming and real time scenarios. The samples include data and queries for Stock analytics, Weather analytics, IOT analytics, Log analytics, Metrics analytics and Automotive operations analytics. Each dataset is accompanied by a KQL Queryset which includes sample queries introducing you to KQL capabilities such as. aggregations, search , transformations using update policies, user defined functions, geospatial and time-series analytics. Model and Query data as graphs using Kusto Query Language (KQL) We are excited to share with you a new public preview feature of Kusto Query Language (KQL) that allows you to model and query data as graphs. Graphs are useful for representing complex and dynamic data that involve many-to-many, hierarchical, or networked relationships, such as social networks, recommendation systems, connected assets, or knowledge graphs. By using KQL with graph semantics, you get the speed and scale of KQL queries with the flexibility and expressiveness of graphs. You can query graphs using KQL graph semantics, which has a simple and intuitive syntax that works well with the existing KQL features. You can also mix graph queries with other KQL features, such as time-based, location-based, and machine-learning queries, to do more advanced and powerful data analysis. For example, you can use time-aware graphs to examine how the graph evolves over time, or use geospatial queries to analyze the spatial distribution or proximity of nodes and edges. To learn more about graph semantics in KQL, check out the following resources: - Introduction to graph semantics in KQL: https://aka.ms/kusto/graph-overview - Graph operators and functions reference: https://aka.ms/kusto/graph-operators We hope you enjoy this new feature and find it useful for your data analysis needs. As always, we welcome your feedback and suggestions on how to improve KQL and its graph semantics extension. Happy querying! Easily connect to KQL Database from Power BI desktop Power BI desktop just released new ways to easily connect to a KQL database. There are two ways to easily locate your KQL Database: Open the Get Data dialog and select the Microsoft Fabric menu item. Select KQL Databases from the list: Open the OneLake data hub dialog from the main ribbon. Select KQL Databases from the list: Eventstream now supports AMQP format connection string for data ingestion AMQP, stands for Advanced Message Queuing Protocol, is a protocol that supports a wide range of messaging patterns. It's a widely used communication protocol that facilitates efficient and reliable communication among IoT devices and systems. Now in Eventstream, you can create a Custom App source or destination and select AMQP format connection string for ingesting data into Fabric or consuming data from Fabric. Eventstream supports data ingestion from Azure IoT Hub Azure IoT Hub is a cloud-hosted solution that provides secure communication channels for sending and receiving data from IoT devices. With Eventstream, you can easily stream your Azure IoT Hub data into Fabric and perform real-time processing before storing it in Kusto Database or Lakehouse. To add an Azure IoT Hub source, follow these three simple steps: Choose "Azure IoT Hub" as your eventstream source. Give this source a name and enter your Azure IoT Hub details. You may need to enter the share access key from your Azure IoT Hub to establish the cloud connection. On the canvas, select the IoT source and click "Data Preview" to get a preview of the incoming data from your Azure IoT Hub. Improved Eventstream Creation Speed In response to user feedback, we've made significant improvements to the Eventstream creation, enabling you to create an Eventstream item in seconds. Setting up your Eventstream is now faster and smoother than ever before. These improvements save you valuable time and allow you to ingest and transform your data streams more efficiently. Data Factory Data Pipeline The Outlook email & Teams activities in Data Factory Pipelines can now be used in collaborative shared pipeline New pipeline activities are now available as well! If you’ve used the Custom activity type in ADF pipelines in the past, you’ll be very familiar with the new Azure Batch activity in Fabric data pipelines. This will allow you to execute custom executables from Azure Batch for powerful pipeline workflows. We’ve also enabled Azure Functions and KQL scripts as activities in Fabric pipelines. We've updated the Pipeline output monitoring view to focus on simplification, easily representing pipeline run status with customizable output layouts Pipeline lineage now includes lineage for Connections Dataflow Gen2 New Service Principal authentication kind Service Principal has been added as an authentication type for a set of data sources that can be used in Dataset, Dataflow, Dataflow Gen2 and Datamart. Azure service principal is a security identity that is application based and can be assigned permissions to access your data sources. Service principals are used to safely connect to data, without a user identity. Supported data sources include: Azure Synapse analytics Azure SQL database Azure data Lake gen 2 Azure data lake Azure blob storage Web Dataverse SharePoint online Note: service principal is not supported on the on-premises data gateway and virtual network data gateway. Support is planned by the end of 2023. Read more about Service Principal support from the official announcement post. Bug fixes and reliability improvements We continue to listen to your feedback and we’re actively working and making strategic fixes and quality improvements to our service. The following is a short list of the most impactful fixes and improvements delivered in the past couple of weeks. Revert changes for the limit of 50 queries per dataflow. In our previous monthly update we implemented a limit to the number of queries that a dataflow could have. We reverted this limit until further notice due to a number of undesired experiences created, but we will reintroduce this limit again in the near future. If you have dataflows with more than 50 queries currently, we encourage you to start splitting them into multiple dataflows with lower number of queries, as this will give you a head-start when the limit is reintroduced in the future. This documentation article describes several best practices when developing complex dataflows that you may find useful: Best practices for designing and developing complex dataflows - Power Query | Microsoft Learn Multiple fixes and improvements to common error messages and scenarios. Thanks to multiple customers who have reached out to us through our support system and our Community Forum, we’ve been able to address several common issues and improve the error messages that were quite frequent. Some of the error that have been fixed and/or modified to provide more descriptive information are: “The current row is too large to write.” “We cannot convert the value null to type Table.” “Dataflow refresh transaction failed with status Failed” Errors with Dataflow connector when creating linked entities. This issue has been fixed. We want to emphasize our commitment to our customers on improving the product and take your feedback into account in our decision making. We encourage you to engage directly with us through our support channel or through the Community forums. Community New learning path: Implement a Lakehouse with Microsoft Fabric To learn more and get started, check out the Implement a Lakehouse with Microsoft Fabric learning path. That is all for this month! Please continue sending us your feedback and as always, keep voting on Ideas to help us determine what to build next. We are looking forward to hearing from you!
Solved Fabric Community posts are now available in the Fabric Help Pane
The Fabric Help Pane provides Fabric users with a fast and efficient way to access self-help content, allowing them to resolve issues independently or direct them to create a support ticket if further help is required. Following its launch last year, we've heard your feedback to improve the relevance of our self-help resources. We are pleased to announce that now you can find solved posts from Fabric Community discussions relevant to your search in the Fabric Help Pane. To navigate to the help pane, start by clicking on “?” on the top right corner of the screen. Once you are on the help pane, you can search for any issues for which you are trying to find resolution. The search results will return relevant content from both Microsoft Fabric Docs and Community. _Community_posts_are_now_available_in_the_Fabric_Help_Pane If you are only interested in Community posts search results, you can also user the filter and select “Forum Topics” If you have any feedback on the self-content you would like to see in the help pane, please leave us a comment in the Join Discussion section below.
Fabric April 2025 Feature Summary
Welcome to the Fabric April 2025 Feature Summary! This update brings exciting advancements across various workloads, including Low-code AI tools to accelerate productivity in notebooks (Preview), session Scoped distributed #temp table in Fabric Data Warehouse (Generally Available) and the Migration assistant for Fabric Data Warehouse (Preview) to simplify your migration experience. Contents Community & Events Get certified in Fabric – for FREE. Free live learning sessions for Data Engineers General Fabric Copilot and AI Capabilities available on all paid SKUs Data Science Low-code AI tools to accelerate productivity in notebooks (Preview) Low-code AI capabilities in Data Wrangler (Preview) Data Warehouse ALTER Table Drop Column and sp_rename column support in Fabric Warehouse (Generally Available) Session Scoped distributed #temp table in Fabric Data Warehouse (Generally Available) Migration assistant for Fabric Data Warehouse (Preview) OPENROWSET function (Generally Available) BULK INSERT statement (Generally Available) Real-Time Intelligence Fabric special for Kusto Detective Agency: solve the Digibus real-time crisis Azure monitor data sources are now fully integrated with KQL Queryset Improvements to Data Exploration (low-code) experience Eventhouse system and KQL Database overview: in-item monitoring enhancements Eventstream's Real-time Weather Connector SQL database in Fabric New regions supported Backup billing Performance dashboard Terraform support, Rest API & CLI support Integrations Graph database support Data Factory Mirroring Mirroring for Snowflake protected by a firewall(Preview) Closing https://youtu.be/shz68BKiibg?si=iPbDBTTkMu6E59WB Community & Events Get certified in Fabric – for FREE. As part of the Microsoft AI Skills Fest, Microsoft is celebrating 50 years of innovation by giving away 50,000 FREE Microsoft Certification exam vouchers in weekly prize drawings. Enter the sweepstakes now to have the most chances to win a free exam voucher for DP-600 or DP-700. Free live learning sessions for Data Engineers Whether you're new to Microsoft Fabric or building on your existing skills, these sessions, hosted by Microsoft Fabric experts, give you the knowledge and confidence to get certified and take your data engineering career to the next level. Register now - live sessions in English start April 30 th . Available on-demand in Spanish and Portuguese. General Fabric Copilot and AI Capabilities available on all paid SKUs We are thrilled to announce a major update in Microsoft Fabric! Starting today, the SKU requirement for Copilot and AI features will be lowered to F2, making it much more accessible for you to explore, test, and utilize Fabric AI capabilities. This exciting change grants you full access to Copilot in Fabric, Fabric data agents, and Fabric AI Functions—all designed to enhance productivity, uncover insights swiftly, and seamlessly enrich your data. With this update, more teams will have the opportunity to experiment with AI-driven workflows within their existing capacity. It's worth noting that while the smallest SKUs provide full feature access, they will support a more limited number of AI requests due to their smaller capacity size. Nonetheless, this will allow more users to experience the benefits of AI capabilities and improve their workflows. We look forward to seeing the creative ways your team will utilize these AI capabilities to boost your projects and productivity! Data Science Low-code AI tools to accelerate productivity in notebooks (Preview) Fabric notebooks allow you to accelerate your productivity with native AI capabilities like Copilot and AI functions. A new notebook tab devoted to AI and ML tools now provides low-code shortcuts for transforming data with Data Wrangler, training custom models with AutoML, chatting with Copilot, and more. Among the updates is a low-code interface to apply AI functions for seamless LLM-powered data enrichment. Just select one of the functions, choose an input pandas or Spark DataFrame and a target column to transform, and fill in any required parameters. Fabric will produce the code for you. To learn more, refer to the Transform and enrich data seamlessly with AI functions documentation. Low-code AI capabilities in Data Wrangler (Preview) All Fabric notebook users have access to Data Wrangler, a low-code tool with an immersive interface for exploring and transforming pandas or Spark DataFrames. Data Wrangler provides a library of common data-cleaning operations that you can browse and apply seamlessly getting real-time previews and generating reusable code. We have new AI-powered capabilities coming to Data Wrangler later this month: Automated suggestions with rule-based AI: A new set of automated suggestions will analyze your data and use rule-based AI from the Microsoft PROSE team to highlight the most relevant Data Wrangler operations for you. Convert natural language to code with Copilot: Need an operation that you don’t see in Data Wrangler? You can now use Copilot to generate custom code. As with any Data Wrangler operation, you’ll get a preview before applying or discarding it. Use AI to translate custom code from pandas to PySpark: Data Wrangler automatically converts Spark DataFrames to pandas for performance reasons, then translates your applied code back to PySpark when you export it. With GenAI in Data Wrangler, custom code operations will also be translated to PySpark—whether you type them in yourself or generate them with Copilot. Data Warehouse ALTER Table Drop Column and sp_rename column support in Fabric Warehouse (Generally Available) There are two powerful new features in Fabric Warehouse that we are happy to introduce: ALTER TABLE DROP COLUMN and SP_RENAME COLUMN. ALTER TABLE DROP COLUMN effortlessly removes unnecessary columns to streamline storage, boost performance, and improve query efficiency. Cloning a table as of a point in time & time travel to a point in time that is before the table was dropped is not supported. Dropping columns from Lakehouse tables is not a supported scenario. SP_RENAME COLUMN easily renames columns without downtime, making schema adjustments faster and reducing the risk of errors.Columns and Tables are not renamable in Lakehouse. These new features make it easier to maintain a clean and efficient data model, allowing your team to quickly adapt to evolving business needs with minimal disruption. Check sp_rename Microsoft learn and ALTER TABLE (Transact-SQL) - SQL Server for additional details and syntax. Session Scoped distributed #temp table in Fabric Data Warehouse (Generally Available) Are you unable to manage intermediate query results efficiently in your batch jobs? Fabric Data Warehouse users can now create session-scoped #temp tables to handle these results seamlessly. These temp tables can be backed by either Parquet (distributed) or mdf (non-distributed), offering flexible options to cater to different needs. Users can create two types of #temp tables: Non-Distributed Temp Tables (mdf-backed) - These are created using syntax like user tables in Fabric DW, with the key difference being the need to prefix the table name with ‘#’. CREATE TABLE #table_name ( Col1 data_type1, Col2 data_type2 ); Distributed Temp Tables (parquet-backed) - These tables are distributed and created using the following syntax: CREATE TABLE #table_name ( Col1 data_type1, Col2 data_type2 ) WITH (DISTRIBUTION=ROUND_ROBIN); Note: data_type1 and data_type2 are placeholders for the supported data types in Fabric Data Warehouse Data types. For additional details on why we offer two types of temp tables, the scenarios they support, and their limitations, refer to the Session-scoped distributed #temp tables in Fabric Data Warehouse documentation. Start leveraging session-scoped temp tables in Fabric Warehouse to streamline your data processing tasks and enhance your workflow efficiency. Happy querying! Migration assistant for Fabric Data Warehouse (Preview) The Migration Assistant for Fabric Data Warehouse is now in preview. The migration experience is built natively into Fabric and enables Azure Synapse Analytics (Data Warehouse) customers to transition seamlessly to Microsoft Fabric. This new DW migration experience allows users to easily migrate both metadata and data from the source database, automatically converting the source schema to Fabric Data Warehouse, helping with data migration, and providing AI powered assistance. With integrated assessment tools and guided support, this capability simplifies migration, enabling customers to leverage Fabric’s capabilities without the complexity of traditional migrations. The Migration Assistant for Fabric Warehouse streamlines the migration process into four steps: Metadata migration Problem resolution Data copying Connection rerouting Each of these steps are explored in detail in this Migration Assistant for Fabric Data Warehouse (Preview) blog post. For a more comprehensive guide, you can also review the migration assistant how-to article for step-by-step instructions and the Fabric Migration Assistant documentation for more in-depth information. OPENROWSET function (Generally Available) The OPENROWSET function in Fabric Data Warehouse and Fabric SQL endpoint are generally available. The OPENROWSET function enables you to seamlessly read Parquet and CSV files stored in Azure Data Lake Storage and Azure Blob Storage, as it is shown in the following example: SELECT TOP 10 * FROM OPENROWSET( BULK 'https://<storage>.blob.core.windows.net/container/file.parquet' ) With OPENROWSET, you can easily browse files before loading them into the Fabric Data Warehouse, allowing you to inspect the schema before creating the target table. This function provides several valuable features that significantly enhance the data ingestion experience: Referencing custom folder structures – the OPENROWSET function can reference URI patterns by using * (wildcard) and /** (recursive child wildcard) that can match multiple source files that match the same pattern or return all files that are recursively placed under the URI. Reading partitioned data sets – the OPENROWSET function can retrieve partition values from the folder names, which is crucial if you are reading data from hive-style partition structures and return these values in the result set. Reading Parquet complex types – the OPENROWSET function supports complex types such as struct, array, and map, returning them as JSON text for easier manipulation and analysis. Customizing result set scheme – the OPENROWSET function allows you to map the result set columns to the source columns and define the optimal column types for all columns using the WITH clause, providing flexibility in how data is presented and utilized. OPENROWSET supports most of the options available in SQL Server, Azure SQL, and Synapse, facilitating seamless migration and code reuse between these platforms. Ingesting data with CTAS or INSERT SELECT statements – the OPENROWSET function enables you to ingest data using Create Table As Select (CTAS) or INSERT SELECT statements by using the OPENROWSET as a source and allowing you to modify the values from the source values at ingestion time. This is crucial in scenario where you need to modify source data that is not in expected format. The OPENROWSET function will significantly improve your data ingestion experience by enabling you to browse files, transform data during ingestion, and facilitate easier migrations from Synapse, SQL Server, and Azure SQL Database to Fabric Data Warehouse. This powerful functionality ensures that you can handle complex data types, manage partitioned data efficiently, and customize your result set schema to meet your specific requirements, all while maintaining compatibility with existing SQL options. BULK INSERT statement (Generally Available) The BULK INSERT statement in Fabric Data Warehouse is generally available, it enables you to ingest data into a table from the specified file path: BULK INSERT table_name FROM file_url_path The BULK INSERT statement is very similar to the COPY INTO statement and enables you to load data from external storage. The key value of BULK INSERT is that it supports traditional SQL Server and Azure SQL syntax, thus facilitating an easy migration of SQL Server databases to the Fabric Data Warehouse without the need for code changes. Additionally, BULK INSERT supports several traditional options used in SQL Server, such as text/xml format files that are used by bcp tool and importing non-Unicode files with the custom code pages. This compatibility ensures that you can migrate your databases to the Fabric Data Warehouse with minimal code changes to your ingestion code while retaining your existing ingestion logic without altering the input files. By leveraging the BULK INSERT statement, you can maintain your data ingestion workflows and schemas, ensuring a seamless transition to the Fabric Data Warehouse. This feature not only preserves the integrity and structure of the data but also enhances the efficiency of the migration process, reducing the potential for errors and downtime. As a result, businesses can continue to operate smoothly while taking advantage of the advanced capabilities and scalability offered by the Fabric Data Warehouse. Real-Time Intelligence OpenAI plugins for Eventhouse Two new OpenAI plugins are available to generate embeddings and leverage the power of OpenAI models within the Eventhouse context. You can use the plugins to build Retrieval Augmented Graph (RAG) applications or augment your data analysis with OpenAI models. ai_embed_text : Integrates OpenAI embedding models to generate embeddings within KQL. ai_chat_completion: Leverages the power of ChatGPT and other OpenAI models to augment data analysis within the Eventhouse context. Fabric special for Kusto Detective Agency: solve the Digibus real-time crisis This challenge is specifically optimized for onboarding to the RTI platform. You can use it to enhance onboarding for customers and create CTF (Capture the Flag) tournaments to learn with hands-on experience. During this quest, you'll learn KQL and utilize powerful tools including: Eventstream: Process and transform real-time data Eventhouse: Store and query massive datasets Real-Time Dashboard: Visualize critical metrics Activator: Trigger automated responses Put your detective skills to work helping the Digibus Digitown transit company solve its crisis. Follow the clues, analyze the data, and uncover the solution with the power of Real-Time Intelligence. Lucky finishers will win prizes! Don't miss this opportunity to enhance your RTI skills while solving an engaging mystery. Get started with Kusto Detective Agency. Azure monitor data sources are now fully integrated with KQL Queryset KQL Queryset has always supported cross-service queries, but now we’re making Azure data sources even more accessible. Application Insights (AI) and Log Analytics (LA) are now first-class citizens, just like Eventhouse and ADX, providing a more seamless and intuitive experience. With this update, you can: Run cross-service queries between Log Analytics, Application Insights, Eventhouses, and Azure Data Explorer (ADX) native clusters, all connected to the same KQL Queryset. Directly query your Log Analytics workspace or Application Insights resources from KQL Queryset. This makes it easier than ever to explore and analyze data across services without extra configuration. Improvements to Data Exploration (low-code) experience We’re continuing to improve the low-code data exploration experience, making it even easier to analyze data in Real-Time Dashboard tiles and KQL Database tables (in Eventhouse and Real-Time Hub). Here’s what’s new: Hierarchal Columns Pane – easy to use summary of key data characteristics to guide exploration and data manipulation includes: List of participating columns Data types Statistical info (avg, min, max, cardinality etc.) Focus mode – work more efficiently by focusing on either the data visualization or results grid. Right-click actions on grid – quickly copy, export, and perform actions directly from the data grid. _Summary Filter builder with OR conditions –create WHERE statements with more flexible filtering options. Datetime picker control – select exact or relative dates effortlessly when filtering datetime columns. Learn more about data exploration experience here Explore data in Real-Time Dashboard tiles. Eventhouse system and KQL Database overview: in-item monitoring enhancements As part of our ongoing commitment to improving system visibility, performance monitoring, and user experience, we've introduced several enhancements across both the Eventhouse System Overview and KQL Database in-item monitoring pages. Eventhouse system overview enhancements New - Eventhouse ingested rows overtime One of the improvements to the Eventhouse System Overview is the ability to view ingested row metrics directly within the interface. Users can now see the number of rows ingested into each database, offering immediate visibility into data volume and ingestion activity. The feature includes time-based filtering, enabling users to analyze ingestion trends over specific periods. New tab for Top ingested databases Another significant update is the introduction of a dedicated tab for Top Ingested Databases. This new section offers a detailed view of ingestion metrics for each database, including the total number of ingested rows and any ingestion failures (currently only partial failures are reported). The addition of time-based filtering makes it easier to identify patterns and anomalies across specific timeframes. Top 10 ingested databases- improvements The new multi-tab interface displays both the most queried databases and the top ingested databases, improving navigability and offering a clearer view of different performance indicators. Users can also benefit from added cache miss rates that were added to the queries database metrics. Eventhouse details moved to the menu bar To streamline access and maintain design consistency, Eventhouse details have been relocated to the main menu bar. Selecting this option opens a side panel that mirrors the familiar layout used in the Database and Table overview panes. To learn more, refer to the Manage and monitor an Eventhouse documentation. Eventstream's Real-time Weather Connector Last month, we introduced several powerful new connectors for Eventstream in Fabric Real-Time Intelligence. Now, we’re taking it a step further with a hands-on video that shows you how to use one of them: the Real-time Weather connector. In the demo video, we walk through how to easily add the Weather connector to an Eventstream and start streaming live weather data—like temperature, humidity, and wind speed—into Fabric. Whether you’re building real-time dashboards, alert systems, or enriching other streams, weather data adds valuable real-world context to your applications. What you’ll see in the video: A quick overview of the new Eventstream connectors and where to find them Step-by-step demo on adding and configuring the Weather connector Live preview of streaming weather data in action This video is a great starting point if you’re exploring how live data sources can enrich your streaming solutions in Fabric. The Weather connector is especially useful for industries like logistics, agriculture, and retail—anywhere environmental conditions influence operations. Watch the demo to see how easy it is to add real-time weather feed into an Eventstream. Can’t find your data sources, let us know! Send us an email at [email protected] or fill out our survey. h2> Databases SQL database in Fabric We have several new advances to share in SQL Database within Fabric. Continuous innovation is at the heart of our development, outlined are several key enhancements. New regions supported Australia Southeast Italy North Japan East Poland Central WestUS3 are new regions that supports SQL databases in Fabric workloads Backup billing SQL database in Microsoft Fabric offers automatic backups from the moment of database creation, ensuring data protection and recovery. The system makes full backups every week, differential backups every 12 hours and transaction log backups every 10 minutes, providing point-in-time restore capability up to 7 days. While compute and data storage are already included in the Fabric capacity billing model, starting April 1, 2025, backup storage will also be billed. Customers will only be billed for backup storage that exceeds the allocated database size. To learn more, check out the Automatic backups in SQL database in Microsoft Fabric documentation. Performance dashboard The SQL in Fabric dashboard now shows the lead blocking query to allow developers to quickly identify SQL queries that are blocking and impacting other queries thereby disrupting their operational workloads. Link to Video. Terraform support, Rest API & CLI support This capability enables customers to automate, scale, integrate, and govern their SQL databases within Microsoft Fabric, using a declarative approach with Terraform. HashiCorp Terraform, an open-source tool that offers a secure, predictable, and consistent method for deploying and managing infrastructure across multiple cloud environments. This functionality extends the capabilities of Fabric through Infrastructure-as-Code (IaC). To learn more about fabric-terraform-quickstart refer to the documentation. Be sure to check out the blog post Terraform support for Fabric GA for a more information. Integrations Fabric data pipelines will now support Fabric SQL database as a data source for Stored Procedure and Script activities, allowing users to just pick a database rather than having to enter connection information. Graph database support The query editor in SQL database in Fabric now have T-SQL support for graph databases. This feature enables the modeling of many-to-many relationships. You can create a graph database with nodes and edges and utilize the new MATCH clause to identify patterns and navigate through the graph. Learn more about how to Create a graph database and run some pattern matching queries using T-SQL from our documentation. Data Factory Mirroring Mirroring for Snowflake protected by a firewall (Preview) You now can mirror Snowflake protected by a firewall. Using either the VNet data gateway or the on-premises data gateway for mirroring is available. The data gateway facilitates secure connections to your source databases through a private endpoint or from a specific private network. Learn more about Mirroring for Snowflake from the Microsoft Fabric Mirrored Databases from Snowflake documentation. Closing We hope that you enjoy the update! Be sure to join the conversation in the Fabric Community and check out the Fabric documentation to get deeper into the technical details. As always, keep voting on Ideas to help us determine what to build next. We are looking forward to hearing from you!Fabric May 2025 Feature Summary
Today kicks off Microsoft Build and we have a lot of new features in store for you. Some highlights are the Fabric Roadmap tool, a way to get glimpse of that is coming soon to Fabric. Chat with your data, powerful AI capabilities that make Power BI even easier. Cosmos DB in Fabric, give you the power of Cosmos DB that's AI-ready. To get a taste of the Build excitement be sure to check out Arun Ulag's Arun Ulag's announcement blog and Kim Manis' announcement blog. Contents Events & Announcements New Fabric Roadmap tool Fabric Platform Additional REST APIs for Fabric Deployment pipelines New capabilities for Fabric Git integration Shortcut transformations (Preview) Data Engineering New regions supported in User Data Functions SPN support for User data functions SPN support for the Livy API Private libraries support for User data functions Data Science Copilot in Power BI now supports Fabric data agents Fabric Data Agent Integration with Microsoft Copilot Studio (Preview) Data Warehouse Warehouse Snapshots (Preview) Real-Time Intelligence Call of the Cyber Duty: a new season of Kusto Detective Agency begins Continuous Ingestion from Azure Storage to Eventhouse (Preview) Fabric Eventhouse now supports Eventstream Derived Streams in Direct Ingestion mode (Preview) Get Data in Fabric Eventhouse from Lakehouse using OneLake Catalog Eventhouse Accelerated OneLake Table Shortcuts (Generally Available) Databases Introducing Cosmos DB in Microsoft Fabric (Preview) Native change data capture (CDC) support in Copy Job (Preview) Semantic Model Refresh Activity (Generally Available) Copilot for Data pipeline - boost your productivity in understanding and updating pipeline with Copilot Mirroring Mirroring for SQL Server On-Premises (Preview) Mirroring for SQL Server 2025 (Preview) New features for Mirroring for Azure SQL Managed Instance Customize retention period for mirrored data Mirroring region expansion Mirroring for Azure PostgreSQL region expansion Fabric Mirroring for Azure Cosmos DB: public preview refresh live with new features Dataflow Gen2 Dataflow Gen2 (CI/CD) (Generally Available) Dataflow Gen2 Public APIs (Preview) Dataflow Gen2 parameterization (Preview) Lakehouse as an incremental refresh destination in Dataflow Gen2 (Preview) SharePoint files as a destination in Dataflow Gen2 (Preview) Natural language to custom column Community Power Designer - unleash your inner report wizard (Generally Available) Closing https://youtu.be/5qbIn80JrqY?si=BxhW9e_0Ck7QN-1M Events & Announcements New Fabric Roadmap tool We’ve heard from you that it’s critical to know when key Fabric features will land, especially those that directly impact your use cases or unblock your organization’s adoption. For example, if you're waiting on Private Link support for Workspaces due to internal security requirements, you need a clear view of when that capability is planned and when it becomes available. Until now, this information was spread across Release Plan documentation pages. Today, we’re making that experience better. The new Fabric Roadmap page brings it all together in one place, with a cleaner interface, real-time updates, and direct integration with the internal planning tool used by the Fabric team. Check it out at https://roadmap.fabric.microsoft.com and tell us what you think in the comments. Power BI Some of the highlights include Chat with your data, a revolutionary new way to use AI in PowerBI. And Translytical task flows, enabling users to automate action directly within the report—streamlining decision-making and operational follow-through. To learn about all of the latest updates to Power Bi head over to the Power BI May 2025 Feature Summary Fabric Platform Additional REST APIs for Fabric Deployment pipelines An additional batch of Fabric public APIs for Deployment pipelines have been released, following our initial release of Deploy APIs a few months ago. With this new release, the full list of available Fabric APIs now matches the APIs available in Power BI, excluding Admin APIs, which will be added later. This marks a significant milestone in our ongoing efforts to enhance the Fabric platform and provide our users with powerful tools to manage their deployment processes more efficiently. Overview of the new APIs The new APIs offer a range of functionalities that streamline the deployment process, making it easier for teams to manage their content across different environments: Pipeline management: Create, update, and delete deployment pipelines with the new APIs. Stage management: Get and update the deployment pipeline stages. Deployments management: List deployment pipeline operations and get details of specific deployment pipeline operations. Workspace assignment management: Assign and unassign workspaces to and from stages. Roles assignment management: List deployment pipeline role assignments and get or delete specific role assignments. Support for Service Principal (SPN) All fabric Deployment pipelines REST APIs are now having the support for Service Principal (SP). This allows for more secure and automated deployments, enabling teams to integrate Fabric into their existing DevOps workflows seamlessly. Getting started To start using the new Fabric public APIs for Deployment pipelines, you can refer to the Automate your deployment pipeline with Fabric APIs documentation. This provides comprehensive guides and examples to help you integrate these APIs into your deployment processes effectively. New capabilities for Fabric Git integration Service Principal (SPN) support for Azure DevOps A few weeks ago, we announced the capability to use Service Principal when working with Fabric Git API and your Git Provider is GitHub. Soon, we will support Azure DevOps as your Git Provider as well. Cross-Tenant support for Azure DevOps Previously, connecting your workspaces using your identity to an Azure DevOps repository required both Fabric and your Azure DevOps organization to reside within the same tenant. However, we're thrilled to announce that this limitation will soon be a thing of the past. With our upcoming update, you'll be able to connect to an Azure DevOps organization even if it belongs to a different tenant than your Fabric tenant. Shortcut transformations (Preview) The preview of shortcut transformations introduces the ability to transform data as it’s shortcut into Fabric including changing the data format into Delta tables or applying AI transformations to unstructured data—such as summarizing text, translating content, or classifying documents. Data Engineering New regions supported in User Data Functions After our preview launch, we have been working on increasing the number of regions supported for this feature. We have recently added the following 14 new regions where you can use this feature from: Australia Southeast Brazil South Canada Central Central India France Central Korea Central North Central US Norway East South Africa North South India UAE North UK West West Europe West US You can find the entire list of supported regions in this article: Fabric Region Availability. This article will be frequently updated to reflect the latest region support. SPN support for User data functions Fabric User data functions now support Service Principal Names (SPN) to run functions. This feature allows organizations to ensure compatibility with enterprise identity and access management systems. By using SPNs, it is possible to implement applications that can call a user data function without requiring user credentials. This aligns with the zero-trust security model, providing a secure way where user data functions are the glue between your application and your data in Fabric. To learn more, refer to the SPN support for user data functions documentation. SPN support for the Livy API The Fabric Livy API for Data Engineering now supports Service Principal Names (SPN) to submit and execute Spark code. This added SPN authentication method allows organizations to ensure compatibility with enterprise identity and access management systems. By using SPNs, it is possible to implement applications that can call a user data function without requiring user credentials. This aligns with the zero-trust security model, providing a secure way where user data functions are the glue between your application and your data in Fabric. To learn more, refer to the Create and run Spark Session jobs using the Livy API documentation. Private libraries support for User data functions A new feature has been introduced: Private libraries support for Fabric user data functions. These private libraries are code created by you or your organization. Data engineering can be challenging, especially with data quality and complex analytics. Private libraries help streamline work and enable proprietary code use within a team securely. Fabric User data functions now allow custom library uploads in .whl format, containing scripts or modules for internal business logic. This can improve developer productivity across your organization allowing you to reuse these libraries for automating various process across different teams or departments in your organization. To learn more refer to the documentation on How to manage libraries for your Fabric User Data Functions. Data Science Copilot in Power BI now supports Fabric data agents Fabric data agents can be used in the new chat with your data experience in Power BI to get answers to your questions and explore your data more effectively. This integration enables users in Copilot in Power BI to not only connect Power BI semantic models, but also to a wider range of data sources in OneLake—such as lakehouses, warehouses, and KQL databases—retrieving insights seamlessly through Fabric data agents. When you ask a question in the new full-screen Copilot in Power BI experience Copilot first searches for relevant Fabric data agents you have access to. If you have the necessary permissions, it uses those data agents to retrieve answers based on your access rights. This helps you discover content, ask questions, perform quick analyses, and refine insights—all without switching tools or leaving Copilot. You can also manually add a data agent to your Copilot session and chat with it directly from Copilot in Power BI, enabling seamless access to your OneLake data. Fabric Data Agent Integration with Microsoft Copilot Studio (Preview) Fabric data agent will be available in preview and can be added as an agent to your custom setup in Microsoft Copilot Studio. With this integration, your custom agent can access data stored in Microsoft OneLake—including lakehouses, warehouses, Power BI semantic models, and KQL databases—and retrieve insights seamlessly through the Fabric data agent. Once you add the Fabric data agent to your custom agent, you can publish your custom agent to various consumption channels, including Microsoft Teams and Microsoft 365 Copilot, and share it with specific users or your entire organization. When a user asks a question from the custom agent in any of these channels, the Fabric data agent is used to retrieve answers—provided the user has the necessary permissions. Responses are always scoped to the user’s access rights, making it easier to discover relevant content, perform quick analyses, and refine insights within the same channel. To extend functionality, you can define actions for your custom agent. Actions such as sending emails or initiating other tasks allow the agent to automate processes on behalf of users, helping streamline workflows and improve productivity without leaving the custom agent experience. Data Warehouse Warehouse Snapshots (Preview) Ensuring data consistency during ETL (Extract, Transform, Load) processes has long been a challenge for data engineers. We are pleased to announce the preview of Warehouse Snapshots, a new feature in Microsoft Fabric designed to offer a stable, read-only view of your data warehouse at a specific point in time. This capability facilitates uninterrupted analytics and reporting. A warehouse snapshot is a read-only representation of a data warehouse at a designated moment, retained for up to 30 days (until configurable retention is available). Warehouse snapshots can be seamlessly ‘rolled forward’ on demand, enabling consumers to connect to the same snapshot (or use a consistent warehouse connection string from third-party tools) to access a curated version of data. This ensures that data engineers can provide analytical users with a consistent dataset, even as real-time updates occur. Analysts can run SELECT queries based on the snapshot without any ETL interference. For more information on CRUD for warehouse snapshots and understanding their considerations and limitations, please refer to Warehouse Snapshot in Microsoft Fabric (Preview). https://youtu.be/cUGGrdpswLk?si=03UTJCbjvTD8WwjU Real-Time Intelligence Call of the Cyber Duty: a new season of Kusto Detective Agency begins Are you ready to put your sleuthing skills to the test? The Kusto Detective Agency is back - and this time, it’s bigger, bolder, and packed with adrenaline. Introducing ‘Call of the Cyber Duty’, a brand-new season of the Kusto Detective Agency challenge designed for the sharpest minds in data. Whether you're a seasoned Kusto veteran or a curious newcomer, this is your chance to dive into a thrilling online race where speed, smarts, and strategy collide. Challenge begins June 8, 2025 Register by June 7, 2025 Why should you care? Because this isn’t just a challenge - it’s a competition. And the stakes, monumental. $10,000 for 1st place Bragging rights across the Fabric community Team up or go solo - form a squad of up to six detectives or take on the mission alone. Who should join? If you're using Microsoft Fabric Real-Time Intelligence and working with Eventhouse, this is your moment. The challenge is built to stretch your KQL muscles, sharpen your investigative instincts, and connect you with a vibrant community of data detectives. How to get started: Watch the trailer: Kusto Detective Agency - Call of the Cyber Duty Visit https://detective.kusto.io to register Rally your team or fly solo Prepare for a season of puzzles, plots, and powerful insights. This is more than a game. It’s a celebration of what’s possible with Kusto and Microsoft Fabric. So, gear up, detectives—the cyber world needs you. Disclaimer: No Purchase Necessary. Must be 14+ to participate. Registration period closes on June 7th, 2025, end of day. Prizes are awarded as digital gift cards to the team leader. Continuous Ingestion from Azure Storage to Eventhouse (Preview) Get Data in Real-Time Intelligence Eventhouse offers a step-by-step process to guide you through importing or inspecting the incoming data, creating or editing the destination table schema, to exploration of the ingested result from multiple sources. One of the sources from which users can bring data into an Eventhouse table using Get Data wizard is Azure Storage, which allows users to ingest one or more blobs/files from the storage account. This capability is now being enhanced with the feature of continuous ingestion, where once the connection between the Azure Storage Account and Eventhouse has been established, any new blob/file uploaded to the storage account will automatically be ingested to the destination table. Continuous Ingestion from Azure Storage to Eventhouse, utilizes Azure Events in Fabric to listen to Azure Storage Account Events. Based on the subscribed events from Azure Events, Eventhouse pulls the corresponding newly created/renamed file from the connected Azure Storage. This simplifies the process of bringing data from your Azure Storage account as it is being generated and eliminates the need for creating and maintaining long complicated ETL pipelines. It also eliminates the need of defining time-based triggers for fetching new data from Azure storage and makes ingestion to Eventhouse near real-time. Continuous ingestion from Azure Storage to Eventhouse is now offered in preview in Microsoft Fabric. To learn more, refer to the Get data from Azure storage documentation. https://youtu.be/RlZnGloBvSA?si=IU5YbxGG8Fombygu Fabric Eventhouse now supports Eventstream Derived Streams in Direct Ingestion mode (Preview) The Eventstreams feature in Microsoft Fabric Real-Time Intelligence allows you to bring real-time events into Fabric, transform them, and then route them to various destinations such as Eventhouse, without writing any code (no-code). You can ingest data from an Eventstream to Eventhouse seamlessly either from Eventstream artifact or using Eventhouse Get Data Wizard. This capability is now being extended to support Eventstream Derived streams in direct ingestion mode. Derived stream is a specialized type of destination that you can create after adding stream operations, such as Filter or Manage Fields, to an Eventstream. The derived stream represents the transformed default stream following stream processing. You can route the derived stream to multiple destinations in Fabric and view the derived stream in the Real-Time hub. Direct ingestion from Derived stream allows you to ingest your event data directly into the Eventhouse without any processing. This can be configured from Eventstream, as well as from Eventhouse Get Data Wizard, including embedded Real-Time Hub in Eventhouse Get Data Wizard. Please refer to Get data from Eventstream to learn more and get started today. Get Data in Fabric Eventhouse from Lakehouse using OneLake Catalog OneLake catalog is the central hub for discovering and managing Fabric content. One of the artifacts that OneLake catalog enables the discovery of is Microsoft Fabric Lakehouse, which is a data architecture platform for storing, managing, and analyzing structured and unstructured data in a single location. Get Data in Eventhouse now embeds OneLake catalog which allows an easy discovery and navigation experience for ingesting data from Lakehouse to Eventhouse. Using OneLake catalog, you can easily look for a Lakehouse through multiple workspaces and identify the Lakehouse you recently used, your favorites or endorsed by your organization. Once you select Lakehouse from the embedded OneLake catalog in Eventhouse Get Data, you can select and ingest a file from the Lakehouse seamlessly, including the files within sub folders. To learn more, refer to the Get data from OneLake documentation and get started! Eventhouse Accelerated OneLake Table Shortcuts (Generally Available) Shortcuts are embedded references within OneLake that point to other files’ store locations without moving the original data. Previously, you could create a shortcut to OneLake delta tables using Eventhouse and query the data, but performance lagged direct ingestion in Eventhouse, as shortcut queries lacked the powerful indexing and caching capabilities of Eventhouse. Accelerated shortcuts are powered by query acceleration which indexes and caches data landing in OneLake on the fly, allowing customers to run performant queries on large volumes of data. Customers can use this capability to analyze real-time streams coming directly into Eventhouse and combine it with data landing in OneLake either coming from mirrored databases, Warehouses, Lakehouses or Spark. Customers can expect significant improvements by enabling this capability, in some cases up to 50x and beyond. How to enable Query Acceleration? You will now see an option to enable Acceleration while creating a new shortcut from Eventhouse. To learn more, refer to the Query acceleration for OneLake shortcuts - overview (preview) documentation. Databases Introducing Cosmos DB in Microsoft Fabric (Preview) Cosmos DB is now available in preview as a new addition to the databases workload in Microsoft Fabric. Cosmos DB in Fabric is easy to set up, with automatic scale and secure by default, enabling you to build AI applications with less overhead. You can store and retrieve semi-structured data within milliseconds, without having to tweak the database settings manually. Equipped with built-in vector indexing and AI-ready full-text, hybrid search capabilities of Cosmos DB, you can now seamlessly build GenAI applications. Your existing or new applications can instantly benefit from deep integration with Fabric OneLake, bringing you databases, analytics, data science, real-time intelligence, and Copilot-powered BI in one place, rather than assembling them individually. You can seamlessly join Cosmos DB data with any other data in OneLake, such as SQL DB, truly unifying your data estate. To get started, please join our preview program by filling in this opt-in form. For more information, refer to Announcing Cosmos DB (Preview). Data pipelines Native change data capture (CDC) support in Copy Job (Preview) Change Data Capture (CDC) in Copy Job is a powerful capability in Data Factory that enables efficient and automated replication of changed data including inserted, updated and deleted records from a source to a destination. This ensures your destination data stays up to date without manual effort, improving efficiency in data integration while reducing the load on your source system. With CDC in Copy Job, you can enjoy the following benefits: Zero Manual Intervention: Automatically captures incremental changes (inserts, updates, deletes) directly from the source. Automatic Replication: Keeps destination data continuously synchronized with source changes. Flexible Incremental Copy Options: Automatically detects CDC-enabled tables, allowing you to choose between CDC-based or watermark-based incremental copy at the table level. Optimized Performance: Processes only changed data, reducing processing time and minimizing load on the source. Learn more in the What is Copy job in Data Factory documentation. Semantic Model Refresh Activity (Generally Available) Semantic Model Refresh activity for data pipelines is now generally available! With this activity, you will be able to create connections to your Power BI semantic model datasets and refresh them from your data pipeline! To learn more, refer to the Semantic model refresh activity in Data Factory for Microsoft Fabric documentation. Copilot for Data pipeline - boost your productivity in understanding and updating pipeline with Copilot Maintaining complex Data pipelines in your ETL project is not easy work, especially when you need to understand complicated Data pipelines created by others or you need to update some configurations for a set of pipelines. Copilot for Data pipeline helps users quickly understand the purpose of a pipeline and the details of its activities. With this new release, it also allows users to update descriptions for pipelines and activities based on its summary. After updating, users can hover over any activity to see a simple explanation of its function. Copilot for Data pipeline empowers users to efficiently update the settings of multiple activities within seconds which is much faster than manual update that may take hours. For example, users can update the timeout of more than ten activities inside pipeline from 12 hours to 1 hour. To learn more, refer to the AI-powered development with Data pipeline documentation. Mirroring Mirroring for SQL Server On-Premises (Preview) Mirroring for SQL Server in Fabric for on-premises versions of SQL Server 2016-2022 is now in Preview! Mirroring in Fabric allows users to enjoy a highly integrated, end-to-end, and easy-to-use product that is designed to simplify your analytics needs. Built for openness and collaboration between Microsoft, and technology solutions that can read the open-source Delta Lake table format, Mirroring is a low-cost and low-latency turnkey solution that allows you to create a replica of your SQL Server data in OneLake which can be used for all your analytical needs. By leveraging Change Data Capture (CDC) technology available in SQL Server, mirroring service in Fabric uses on-premises data gateway (OPDG) to connect to SQL Server and read the initial snapshot as well as subsequent changes to data at the source. OPDG then pulls the data into OneLake and converts into an analytics-ready format in Fabric. To learn more, refer to the Mirroring for SQL Server in Microsoft Fabric (Preview) blog post. Mirroring for SQL Server 2025 (Preview) With the announcement of Microsoft SQL Server 2025 at Build, customers can also leverage mirroring from this version in Fabric. The overall user experience is like mirroring from other SQL Server versions and Azure SQL. Mirroring for SQL Server 2025 uses change feed instead of Change Data Capture and SQL Server keeps track and replicates the initial snapshot and changes to the landing zone in OneLake which is then converted to an analytics-ready format by the mirroring engine. To learn more, refer to the Mirroring for SQL Server in Microsoft Fabric (Preview) blog post. New features for Mirroring for Azure SQL Managed Instance We have made substantial updates to Mirroring for Azure SQL Managed Instance in Fabric. Based on user feedback, new features have been developed to address data replication needs: Mirror Azure SQL Managed Instance via private endpoint: VNet data gateway or on-premises data gateway can be used as a way to connect to your Azure SQL Managed Instance database for mirroring, removing the necessity of opening public access. The data gateway ensures secure connections to the source databases via private endpoint. Mirror tables without Primary Keys: We’ve relaxed the limitation to let you mirror tables even if they don’t have a primary key sometimes referred to as heap tables, offering increased flexibility. Support for expanded Data Definition Language (DDL): In addition to Alter/Drop/Rename tables/column, now you can Truncate tables in the source databases while mirroring is active. To learn more, refer to the Mirrored Databases from Azure SQL Managed Instance documentation. Customize retention period for mirrored data Mirroring in Fabric continuously replicates your existing data estate from various databases into OneLake in Delta Lake table format. To keep the mirrored data efficiently stored and always ready for analytics, mirroring automatically runs vacuum to remove old files no longer referenced by a Delta log. We now offer you the flexibility to customize the retention setting according to your requirements. For instance, you may choose a shorter retention period to reduce mirroring storage consumption or extend the retention period to utilize Delta’s time travel capabilities for analytics. Currently, this value can be set via API. To learn more, refer to the retention for mirrored data documentation. Mirroring region expansion Mirroring now supports all regions that are available for workloads in Microsoft Fabric. We have recently added a new region support for West US 3 to meet the growing customer demand. For detailed information about the Fabric regions that support mirroring, please refer to the supported regions documentation. Mirroring for Azure PostgreSQL region expansion Alongside the region expansion for all Mirroring in Fabric, Mirroring for Azure PostgreSQL will also expand region support from the initial 4 regions: Canada Central, West Central US, East Asia, and North Europe to all regions supported by Mirroring in Microsoft Fabric to ensure that customers have the best performance when replicating data from Azure PostgreSQL flexible server. To learn more, refer to the Simplify Your Data Strategy: Mirroring for Azure Database for PostgreSQL in Microsoft Fabric for Effortless Analytics on Transactional Data. Fabric Mirroring for Azure Cosmos DB: public preview refresh live with new features We’re thrilled to announce the latest refresh of Fabric Mirroring for Azure Cosmos DB! This update introduces key enhancements like Microsoft Entra ID authentication, container selection, support for special characters in column names, and even vector search compatibility for AI workloads. With features like auto schema inference and full CRUD API support, this release makes it easier than ever to build secure, scalable, and real-time analytics pipelines with Cosmos DB data in OneLake. To learn more, refer to the Fabric Mirroring for Azure Cosmos DB with new features blog post. Dataflow Gen2 Dataflow Gen2 (CI/CD) (Generally Available) With this new set of features now generally available, you can seamlessly integrate your Dataflow with your existing CI/CD pipelines and version control of your workspace in Fabric. This integration allows for better collaboration, versioning, and automation of your deployment process across dev, test, and production environments. New Dataflow Gen2 item experience with the option to enable Git integration, deployment pipelines and Public API scenarios. Key benefits Automated deployments: streamline your deployment process by integrating Dataflow with your CI/CD processes in Fabric. Version control: use GIT to manage and version your Dataflow Gen2, ensuring you have a history of changes and can easily roll back if needed. Collaboration: enhance team collaboration by leveraging GIT’s branching and merging capabilities. Multitasking support: you can now have multiple Dataflows open at the same time as other Microsoft Fabric experiences. These new features will significantly improve your workflow and productivity when working with Dataflows Gen2 in Fabric. We look forward to hearing your feedback and suggestions as we continue to enhance this feature. To learn more, refer to the Dataflow Gen2 with CI/CD and Git integration support (Preview) documentation. Dataflow Gen2 Public APIs (Preview) Data Factory in Fabric now provides a robust set of APIs that enable users to automate and manage their dataflows efficiently. These APIs allow for seamless integration with various data sources and services, enabling users to create, update, and monitor their data workflows programmatically. The APIs support a wide range of operations -- including dataflows CRUD (Create, Read, Update, and Delete), scheduling, and monitoring -- making it easier for users to manage their data integration processes. The APIs for dataflows in Fabric Data Factory can be used in various scenarios: Automated deployment: Automate the deployment of dataflows across different environments (development, testing, production) using CI/CD practices. Monitoring and alerts: Set up automated monitoring and alerting systems to track the status of dataflows and receive notifications in case of failures or performance issues. Data integration: Integrate data from multiple sources, such as databases, data lakes, and cloud services, into a unified dataflow for processing and analysis. Error handling: Implement custom error handling and retry mechanisms to ensure dataflows run smoothly and recover from failures. Learn more about Dataflow APIs in the documentation. Dataflow Gen2 parameterization (Preview) Parameters in Dataflow Gen2 enhance flexibility by allowing dynamic adjustments without altering the dataflow itself. They simplify organization, reduce redundancy, and centralize control, making workflows more efficient and adaptable to varying inputs and scenarios. Leveraging query parameters while authoring Dataflows Gen2 has been possible for a long time, however, it was not possible to override the parameter values when refreshing the dataflow. The ability to pass values from a pipeline into a Dataflow parameter for refresh has been one of the top ideas in the Fabric ideas portal since Dataflow Gen2 was released. We are happy to announce the preview of the public parameters capability for Dataflow Gen2 with CI/CD support as well as the support for this new mode within the Dataflow refresh activity in Data pipelines. Public parameters in Dataflow Gen2 with CI/CD support allow users to refresh their Dataflows by passing parameter values outside of the Power Query editor through the Fabric REST API or native Fabric experiences. This enables a dynamic experience with Dataflows, where each refresh can be run with different parameters that affect how the Dataflow is refreshed. To learn more about this new feature, refer to the Use public parameters in Dataflow Gen2 (Preview) documentation. Lakehouse as an incremental refresh destination in Dataflow Gen2 (Preview) Incremental refresh for Lakehouse destinations in Dataflow Gen2 is now in preview! This feature introduces a powerful way to optimize performance and ETL pipeline for one of the most popular data destinations to date. With incremental refresh, users can ensure faster refresh cycles, improve system efficiency, and reduce resource consumption, making it an ideal solution for large-scale analytics and operational data scenarios. This functionality is particularly valuable for businesses leveraging Lakehouse centric solutions to consolidate structured and unstructured data into a unified data model. To use this capability, configure your Dataflow Gen2 with a Lakehouse destination and enable incremental refresh settings within your dataflow editor as usual. Make sure to check out our documentation here to learn more about the considerations when you are using Lakehouse as a destination. To learn more, refer to the Incremental refresh in Dataflow Gen2 documentation. SharePoint files as a destination in Dataflow Gen2 (Preview) SharePoint data destinations in Dataflows Gen2 is now in preview! This innovative feature empowers users to seamlessly write CSV files directly into their designated SharePoint sites, streamlining data integration and enhancing team collaboration within Office 365. Using this new capability, users can effortlessly configure their dataflow queries to output data into specific folders within SharePoint, facilitating smoother workflows and ensuring that your data remains accessible and actionable in your operational processes. We encourage you to explore the possibilities of this feature and provide valuable feedback to help us refine and expand its functionality. Stay tuned for more updates and improvements as we continue to evolve data destinations for Dataflows Gen2! To start using SharePoint data destinations in Dataflows Gen2, follow these simple steps: Create a Dataflow Gen2 Get some data from one of your data sources In the destination settings, choose SharePoint as your output location. Provide the URL of the specific SharePoint site where you want your CSV files to be saved. Make sure you select the correct authentication method. Execute the dataflow to generate and store your CSV files in the selected SharePoint destination. To learn more, refer to the Dataflow Gen2 data destinations and managed settings documentation. Natural language to custom column Copilot is now available within the Custom column dialog of Dataflow Gen2. You can leverage a new Copilot experience where you can have Copilot write a custom column formula based on a prompt that you provide. For example, for a table that has the fields **OrderID**, **Quantity**, **Category**, **Total** you can pass a prompt like the following: If the total order is more than 2000 and the category is B, then provide a discount of 10%. If the total is more than 200 and the category is A, then provide a discount of 25% but only if the quantity is more than 10 otherwise just provide a 10% discount. After submitting this prompt, Copilot will process it and modify the custom column formula for you and adding a name and a data type if necessary. Be sure to give this new Copilot experience inside Dataflow Gen2 a try and share your feedback with us. Community Power Designer - unleash your inner report wizard (Generally Available) PowerBI.tips, in collaboration with Microsoft Fabric, is thrilled to announce that Power Designer is now Generally Available! This is a real time-saving application you won’t want to miss! Transition from generic, standard reports to sophisticated and highly customized presentations. What is Power Designer about? Power Designer is sleek, intuitive, and fun, making designing reports feel less like work and more like unleashing your inner artist. Craft themes like a pro: create detailed theme files for your Power BI reports with ease. Customize colors, fonts, and styles to match your brand. Real-Time visuals: watch your Power BI visuals update live as you build your style. Multipage mastery: add background images to each page with a snap, transforming your reports into polished, magazine-worthy layouts. AI-Powered: let AI take the wheel with auto-placement of visuals in your multipage templates. Preview: test your new theme on reports already published in your workspaces with the preview feature. Now that Power Designer has been released, it’s time to jump in and start creating. Head to your Fabric Workspaces, fire up Power Designer, and let your imagination run wild. To learn more, refer to PowerBI.tips Designer Now in Fabric – Power Designer. Check out the YouTube Video: Introducing Power Designer: Unleash Your Inner Report Wizard! Closing We hope that you enjoy the update! Be sure to join the conversation in the Fabric Community and check out the Fabric documentation to get deeper into the technical details. As always, keep voting on Ideas to help us determine what to build next. We are looking forward to hearing from you!
Statsig Experimentation Analytics (Preview)
Accelerating Product Innovation with Statsig Analytics on Microsoft Fabric Experimentation Analytics from Statsig offers a powerful new capability that’s set to transform how product teams innovate, measure product performance and make data-driven decisions to accelerate product adoption and growth. Imagine being able to unify experimentation, feature rollout, and impact analysis in one seamlessly integrated experience. That’s exactly what this workload delivers, now available to you via the Microsoft Fabric Workload Hub. With Statsig’s Experimentation Analytics in Microsoft Fabric, you can analyze product and behavioral data stored in OneLake in a frictionless, performant, secure, and powerful way. There is no need to move data or rely on external tools to build complex ETL pipelines, and you still have access to a robust Experimentation Analytics tool directly within the Fabric ecosystem. Why this Matters Product teams often face a dilemma and question: Will this feature drive engagement or increase revenue? With Statsig’s Experimentation Analytics, you can now answer that with confidence. Design and analyze experiments directly on your Fabric data. Define custom metrics – user engagement, retention, funnel analysis and more. Connect exposure data without moving it outside Fabric. Run rigorous statistical tests with built-in support for analytical tools. Automatically refresh results as new data lands in One Lake. Discovery: How to access workload in Fabric Start in the Fabric Workload Hub, we discover both first-party workloads and third party (ISV) published workloads. This is where you will find Statsig Analytics - offering a unified solution for experimentation, feature rollout, and impact analytics. It connects natively to OneLake, keeping all data governed and secure inside Fabric. How to: Step-by-Step approach From Metrics to Meaningful Insights Begin by connecting to your Fabric Data warehouse in your Fabric workspace. If already set up, define the metrics source using Statsig’s Metrics Explorer. For example, you may choose to measure User Engagement, Total Revenue or Checkout CTR (Click-through-rate) for your product. Experiment Setup and Assignment With your Fabric Data Warehouse connections and metrics definitions setup you connect to assignment sources to pull in exposure data. This data represents the control and variant groups that were created during the experiment design, including their exposures to different experiments over time. Statsig’s powerful statistical engine offers advanced A/B/n testing capabilities that allow you to choose from a range of statistical methods, Frequentist testing, Confidence intervals, Sequential Testing, and CURE for variance control to name a few. Real-Time, Incremental Analysis With your Datasource, metrics source and assignment source setup you can run deep- analysis, drilldowns, funnel analysis hypothesis testing on the data. With your automatic and incremental data collection and experiment setup Statsig automatically collects and refreshes results daily. No manual updates are necessary. You can simply monitor the scorecard to track the winning variant and expedite decision making. From Insight to Action In Microsoft Fabric if you were tracking the performance of your different experiments in Power BI to manage feature rollout readiness of your product; with insights from Statsig’s experimentation analysis you will be empowered to make a data-driven decision with confidence. A KPI turns green on the Power BI dashboard signaling greenlighting the winning variant for a staged release. Ready to Experiment? Whether you're a product manager, data scientist, or analytics leader—Statsig Analytics in Fabric is your new superpower. Start experimenting today and turn insights into impact. Try it now in the Microsoft Fabric Workload Hub and see your data turn to insights. To learn more about how to purchase a license visit Statsig’s Product page on the Azure Marketplace.Unlocking Geospatial Intelligence in Microsoft Fabric with Esri’s ArcGIS Maps Workload (Preview)
Introducing ArcGIS Maps for Fabric ArcGIS Maps for Microsoft Fabric is now available, bringing Esri’s industry-leading geospatial analytics along with dynamic, interactive mapping and visualization capabilities directly into Fabric. The new ArcGIS Maps workload empowers data professionals to visualize, explore, analyze, share location-based insights and integrate Esri’s authoritative data assets with their ArcGIS online service all seamlessly integrated with your organization’s data in OneLake. What are ArcGIS Maps for Fabric? ArcGIS Maps is a flexible, interactive mapping tool designed for visual data exploration and insight generation. It is an amalgamation of art and science. This is a tool that not only gives you the ability to work with data but also offers a rich graphical interface that allows users to tell a story. Unlike rigid workflows, ArcGIS Maps lets users adapt their analysis as new questions arise, making it easy to uncover patterns, trends, and outliers in their data. End-to-End Integration in Microsoft Fabric OneLake Integration: With the data in OneLake and access to the OneLake data Catalog users can select the most relevant dataset from across their organization’s data assets to use for their analysis. Data Engineering Workflows: Can perform advanced geospatial analysis in Fabric notebooks leveraging of the scale of Spark’s parallel processing to prepare massive volumes of data, that get written back to OneLake and available for visualization and reporting using ArcGIS Maps solution. Power BI Integration: Users can embed interactive maps in Power BI reports and organizational apps, making geospatial insights accessible to everyone. Getting Started Launch ArcGIS for Maps from the Fabric workload hub. With a single click you can create your first web map item which behaves just like any Fabric artifact. Key Features Effortless Data Onboarding Load data into your web map item from OneLake or use Fabric’s built in data-connectivity experience to bring data in from hundreds of different sources. ArcGIS for Maps automatically detects geospatial attributes in your dataset or prompts for coordinates, ensuring your data is ready for mapping. Rich Visualization & Smart Mapping Instantly plot geospatial data and visualize distributions—add cartographic polish, like visual filters that creates a vignette that offers qualitative context to your quantitative analysis. Use smart mapping features like heat maps, color ramps, clustering, and binning to highlight differences, trends, and hotspots. Switch between bubbles, pins, and heat maps with intuitive UI controls to make your data pop. Advanced Analysis & Attribute Table Ability to inspect data columns, view summary statistics, and rename fields for clarity before visualizing on a map. Aggregate large volumes of data (e.g., Customer Sales across a geographic region), by using clustering that dynamically redraws itself as you add and remove data points. It creates and adjusts a histogram of the Sales data by region and as you adjust the cluster size, offering full interactively to explore details by clicking on map features. Integration with ArcGIS Living Atlas & External Data Enhance your maps by adding authoritative data and imagery from ArcGIS Living Atlas of the World or your own ArcGIS Online content to enrich your webmap. Access population density, world terrestrial ecosystems, and many more third-party datasets to enrich your analysis. AI-Powered Arcade Assistant This is a scripting engine that augments the Smart mapping features to provide a natural language prompt interface to conduct additional analyses directly on the active web map data powered by a fine-tuned AI model. Create HTML tables, summaries, and advanced calculations without deep scripting knowledge, thanks to the Arcade scripting language and AI assistant. Scalable Performance & Security ArcGIS for Maps can handle hundreds of thousands of data points in browsers; from very large datasets, pre-processed with GeoAnalytics libraries in Spark Notebooks for optimal performance. Share maps securely with users across your organization, securely with granular access control and consent management. Ready to unlock the power of Geospatial Intelligence in Fabric? Try ArcGIS Maps for Microsoft Fabric today, Microsoft Fabric Workload Hub Access the documentation for a step-by-step guide on using the product.Fabric February 2026 Feature Summary
Welcome to the February 2026 Microsoft Fabric update! This month brings a wide range of enhancements across the Fabric platform—from improvements to the OneLake Catalog and developer experiences, to meaningful updates in Data Engineering, Data Factory, Real‑Time Intelligence, and more. Whether you’re building, operating, or scaling solutions in Fabric, there’s plenty here to explore. And with FabCon just weeks away, February’s updates are a great preview of what’s ahead. Don’t miss your chance to get Fabric certified for FREE If you are ready to take your Fabric exam in the next month, the Fabric team would like to give you a 100% voucher to cover the cost. Request a voucher by February 28, 2026. Terms and conditions apply. Three weeks until FabCon – will we see you there? Join us for the ultimate Power BI, Microsoft Fabric SQL, Real-Time Intelligence, AI, and Databases community-led event from March 16-20, 2026, in Atlanta, GA. The third annual FabCon Americas will feature sessions from your favorite Microsoft and community speakers, keynotes, more opportunities to Ask the Experts for 1:1 support, an engaging community lounge with opportunities to network and connect with your peers, a dedicated partner pre-day, a packed expo hall, attendee favorites Power Hour and the Data Viz World Championships live finals, and a can’t-miss attendee party at the Georgia Aquarium. Register with code FABCOMM to save $200. Contents Fabric Platform Workspace Apps now in the OneLake Catalog Streamlined item details Managing Fabric Identity limits within your tenant Horizontal Tab Display Settings Data Engineering Enhanced notebook version history with multiple sources Python notebooks add %run support Full size mode in Fabric notebook Announcing Private Link Support for Microsoft Fabric API for GraphQL CI/CD for API for GraphQL (Generally Available) Support for default arguments for Fabric user data functions Microsoft ODBC Driver for Microsoft Fabric Data Engineering (Preview) Customer Managed Key Encryption Support for Notebook Code Data Science Semantic Link 0.13.0 is Live Monitoring Real-Time Scoring Model Endpoints Data Warehouse Export migration summary SQL Pool Insights Real-Time Intelligence Effortless Real-Time Data Connection Streaming real-time data from private networks into RTI with Eventstream connectors Faster insights: real-time dashboard performance improvements Data Factory Recent data: Get back to your data faster (Preview) Improvements to the Fabric variable libraries integration in Dataflow Gen2 Relative references with Fabric connectors in Dataflow Gen2 Introducing Dataflow Gen2's just-in-time publishing mechanism Modern Evaluator for Dataflow Gen2 (Generally Available) Incremental copy from Fabric Lakehouse now supports both CDF and watermark-based methods in Copy job SAP Datasphere outbound for Amazon S3 and Google cloud storage in Copy job Column Mapping in CDC for Copy Job Rowversion now supported as an incremental column in SQL database Copy job Copy job activity now supports Service Principal and Workspace identity authentication Parallel Read Support for Large CSV Dataset Adaptive Performance Tuning: Intelligent Optimization for Data Movement (Preview) Other Fabric VS Code extension for browsing, editing item definitions, and MCP support February Monthly Update Video Fabric Platform Workspace Apps now in the OneLake Catalog Workspace Apps (Apps V2) are now supported in the OneLake Catalog, so you can discover, browse, and open apps directly from the Insights category. This category focuses on business-ready content designed to help you analyze, visualize, and report on data to drive actionable insights. With this update, Workspace Apps appear alongside other business-facing content such as organizational apps and reports, making it easier to explore all relevant insights in one place without needing to switch between different experiences. The Catalog also surfaces key metadata for each Workspace App, helping you quickly understand what the app contains before opening it. From there, you can open the app directly and start exploring insights right away. With the addition of Workspace Apps, the OneLake Catalog now includes all item types available in Microsoft Fabric, making it the central place to discover, understand, and access your Fabric content. Figure: Workspace Apps displayed in the OneLake Catalog under the Insights category. Learn more about the OneLake catalog in the OneLake catalog overview documentation. Streamlined item details The updated Item Details experience now extends beyond the OneLake Catalog to include full-page experiences for items accessed from outside the catalog. For example, when you open a Semantic Model directly from Workspaces, you will now see a modern, unified details page that matches the streamlined in-context experience found within the OneLake Catalog. This update brings a consistent design language across Fabric, offering improved usability and access to richer metadata: the enhanced details page now features the complete schema of all OneLake stored data items, making it easier to understand item characteristics. It also shows and visualizes item-level lineage, managing permissions and monitoring run/refresh history, all in one place. You can quickly find this key information, whether they’re navigating through the catalog or opening items in a standalone context. Figure: A Lakehouse in the new item details experience Managing Fabric Identity limits within your tenant Fabric Identity governance at scale just got easier. We are introducing a new tenant admin setting that gives you control over the maximum number of Fabric identities (hence Workspace identities) in your organization. With this update, Fabric tenant admins can: Scale beyond previous constraint—the default limit for number of Fabric identities in an organization increases from 1,000 to 10,000 identities. Set custom limits for how many Fabric identities can be created in their tenant. Manage limits programmatically by using the Update Tenant Setting REST API. How it works: The new setting “Define maximum number of Fabric identities in a tenant” is in the Fabric Admin portal in Tenant settings, within Developer settings. When the setting is disabled (the default), your tenant supports up to 10,000 Fabric identities—a 10x increase from the previous limit. Enable the setting to specify your own maximum. The value you enter becomes the upper limit for Fabric identity creation across your tenant. Note: Fabric doesn't validate that your custom limit falls within your Entra ID resource quota. Before setting a custom limit, check your organizations Entra ID service limits. If a workspace admin tries to create a new workspace identity that would exceed the limit, they'll see a clear error message explaining the reason. Figure: Configuring Maximum number of Fabric Identities in a tenant. You can also manage this setting programmatically using the Update Tenant Setting API. Sample HTTP request: POST https://api.fabric.microsoft.com/v1/admin/tenantsettings/ConfigureFabricIdentityTenantLimit/update { "enabled": true, "properties": [ { "name": "FabricIdentityTenantLimit", "value": "100", "type": "int" } ] } Sample JSON response: Status code: 200{ "tenantSettings": [ { "settingName": "ConfigureFabricIdentityTenantLimit", "title": "Define maximum number of Fabric identities in a tenant", "enabled": true, "canSpecifySecurityGroups": false, "tenantSettingGroup": "Developer settings", "properties": [ { "name": "FabricIdentityTenantLimit", "value": "100", "type": "Integer" } ] } ] } To learn more about identities in Fabric, see the documentation. For more information about all the tenant admin settings in Fabric, see the Tenant settings index. Horizontal Tab Display Settings To give developers more control over how they navigate open items in Microsoft Fabric, we’ve introduced new horizontal tab display settings. These settings let you tailor how tabs appear across the top of the Fabric interface—helping you stay organized and maintain focus during complex multitasking workflows. What’s new: Open the tab settings menu to quickly access tab display options by right‑clicking any tab. Figure: Open the horizontal tab display settings directly from a tab’s right-click menu. Two display modes Full tab names always show each tab’s full name for maximum clarity. Adaptive truncated names automatically shorten names when space is limited, allowing more tabs to remain visible. Figure: Horizontal tab display modes, including full tab names and adaptive truncated names, configured in Preferences page in Settings. Overflow menu When space runs out, tabs automatically collapse into a clean overflow list, making it easy to jump to any open item. Figure: Tabs automatically move into an overflow menu when there is insufficient space in the horizontal tab bar. These enhancements streamline navigation for developers working across multiple items and workspaces, reducing friction and improving overall multitasking efficiency. Find more details in this documentation. Data Engineering Enhanced notebook version history with multiple sources Keeping track of how a notebook evolves gets tricky when changes can come from different entry points—editing in the Fabric portal, syncing from source control, or other update flows. Fabric notebooks seamlessly integrate with Git, deployment pipelines, and Visual Studio Code. Each saved version is automatically captured in the notebook’s version history. Versions may originate from direct edits within the notebook, Git synchronizations, deployment pipeline, or publishing via VS Code. The source of each version is clearly labeled in version history to provide full traceability. Figure: Multiple sources records in notebook version history With the Enhanced Notebook Version History with Multiple Sources support, Fabric Notebooks now surface a clearer, more trustworthy history by reflecting versions from multiple origins, helping you trace changes, collaborate with confidence, and roll back to the right point when needed. Especially in CI/CD workflows (Git sync, deployment pipeline, public API), and team-authored workflows. Learn more about version history in the Version history documentation. Python notebooks add %run support Python developers often want to keep notebooks modular—shared utilities, setup logic, and reusable helpers shouldn’t be copy‑pasted everywhere. Python notebooks now support %run, enabling a familiar pattern for executing shared “code modules” and reusing logic across notebooks. This makes it easier to structure projects cleanly, iterate faster, and maintain common code in a single place. Figure: Reference another python notebook with intellisense Figure: Reference run python notebook You can use %run to reference and execute other notebooks within the same execution context, allowing you to directly call functions and reuse variables defined in those notebooks. Currently, %run in Python notebooks supports referencing notebook items only. Support for running code modules (such as .py files) from the notebook resources folder is coming soon—stay tuned. You can reference the reference run a notebook documentation for the detailed usage. Full size mode in Fabric notebook Full-size mode of cells is now available on Fabric notebook. When you’re working on a long or complex cell, the surrounding UI can get in the way. Full Size Mode lets you expand a single cell to fill the notebook for distraction‑free editing—ideal for deep refactors, large SQL or Python blocks, or screensharing. In full‑size mode, you retain full editing capability, stay focused on the selected cell, and can conveniently navigate to the previous or next cell without leaving the focused view. Figure: Enable full size mode on cell toolbar. Figure: Example of full-size mode. Learn more: Develop, execute, and manage notebooks Announcing Private Link Support for Microsoft Fabric API for GraphQL Microsoft Fabric API for GraphQL now supports Tenant Level Private Link, bringing enterprise-grade network security to your data APIs. This highly requested feature enables organizations to access their GraphQL APIs through private connectivity, ensuring data traffic never traverses the public internet. Secure data access with Private Link: This feature allows organizations to access GraphQL APIs through Microsoft’s private backbone network, improving security by preventing exposure to public internet threats and supporting compliance requirements. Simplified network management: Private Link reduces the need for complex firewall rules or VPN setups by allowing API calls only through approved private endpoints, easing governance and integration with existing Azure Private Link infrastructure. Enterprise-ready security model: Enabling Private Link at the tenant level integrates GraphQL APIs into a secured network environment complemented by Microsoft Entra ID authentication and flexible security options like single sign-on and saved credentials. To enable private link, update your tenant admin settings to enable Azure private link. Figure: Enable private link for your tenant Learn more Private link support for API for GraphQL. CI/CD for API for GraphQL (Generally Available) With this release, we have made improvements to reliability and performance on of the experience with Fabric CI/CD and deployment pipelines experience. Your teams can manage GraphQL artifacts in Git, collaborate with familiar pull-request workflows, and promote changes across environments using CI/CD—bringing the same engineering rigor to APIs that you already use for code and data. With CI/CD support, you can do the following: Git-enabled source control for your GraphQL API artifacts so you can version, review, and roll back changes. Support with Fabric deployment pipelines that allow you to build release pipelines for managing API for GraphQL items. Improved collaboration with pull requests, code reviews, and branching strategies applied to API changes. Figure: Screenshot of source control for API for GraphQL Learn more about API for GraphQL CI/CD and source control. Support for default arguments for Fabric user data functions Fabric User data functions now support default argument values, allowing omitted arguments to use preset defaults, which simplifies function calls and enhances code flexibility. This feature supports various input types including strings, boolean, floats, int, arrays, and objects. Functions become more versatile as they can handle common use cases with fewer arguments, while still allowing for customization when needed. Figure: Code snippet of using function with default arguments Learn more about default arguments for user data functions. Microsoft ODBC Driver for Microsoft Fabric Data Engineering (Preview) ODBC (Open Database Connectivity) is a widely adopted industry standard that enables applications to connect to and work with data across databases and big data platforms. Today, we’re introducing the Microsoft ODBC Driver for Microsoft Fabric Data Engineering (Preview) - an enterprise‑grade connector that delivers secure, reliable, and flexible Spark SQL connectivity for .NET, Python, and other ODBC‑compatible applications and BI tools, all powered through Microsoft Fabric’s Livy APIs. Built specifically for Fabric Data Engineering, this driver offers deep integration with OneLake and Lakehouse data, supports environment‑based execution, and enables flexible Spark configuration tailored to your workloads. With full ODBC 3.x compliance, Microsoft Entra ID authentication, comprehensive Spark SQL and data type support, performance optimizations for large datasets, and enterprise‑ready features like proxy support and session reuse, the Microsoft ODBC Driver helps teams accelerate Spark‑powered data engineering with the security, reliability, and performance expected in modern enterprise environments. Figure: The animated GIF demonstrates how to get started using ODBC driver To download and learn more about the Microsoft ODBC Driver for Microsoft Fabric Data Engineering, please refer to official documentation: Microsoft ODBC Driver for Microsoft Fabric Data Engineering. Customer Managed Key Encryption Support for Notebook Code Enterprise teams can now run Microsoft Fabric Notebooks in CMK‑enabled workspaces with Notebook content and metadata encrypted at rest using customer‑owned keys in Azure Key Vault, supporting stricter governance and compliance requirements without changing developer workflows. What’s new: Notebooks are fully supported in CMK‑enabled workspaces With this update, Notebooks can be created and used in workspaces where CMK encryption is enabled, and the Notebook content and associated Notebook metadata stored as part of Data Engineering items are protected using the workspace’s customer‑managed key. Concretely, this covers core Notebook content artifacts such as cell source, cell output, and cell attachments, so the key you control can be applied consistently to what developers author and what the system stores for notebook execution and collaboration. To enable CMK for your Fabric workspace (and use Notebooks in that CMK‑enabled workspace), follow the official documentation: Customer‑managed keys for Fabric workspaces Data Science Semantic Link 0.13.0 is Live With the 0.13.0 release, Semantic Link continues to expand its Fabric coverage and management capabilities. This update introduces new modules for lakehouse, reports, semantic models, SQL endpoints, and Spark, enabling end‑to‑end workspace operations—from creating and managing lakehouses and tables, to cloning and rebinding reports, refreshing and monitoring semantic models, and administering SQL and Spark settings. Several Fabric APIs are now surfaced consistently across modules, simplifying common workflows and improving API discoverability. The release also includes targeted API refinements and bug fixes, improving reliability for service principal authentication and correctness when evaluating measures. Overall, 0.13.0 makes it easier to manage Fabric assets programmatically at scale with stronger consistency and control. Explore the release notes. To help you explore these scenarios in practice, we’ve also published three short demos showcasing Sempy for data science, Sempy for Power BI automation, and Sempy for data engineering, illustrating how Semantic Link can unify workflows across personas and accelerate development within Fabric. Cast your vote for additional capabilities from Semantic Link Labs to be included in Semantic Link. Monitoring Real-Time Scoring Model Endpoints The new monitoring experience for real‑time scoring endpoints in Microsoft Fabric provides clear visibility into request volume, error rates, and latency as models run in production. Teams can easily compare these metrics across endpoint versions to validate improvements, catch regressions early, and make confident rollout or rollback decisions based on real usage. From tracking adoption to diagnosing issues and ensuring consistent performance underload, endpoint monitoring helps teams move faster from insight to action—delivering more reliable ML experiences while staying focused on scaling impact and business value. Data Warehouse Export migration summary Export Migration Summary is a new capability in Migration Assistant that makes it simple, reliable, and secure to download your full migration results in formats that best fit your workflow. The export option is available directly from the Migration Assistant’s summary view and full screen view. Figure 1: Export menu Figure 2: Export file formats Once triggered, the export runs reliably in the background, even if the Migration Assistant window is closed, ensuring a smooth workflow for large, multi-object migrations. The following output formats are supported: Excel Fully structured workbook with two worksheets: Migrated Objects and Objects To Fix MIP-compliant and aligned with your organization’s sensitivity labels. CSV Lightweight and tool-friendly Each exported file provides a structured, comprehensive view of your migration results, including: Field Description Object name Name of the SQL object Object type SQL object types such as table, view, function, stored procedure. State Translation State Adjusted: Fabric Data Warehouse compatible updates are applied Not adjusted: No change in the original script Details List of adjustments applied or error messages Type of error Type of error as Translation message, Translation error, Translation apply error Figure 3: Fields in exported file This structure enables teams to aggregate migration details at object or object type level and identify patterns across objects. Export Migration Summary removes a major blocker for customers who need shareable, reliable artifacts that reflect the true state of their migration progress. Learn more in the Migration Assistant for Fabric Data Warehouse documentation. SQL Pool Insights Understanding why workloads slow down often require visibility beyond individual queries. SQL Pool Insights extends the existing Query Insights experience with pool‑level telemetry, helping you understand how resources are allocated and when pools are under pressure in Microsoft Fabric Data Warehouse. Figure - The image shows the sql_pool_insights schema With SQL Pool Insights, you can: Monitor the health of built‑in SELECT and NON SELECT SQL pools. Track pressure events, configuration changes, and capacity updates over time. Correlate pool‑level pressure with query performance using existing Query Insights views. Validate resource isolation between read‑optimized and write‑optimized workloads. This feature adds a new system view — queryinsights.sql_pool_insights — that logs pool state changes and sustained pressure events, giving you actionable signals for troubleshooting performance issues and planning capacity more effectively. Learn more about SQL Pool Insights Real-Time Intelligence Effortless Real-Time Data Connection Connecting data is often the first step on a user’s Real-Time Intelligence journey—and it should be effortless. Previously, the left navigation in the Real-Time hub (RTH) included two separate entries for connecting data—Data sources and Azure sources. While well intentioned, this distinction didn’t always match how users think about the task at hand. The most common question wasn’t about categories; it was just how to add data. To better reflect that reality, we’ve unified these entry points into a single menu item: Add data. With this update: There’s now one clear place to begin when connecting data. The navigation focuses on intent, not source taxonomy. Users can move faster without second‑guessing their choices. Under the hood, nothing has changed. You still have access to the same rich set of data sources, including all out-of-box data connectors, Azure sources, Azure Diagnostics logs, and more. What’s changed is the experience—clearer, simpler, and designed to help you get value faster. Note: we are rolling this change out gradually, so you may see it in the coming weeks. Figure: Real-Time hub left navigation before the change Figure: Real-Time hub left-navigation after the change Figure: New data connector context menu Learn more about Real-Time Hub. Try it out and share your feedback. Streaming real-time data from private networks into RTI with Eventstream connectors Real-Time Intelligence Eventstream is designed to bring real-time data from diverse sources, transform it, and effortlessly route it to various destinations. For sources that run in private network environments , such as cloud virtual network or on-premises infrastructures, a secure method is required to allow Eventstream to access the source. The streaming connector’s support for virtual networks (vNet) and on-premises environments offers a secure, managed pathway, enabling Eventstream to reliably connect with these private-network streaming sources. To enable data transfer from a source within a private network into Eventstream, it is necessary to establish an Azure managed virtual network as an intermediary bridge, as illustrated in the diagram. The Azure virtual network should be connected to the private network hosting the data source using appropriate methods, such as VPN or ExpressRoute for on-premises scenarios, and private endpoints or network peering for Azure sources, etc. Subsequently, the Eventstream streaming connector instance will be injected into this virtual network through SWIFT injection, allowing secure connectivity between the connector and the data source located within the private network. Figure: Eventstream connectors private network support overview To facilitate streaming connector vNet injection into an Azure virtual network you’ve created, Fabric provides a centralized location for network or data engineers to manage the references to Azure virtual network resources. The streaming virtual network data gateway in Fabric serves this purpose for Eventstream. Unlike 'Virtual network data gateways' and 'On-premises data gateways' , this new option does not require cluster provisioning or additional capacity. However, the user experience across all three gateway types remains largely similar. Create and manage the ‘streaming virtual network data gateway’ in the ‘Manage Connections and Gateways’ page in Fabric. Select it when setting up streaming connections for Eventstream sources using the Get Events wizard or 'Streaming virtual network'. After that, you can configure your Eventstream data source as usual. Figure: Streaming virtual network data gateway configuration for Eventstream To help familiarize yourself with the end-to- end flow of the feature check out this detailed demo on Streaming Real-Time Data from Private Networks into RTI with Evenstream Connectors. For a step-by-step guide on getting started, please refer to the document: Connect to Streaming Sources in Virtual Network or On Premises with Eventstream. Faster insights: real-time dashboard performance improvements Based directly on community feedback, we’ve optimized the Real-Time Dashboard from the ground up, ensuring a snappier, high-performance experience. Thanks to a series of performance optimizations across the dashboard experience, we’ve achieved a significant double-digit reduction in full dashboard load time, along with major improvements in common interactions:‑digit reduction in full dashboard load time, along with major improvements in common interactions: Much faster initial dashboard load—in some scenarios, up to 6× faster Large dataset visualizations load dramatically quicker, reducing wait time and friction Charts render more efficiently (including up to 10× faster pie charts) Smoother, more responsive UI, with freezes and visual jumps eliminated Whether you are loading large datasets or refreshing live visuals, the UI is now smoother and significantly more responsive, ensuring your data keeps pace with your decisions. Note: The video demonstrates performance benchmarks conducted in a controlled internal environment. Learn more about What is Real-Time Dashboard? Data Factory Recent data: Get back to your data faster (Preview) When you work with the same data sources repeatedly in Dataflow Gen2, the new Recent data (Preview) module helps you access your most frequently used data faster. Why this matters Provides quick access to frequently used tables, files, folders, databases, sheets, etc. Eliminates repetitive navigation steps Improves productivity in transforming data by efficiently connecting and ingesting data How to access Recent data Getting started is straightforward. Open any Dataflow Gen2 in your Fabric workspace and you'll find two convenient ways to access Recent data. First, you can select Recent data directly from the Power Query ribbon for immediate access to your history. Figure: Recent data in the Power Query Ribbon Alternatively, select Get data and choose the Recent data module from the home tab or dedicated tab. Figure: Recent data in Modern Get Data When you select an item from your Recent data module, it loads directly into the Power Query editor without additional navigation steps required by default. You can start applying transformations immediately. If you need to explore related items in the same location, select Browse location to discover other tables or files in the same folder or database, making it easy to include additional related data in your dataflow. This Preview feature is available now, learn more in the Recent data documentation. Improvements to the Fabric variable libraries integration in Dataflow Gen2 In September 2025, we released a preview of the Fabric variable libraries integration with Dataflow Gen2. This update address two of the most common feedback themes: Variable limit: Dataflows no longer have a limit on how many variables it can retrieve per evaluation. Power Query editor support: the data preview shown in the Power Query editor evaluates the variables. This includes both the usage of the Variable.Value and Variable.ValueOrDefault functions. Figure: Screenshot of the Power Query editor for Dataflow Gen2 rendering the output of the Variable.Value function for a Variable Library with the name My Library and a variable with the name MyDateTime We also identified and fixed issues that caused saving issues when variables were used in data destinations, and we improved the overall Power Query editor experience when variables are used in navigation steps. We’re continuing to improve our experience and will share updates in the coming months. Be sure to leave your feedback in the Data Factory community forum where you can engage directly with us if you have any questions or suggestions. Learn more from the Use Fabric variable libraries in Dataflow Gen2 (Preview) documentation. Relative references with Fabric connectors in Dataflow Gen2 One of the core principles of Fabric and Data Factory is enabling solutions that are CI/CD-ready. In Dataflow Gen2, you can already use public parameters and Fabric variable libraries to make your solutions dynamic and compatible across deployment pipelines. We’re introducing a new capability to simplify CI/CD scenarios when using Fabric connectors: Relative References. What’s changing? Previously, when you used Fabric connectors (Lakehouse, Warehouse, or SQL Database), the generated script relied on absolute references—such as Workspace ID and item IDs (e.g., Lakehouse ID, Warehouse ID). With Relative References, you’ll see a new node in the navigation dialog called (Current Workspace). This allows you to select items within the current workspace context. Once selected, the script will reference the item name instead of unique IDs. Figure: Diagram comparing the possible experiences between absolute and relative references and the M script crated for each. Why it matters This approach ensures that when you move your solution from development to testing or production, no script changes are required. Your Dataflow will continue to work based on item names, making deployments seamless without adding any extra components Learn more Check out the documentation for Fabric Lakehouse, Fabric Warehouse, and Fabric SQL connectors for more information. Introducing Dataflow Gen2's just-in-time publishing mechanism Previously, Dataflow Gen2 required you to manually trigger a publishing operation before running or refreshing a dataflow whenever unpublished changes were present. With the updated experience, the run/refresh operation now automatically checks if a publication is needed and completes it as part of the job. This simplifies the workflow and ensures that runs succeed without requiring an explicit publish step. You can still rely on the following behaviors: Explicit publish control: You can continue to trigger a publish directly using the Publish job when you need full control. Saving in the UI: Saving a dataflow in the authoring UI still performs a publication as part of the save process. Longer first refresh: The first refresh after making changes may take longer, because publishing now happens automatically as part of that initial run. CI/CD deployments: When deploying across environments, a separate publishing step is no longer required. The first run in the target environment will be published automatically if needed. Learn more about this new mechanism from: Dataflow Gen2 with CI/CD and Git integration. Modern Evaluator for Dataflow Gen2 (Generally Available) The Modern Query Evaluation Engine (Modern Evaluator) for Dataflow Gen2 brings substantial performance and reliability improvements to data transformation workloads across Microsoft Fabric. Built on .NET 8, this engine delivers faster execution, more efficient processing, and improved scalability for complex dataflows. As part of its GA rollout, the Modern Evaluator now supports more than 80 connectors, significantly expanding coverage across enterprise and SaaS data sources. Key improvements in this release Broad connector support: the Modern Evaluator now works with 80+ connectors—including Azure Data Explorer, Lakehouse, Warehouse, Salesforce, Google Analytics, Fabric-native sources, and many more. This includes some SQL-based connectors such as Fabric SQL Database, SQL Server Database and others. This expanded coverage ensures that most Dataflow Gen2 scenarios can benefit from the improved engine. Faster and more efficient Web requests: Enhancements to Web connector handling result in lower overhead for HTTP-based data sources. Customers can expect smoother query execution and improved resilience when working with REST APIs or other web endpoints-based data sources. Customers can expect smoother query execution and improved resilience when working with REST APIs or other web endpoints. ‑based data sources. Customers can expect smoother query execution and improved resilience when working with REST APIs or other web endpoints. Learn more about the modern query evaluator in Dataflow Gen2 and its compatible connectors: Modern Evaluator for Dataflow Gen2 with CI/CD. Incremental copy from Fabric Lakehouse now supports both CDF and watermark-based methods in Copy job When performing incremental copy from a Fabric Lakehouse table, we strongly recommend using CDF (Delta Change Data Feed) to capture row inserts, updates, and deletions, and replicate them to supported destinations. Figure: Incremental copy from Fabric Lakehouse via CDF. However, you can now also optionally use watermark-based incremental copy without enabling CDF. In this mode, you can select an incremental column for each table to identify changes. To enable this, go to the Advanced Settings button after creating the Copy job, where you will have the option to switch from CDF to using a watermark column. Figure: Incremental copy from Fabric Lakehouse via watermark-based method Learn more from the What is Copy job in Data Factory documentation. SAP Datasphere outbound for Amazon S3 and Google cloud storage in Copy job Previously, you could use SAP Datasphere Outbound for ADLS Gen2 in Copy job to perform CDC replication from SAP to any supported destination. For more details, see Tutorial: Copy job with SAP Datasphere Outbound (Preview). You can now also use SAP Datasphere Outbound for Amazon S3 and SAP Datasphere Outbound for Google Cloud Storage, expanding staging storage support across multiple clouds so you can choose the option that best fits your scenario. Figure: Selecting Copy data from SAP Datasphere Outbound for Amazon S3 and Google Cloud Storage Learn more from Tutorial: Copy job with SAP Datasphere Outbound (Preview) - Microsoft Fabric | Microsoft Learn Column Mapping in CDC for Copy Job Column mapping from source to destination is now supported during CDC replication within the Copy job. This is useful when you want to rename columns, change data types, or otherwise customize the schema in the destination store. Column mapping now is supported across all data movement patterns in Copy job, including full copy, watermark-based incremental copy and CDC replication. Figure: Column Mapping in Copy job. Learn more from Change data capture (CDC) in Copy Job - Microsoft Fabric | Microsoft Learn. Rowversion now supported as an incremental column in SQL database Copy job Copy job simplifies data movement from many sources to many destinations by natively supporting multiple delivery styles, including bulk copy, incremental copy, and change data capture (CDC) replication. For incremental copy, the first run performs a full copy, and subsequent runs transfer only new or changed data from the last run to save time and resources. If CDC is not enabled on your database, you must select an incremental column for each table. This column acts as a marker, allowing Copy job to identify rows that are new or updated since the last run. Previously, this column was limited to date/time values or increasing numeric values. You can now also select RowVersion as the incremental column when performing incremental copy from SQL Server, Azure SQL Database, SQL Managed Instance, or SQL in Fabric. Figure: Selecting RowVersion to identify changes when incremental copy from SQL database Learn more from What is Copy job in Data Factory - Microsoft Fabric | Microsoft Learn. Copy job activity now supports Service Principal and Workspace identity authentication We are expanding authentication support in the Copy job activity in pipeline, making it easier than ever to securely connect, integrate, and move data across a broad range of enterprise and SaaS systems. This enhancement reflects our continued commitment to delivering an enterprise-ready data integration platform that balances security, flexibility, and ease of use. The Copy job activity now supports additional authentication methods to your Copy job item in pipeline, enabling customers to choose the security model that best fits their organizational standards and compliance requirements. The added authentication types are service principal and workspace identity. Figure: Authentication kind in Copy job activity. Why this matters Authentication is no longer just a connection detail; it is a foundational requirement for enterprise-scale data integration. With expanded authentication support in the Copy job activity, customers can now: Strengthen security posture by minimizing long-lived secrets and adopting identity-based access. Accelerate time to value by connecting to copy job items using native, first-class authentication mechanisms. Simplify compliance and audits through standardized authentication patterns aligned with enterprise security policies. Improve operational reliability by leveraging managed identity and token-based access. These improvements are especially impactful for organizations operating in regulated industries or managing large-scale hybrid environments where security consistency is non-negotiable. Getting started The new authentication options are available within the Copy job activity setting in Fabric Data Factory. Explore these new capabilities and start standardizing modern, secure authentication patterns across your data integration workflows. Learn more about the authentication capability in copy job activity. Parallel Read Support for Large CSV Dataset As we continue improving ingestion performance, we’ve introduced a new enhancement for reading CSV datasets in Data Factory. It significantly boosts ingestion throughput for large CSV files—a common customer challenge when a single file can’t fully take advantage of parallel reads. With this update, Data Factory can now read large CSV files in parallel when the format allows for safe partitioning, delivering better performance and scalability while preserving correctness. By using your multiline configuration, the service can determine how to split the file and process it in parallel, dramatically improving read performance. When multiline behavior is explicitly defined, the service can: Safely identify record boundaries even in large files. Partition the file into multiple logical chunks. Read and process those chunks concurrently. Figure: Multiline rows setting for reading delimited text. This enables higher throughput without compromising data correctness. Why multiline configuration matters CSV files that contain multiline records such as fields with embedded line breaks enclosed in quotes which require special handling. Without explicit configuration, the system must assume the most conservative parsing model, which prevents parallelization. By specifying multiline information on the source, users provide the necessary context for the service to: Correctly interpret row boundaries. Avoid recording corruption during parallel reads. Confidently enable parallelism where it is safe. This opt-in design ensures that performance improvements are applied only when they are valid for the data format. Getting started We encourage customers working with large CSV datasets to review the source configurations and unlock the benefits of this new capability to take advantage of parallel reads for large CSV files. Learn more about the performance optimization for copying delimited text files. Adaptive Performance Tuning: Intelligent Optimization for Data Movement (Preview) Adaptive Performance Tuning is designed to intelligently optimize data movement performance based on your configuration and runtime context. This feature represents a major step forward in making performance tuning simpler, safer, and more effective without requiring deep manual expertise or trial-and-error adjustments. As data volumes grow and integration scenarios become more diverse, achieving optimal performance has become increasingly complex. Customers must balance throughput, reliability, cost, and data correctness across a wide range of sources, destinations, formats, and network environments. Adaptive Performance Tuning addresses this challenge by allowing the service to dynamically apply performance optimizations informed by customer configurations and real execution conditions. Adaptive Performance Tuning is designed with safety and predictability as first principles. Optimizations are applied only when they are compatible with the configured semantics of the task, ensuring that performance gains do not compromise data accuracy or expected behavior. As a preview feature, Adaptive Performance Tuning is: Explicitly opt-in, giving customers full control. Non-breaking, with no required changes to existing configurations. Incrementally evolving, informed by customer feedback and real-world use. Adaptive Performance Tuning is part of a broader vision to make Data Factory a more intelligent, self-optimizing platform. By combining rich configuration signals with service-side intelligence, we aim to help customers focus less on infrastructure tuning and more on delivering business value from their data. Figure: Adaptive performance tuning setting. As the preview evolves, we plan to expand the range of supported optimization scenarios, continuously improving performance outcomes. Getting started with the preview Customers can enable Adaptive Performance Tuning directly within their pipeline settings and begin benefiting from service-driven performance optimization. We encourage users to try the preview, monitor performance improvements, and share feedback to help shape the future of this capability. Learn more about the Adaptive Performance Tuning feature. Other Fabric VS Code extension for browsing, editing item definitions, and MCP support The Microsoft Fabric extension for Visual Studio Code has been enhanced to improve the user experience in exploring, editing, and managing Fabric items directly within the editor and through integration with Fabric MCP server and GitHub Copilot chat. Browse workspace folders: Users can now view and drill into folders and their contents within the workspace to better understand the organization of Fabric content without leaving VS Code. View and edit Fabric item definitions: The extension supports viewing item definitions in read-only mode by default, but you can enable editing through extension’s settings. Changes saved directly update the Fabric item in the workspace, but users should proceed cautiously to avoid breaking changes. Fabric MCP server integration: The Fabric MCP server extension can be enabled alongside the Fabric and GitHub Copilot Chat extensions, offering tailored tools for working with Fabric artifacts, including CRUD operations, generating design documents, and accessing Microsoft Fabric documentation through a specialized agent mode. Figure: Using Fabric MCP in VS Code to design a data analytics solution. Learn more about the new features enabled for Fabric extension for VS code. That’s a wrap for February! We hope these updates help you work faster, build with confidence, and get even more value from Microsoft Fabric. With FabCon right around the corner, it’s an exciting time to connect with the community, learn from experts, and see these capabilities come to life. If you’re heading to Atlanta, we can’t wait to see you there—and if not, there’s still plenty to dig into until next month’s update.Fabric March 2025 Feature Summary
Welcome to the March Feature Summary! From the innovative Variable library (Preview) to the powerful Service Principal support in the CI/CD features, there's a lot to explore. Dive in and discover how the new Partner Workloads in Fabric bring cutting-edge capabilities to your workspace. Plus, enhanced OneLake security ensures your data is protected. And don't miss out on the expanded regional availability for Eventstream's managed private endpoints, making it easier for organizations worldwide to build secure, scalable streaming solutions. With FabCon kicking off today, the announcements are rolling in! Get ready to explore these features and more in the March 2025 updates for Fabric! Contents Power BI Fabric Platform Variable library (Preview) New CI/CD features New Partner Workloads in Fabric Workload Development Kit improvements Drive data discovery & curation with tags (Generally Available) Fabric Domains OneLake catalog and Modern Get Data are now integrated into Excel for Windows Enhanced search on tables and columns in the OneLake catalog New quick action for seamless navigation in the OneLake catalog Purview Data Loss Prevention (DLP) policies for KQL and Mirrored DBs Multi-tenant organization (MTO) (Generally Available) OneLake OneLake security OneLake shared access signatures (SAS) (Generally Available) Data Engineering Write capabilities and PySpark support in Spark Connector for Fabric DW Esri’s ArcGIS GeoAnalytics integration with Microsoft Fabric Spark (Preview) Deployment pipeline inside Spark Job Definition (Preview) Row-level and Column-level security in Spark Introducing Pylance language support for Fabric Notebook Environment sharing across workspaces (Preview) Shortcuts now supported in Lakehouse Git metadata representation and in Fabric Deployment pipelines Announcing Fabric User Data Functions (Preview) OPTIMIZE FAST and FSCK commands in Fabric Runtime 1.3 for Apache Spark (Generally Available) Fabric Spark Monitoring APIs (Preview) Notebook Integration with User Data Functions (UDFs) (Preview) Data Science Copilot in Notebooks Agentic and UX enhancements Fabric data agent Copilot and AI capabilities now available across all paid SKUs Fabric data agent integration with Azure AI Agent Service (Preview) Fabric data agent SDK (Preview) Data Warehouse AI functions in Data Warehouse Fabric User Data Functions in Data Warehouse Scalar SQL User-defined Functions Developer Experiences in Fabric Warehouse JSON data in OPENROWSET SQL audit logs (Preview) Fabric Data Warehouse item permissions OneLake Security for Lakehouse Analytics SQL Endpoints Private Preview Real-Time Intelligence Eventstream sources: MQTT, Solace, ADX, weather & Azure Event Grid Eventstream CI/CD & REST APIs (Generally Available) Expanded regional availability for Eventstream's managed private endpoints (Secure Outbound) Connect to Eventstream using Microsoft Entra ID authentication Preview Real-Time Data Streams for Apache Kafka, Confluent Cloud, Amazon MSK & Amazon Kinesis Continuous improvements to Eventhouse Get Data Experience Build event-driven workflows with Azure and Fabric Events (Generally Available) Eventhouse OneLake availability now supports backfill Improved Activator alerts from Power BI End-to-end Real-Time Intelligence samples Synapse Data Explorer to Eventhouse migration tooling (Preview) Data Factory Enterprise readiness VNET Gateway support for Data pipelines New and updated Certified Connectors for Power BI and Dataflows Simplifying Data Ingestion with Copy Job Mirroring Mirroring for Azure SQL Database protected by a firewall(Preview) Mirroring for Azure Database for PostgreSQL Flexible Server (Preview) Open Mirroring UX improvements Transformations Save a new Dataflow Gen2 with CI/CD support from a Dataflow Gen1, Gen2, or Gen2 (CI/CD) Incremental Refresh for Dataflow Gen2 and new support to Lakehouse as destination with incremental refresh (Generally available) Check ongoing validation status of a Dataflow Gen2 with CI/CD support Orchestration Apache Airflow Job (Generally Available) OneLake file triggers for pipelines Variable libraries for pipelines (Preview) Spark Job Definition pipeline activity parameter support Azure Databricks jobs activity now supports parameters User data functions in Data pipelines (Preview) Data Factory pipelines now support up to 120 activities Dataflow Gen2 Dataflow Gen2 with CI/CD capabilities Data pipelines Mounting ADF (Preview) Mirrored database (Generally Available) Copy Job (Generally Available) Parameterization Parameterized connections in Data pipelines Table Name parameter support for data destinations AI-powered experiences Efficiently build and maintain your Data pipelines with enhanced capabilities for Copilot in Data Factory Data integration shared experiences Partner workloads Workloads (Generally Available) Power BI Designer (Generally Available) Newly released workloads Profisee MDM Workload Lumel PowerTables Workload SAS Decision Builder Workload (Preview) Striim SQL2 Fabric Workload Closing Power BI This month, we're excited to introduce a range of new features and improvements that will elevate your data analysis and visualization experience. Among the highlights are the Copy report object name feature, which simplifies locating and identifying objects within the PBIR folder, and the better storytelling with Data annotations in Power BI for PowerPoint, allowing you to add descriptive text directly to visualizations in your presentations. Additionally, we've made significant enhancements to Reference Lines, enabling you to add shade areas for all reference line types and support reference lines on the Y-axis for Line and stacked column charts. The Category enhancements for new cards bring new styles for categories, including table style and cards style, with conditional formatting options. Dive in to explore these exciting features and see how they can help you make the most of your data. To find out more about these features and more, head over to the Power BI March 2025 Feature Summary. https://youtu.be/2ft7aZKnaXY?si=_5tz-qmlC6NiL4nF Fabric Platform Variable library (Preview) We are excited to announce the upcoming preview of a new CI/CD feature - Variable library item in Microsoft Fabric. This feature is designed to provide a unified and centralized way to manage configurations, reducing the need for hardcoded values and simplifying your CI/CD processes, making it easier to manage configurations across different environments. What is the Variable Library? The Variable library is a new item type in Microsoft Fabric that allows users to define and manage variables at the workspace level, so they could soon be used across various workspace items, such as data pipelines (already available!), notebooks, Shortcut for lakehouse and more. Key features and benefits Environment-specific configurations: With Variable library, you can define different sets of values for your variables, e.g. one for each stage of your release pipeline. This means you can easily switch configurations based on the deployment environment, such as development, testing, and production. Centralized management: The Variable library provides a centralized location to manage all your configuration variables. This makes it easier to update and maintain configurations, ensuring consistency across your deployments. 2. Integration with CI/CD pipelines: The Variable library integrates seamlessly with your CI/CD practices – It’s a fabric item which is supported in Git integration and Deployment pipelines, and it has APIs to automate its management. 3. Support for multiple Variable types: The Variable library supports various variable types, including boolean, integer, number, string, GUID, and DateTime. This flexibility allows you to define and use variables that best suit your needs. Fabric items supporting Variable library The Variable library is supported soon through various fabric items: Data pipeline – Where Variable library can be used in dynamic content fields. Notebook – Which will support using variables in Notebook code natively. Shortcut for Lakehouse – Where Variable library will be used to parameterize the shortcut configuration. This includes managing connections to data sources and defining paths. More supporting items are underway, so stay tuned! The new CI/CD feature of the Variable library will be available in early April in Microsoft Fabric and requires admin approval. Try it out starting mid-April 2025 and be part of this exciting journey! What is Fabric Variable library? To learn more, refer to the Variable library documentation. New CI/CD features In addition to Variable library, the CI/CD platform is releasing a few important updates to improve the developer experience when setting up your CI/CD process in Fabric. Service Principal support The following set of APIs will start supporting Service Principal as well: Deployment pipelines APIs GitHub (through Git APIs) Calling Git APIs when working with Azure DevOps as your git provider is still being worked on and will be released in the upcoming few months. Please stay tuned and thank you for your patience! If you want to learn more about how to automate your CI/CD process in Fabric, you can use one of the following resources: Automate Git integration by using APIs Automate deployment pipeline by using Fabric APIs fabric-cicd Branch out to existing workspace When working in Fabric using Source control, we recommend working on your own feature branch in an isolated environment. In Fabric, this means you need another workspace. We have made this process easy with the ability to ‘branch out’, landing you directly in a new workspace, already connected and synced to the new branch. Now, we are making things even easier, as you can branch out to an existing workspace. If you have your own developer workspace, you don’t need to create another one to work on your next task. You can simply choose the same workspace, which already has all settings configured and data in place and continues working instantly after connecting it to the new branch. New Partner Workloads in Fabric Our amazing Fabric partners are delivering new capabilities fully integrated with Fabric as Workloads! This allows for the creation of new item types in shared workspaces for team collaboration. The Workload Hub is Fabric’s in-product marketplace for Partners that natively integrated with Fabric to provide our community with the ability to try and purchase leading data applications performing from data storage, transformation and connectivity tools to MDM platforms and visualization - all in the native Fabric experience we know and love! Add a workload in the workload hub describes how customers can add and manage workloads that have been published as Fabric Workloads. Check out the Partner Workloads section in this blog to learn which workloads were released in the past month. Workload Development Kit improvements The Microsoft Fabric Workload Development Kit is designed to enhance the Microsoft Fabric experience by integrating custom capabilities into Fabric. It allows developers and Microsoft Partners to create and publish workloads providing a seamless user experience without leaving Fabric. By using the Workload Development Kit, developers can embed new capabilities in Microsoft Fabric, streamline analytics processes, and explore new avenues for revenue generation. We are introducing several new functionalities that will empower our community to build more integrated workloads for Fabric independently. OneLake integration Workloads can now leverage the new OneLake integration, which allows for storing both structured and unstructured data directly as part of the partner workload item. This integration enables customers to access the data through standard OneLake APIs and expose it as a data item in the OneLake catalog. Importantly, all customer data is stored and protected within the customer tenant. Enhanced navigation experience We have improved the navigation experience over the Workload Development Kit. The community can now build workloads that open new tabs and navigate directly to other items within the workspace, providing a smoother and more intuitive user experience. Promoting Workload Solutions In response to requests from workload developers, we are excited to introduce support for embedding videos on the workload page. Additionally, we have rolled out new Fabric monetization guidelines that the community can utilize as part of the Fabric UX system. Real-Time Intelligence integration For workload developers, we have extended our example to include Real-Time Intelligence. Partners can now use the Event House selector in their workloads to offer customers rich real-time experiences. We have also included an example of how to use the real-time APIs and execute queries against the Event House. There are several additional changes that are helping the community build new workloads. Be sure to check out the Workload Development Kit - Announcing OneLake support and Developer Experience enhancements to learn more. Drive data discovery & curation with tags (Generally Available) Tags in Fabric enable flexibility in how you structure and manage your data estate and are now generally available. By providing the ability to apply additional metadata to items in Fabric, tags help admins and data owners categorize the data, enhancing the searchability and boosts success rates and efficiency for end users. To learn more on Tags in Microsoft Fabric refer to our documentation. In the OneLake catalog, the tagging experience has been further refined with context-aware applied tags. Now, when users filter data by tags, they will only see relevant tags applicable to their current context instead of browsing through the entire organization’s tag collection. This enhancement reduces clutter and improves efficiency when searching for tagged assets. Fabric Domains Microsoft Fabric’s data mesh architecture supports organizing data into domains & sub domains helping admins to manage and govern the data per business context with various delegated settings. Domains & sub domains structure enables data consumers to filter and discover content from the area most relevant to them. We have improved the visibility of the selected domain within OneLake catalog and enriched the domain image gallery with new, vivid imagery. Now, when users filter by domain in OneLake catalog they'll see the domain's cover image displayed in the background. This will create more clarity for users in their current context as they browse the catalog. Coming soon - create Tags in domains Domain admins will soon be able to create a list of tags in their domain. Item owners will be able to apply these tags to their items within the domain and data consumers will be able to use them to filter and search relevant data. OneLake catalog and Modern Get Data are now integrated into Excel for Windows It’s now even easier than ever to work with your Fabric data in Excel! The OneLake catalog is now integrated into the Modern Get Data experience in Excel, allowing users to effortlessly discover and connect to their Lakehouse or Warehouse assets. With just a few clicks, you can bring Fabric data into Excel for analysis and decision-making. This integration is currently available to customers enrolled in the M365 Insiders program on the Beta Channel (Insiders Fast), with plans for a broader rollout soon. For more information, refer to Announcing a new modern data connectivity and discovery experience in Dataflows. Enhanced search on tables and columns in the OneLake catalog Navigating through your data just got more efficient! Our new enhanced search capability allows users to search for sub-items such as tables, columns, and measures directly within their data items. This search functionality is available for users after they click on a specific item, ensuring quick access to relevant information without unnecessary navigation. New quick action for seamless navigation in the OneLake catalog To optimize workflows, you can now quickly open an item editor or viewer with a single click through a dedicated quick action instead of navigating to the Item Details page first. This improvement speeds up access to frequently used items and enhances productivity for all users. Purview Data Loss Prevention (DLP) policies for KQL and Mirrored DBs Security teams can use DLP policies to meet security and compliance requirements for sensitive data in the cloud. These policies leverage content scan to automatically detect the upload of sensitive information, and to trigger risk remediation actions (such as policy tips, audit logs and alerts) in semantic models and lakehouses. DLP policies support KQL DBs and Mirrored DBs (including Snowflake and Azure DBs). DLP coverage with this enhancement: KQL Database Mirrored Azure Cosmos DB Mirrored Azure DB for PostgreSQL Mirrored Azure SQL Database Mirrored Azure SQL Managed Instance Mirrored database Mirrored Snowflake Mirrored SQL Server Database Lakehouse (previously supported) Semantic model (previously supported) Get started with Data loss prevention policies for Fabric and Power BI to learn more. DLP policies restrict access action for lakehouses DLP Policies in Fabric help organizations detect sensitive information within their tabular data and surface it to end users and security administrators through policy tips, audit logs and alerts. The Restrict Access Action allows further control over data items once sensitive information has been discovered, by enabling security admins to define who can access the item upon DLP detection. This announcement means that once sensitive information is found within Fabric Lakehouse, unauthorized users will be blocked from accessing it until the data is removed. Items can be blocked from all users (excluding the data owners who always maintain access) or from guest users in the tenant. Learn more about Restrict Access in DLP in Fabric. Multi-tenant organization (MTO) (Generally Available) Support for multi-tenant organizations in Fabric is now generally available (GA). Entra ID users of type external member are supported across the Fabric platform. Users can authenticate, bring their own licenses from their home tenants and use Fabric workloads for development and consumption. There are some limitations when using Fabric with an external user. For more information refer to the Distribute Power BI content to external guest users with Microsoft Entra B2B documentation. https://youtu.be/qilDzEjPig4?si=qmQxTeaeBhjoPJIL OneLake OneLake security Managing granular data security across multiple applications and analytics engines is complex, often leading to either excessive restrictions or accidental exposure. That’s why we’re introducing OneLake security as a breakthrough in data protection. With OneLake security, you define access once, and Fabric enforces it consistently across all engines. Data owners can create security roles, grant precise permissions, and control access at the row and column level—for example, restricting Personally Identifiable Information (PII) while keeping other data available. This security propagates automatically, ensuring that whether users query via SQL or build Power BI reports, they only see what they’re authorized to access. OneLake security replaces the existing OneLake data access roles preview feature. Users start by creating OneLake security roles that grant access to specific data in a lakehouse. In addition to selecting tables and folders, OneLake security also allows for row and column level security to be defined. Using T-SQL, table access can be restricted to only specific rows where the T-SQL statement is true. To secure entire columns, roles can contain column level security definitions that block access to the sensitive columns. Assign members to your role to grant them access to only the allowed items in that role. With the role created, users can use any Fabric engine to query the data and see consistent results. Any queries through a Spark notebook are secured with OneLake security. The SQL Analytics Endpoint now uses the OneLake security definition to secure data when running in user’s identity mode. Semantic models can use Direct Lake mode to secure data using the security from OneLake. Even if users access the data in OneLake directly through API calls or OneLake file explorer, users are always restricted by the relevant OneLake security roles. OneLake security will be preview in the coming months, sign up for early access. Head over to OneLake security documentation for more information. External data sharing enhancements We have recently released several much-anticipated enhancements to the external data sharing feature. External data sharing allows in-place sharing of OneLake data across tenant boundaries. These updates include support for sharing multiple tables and folders, as well as entire Lakehouse schemas. Changes made to a shared lakehouse schema are automatically and immediately reflected in the consumer’s Lakehouse. Additionally, externally shared tables can now be consumed via the lakehouse’s SQL Analytics Endpoint and Semantic model, enabling seamless integration with Power BI reports. We have also expanded the types of data that can be shared to include KQL and SQL databases and introduced service principal support in the external data sharing APIs for automated management. For more details, check out the full announcement of external data sharing OneLake shared access signatures (SAS) (Generally Available) OneLake shared access signatures (SAS) are now generally available (GA)! OneLake SAS tokens provide secure-short-term, delegated access to your resources in OneLake, helping you share or distribute data through scoped-down SAS tokens. SAS tokens and user delegation keys are always backed by a Microsoft Entra identity and always limited to a 1-hour lifetime, ensuring that all access to your data is through an approved identity and for a limited period. You can learn more about how to expand your data estate with OneLake SAS in the OneLake documentation: What is a OneLake shared access signature. https://youtu.be/qilDzEjPig4?si=qmQxTeaeBhjoPJIL Data Engineering Write capabilities and PySpark support in Spark Connector for Fabric DW We are pleased to announce the addition of writing capabilities with the Fabric Spark connector for Fabric Data Warehouse (DW) in the Fabric Spark runtime. This connector utilizes a two-phase write process to a Fabric DW table. Initially, it stages the Spark dataframe data into intermediate storage, followed by the COPY INTO command to ingest the data into the Fabric DW table. This approach ensures scalability with increasing data volumes and supports multiple modes for writing data to a DW table. Additionally, we are excited to announce PySpark support for this connector. This means you no longer need to use a workaround to utilize this connector in PySpark, as it is now available as a native capability in PySpark. The connector will be included as a default library within the Fabric Runtime, eliminating the need for separate installation. To learn more about Spark Connector for Fabric Data Warehouse (DW), please refer to the documentation: Spark connector for Fabric Data Warehouse. Esri’s ArcGIS GeoAnalytics integration with Microsoft Fabric Spark (Preview) Esri is recognized as the global market leader in geographic information system (GIS) technology, location intelligence, and mapping, primarily through its flagship software, ArcGIS. Esri empowers businesses, governments, and communities to tackle the world's most pressing challenges through spatial analysis and location insight. We are pleased to share that Microsoft and Esri have partnered to bring spatial analytics into Microsoft Fabric and have launched public preview. Our collaboration with Esri introduces cutting-edge visual spatial analytics right within Microsoft Fabric Spark notebooks and Spark job definitions (across both Data Engineering and Data Science experiences). With its integrated product experience, it empowers Spark developers or data scientists to natively use ArcGIS capabilities to run GeoAnalytics functions and tools within Fabric Spark for transformation, enrichment, and pattern / trend analysis of data across different use cases without any need for separate installation and configuration. Example - How to transform the data with ArcGIS spatial function to uncover the pattern of interest, for instance summarizing the total number of policies of insured properties by hexagonal bins: Example - Understand the impact of natural hazards or current events on insured properties by bringing a dataset with probabilities of hurricane force winds and spatially joining it with insured properties. Spatial join links insured properties with wind speed probabilities, and with that for each property we would know the likelihood of hurricane force winds and can run predictive models to assess potential insurance claims. To learn more about this integration and capabilities, please refer to the documentation: ArcGIS GeoAnalytics for Microsoft Fabric (Preview). Deployment pipeline inside Spark Job Definition (Preview) The Spark Job Definition now supports the Deployment Pipeline. With this update, you can easily deploy your SJD (Spark Job Definition) item across different stages (Development, Testing, Production) and ensure that the proper state of the SJD item is synchronized across these stages. You can also customize the deployment with deployment rules to specify the default lakehouse and additional lakehouse of the SJD. Before triggering the deployment, you can verify the detail difference with the ‘Compare’ view. After the deployment is done, in the target stage/workspace, a new SJD item will be created based on the state from the source stage/workspace, and the association with other artifacts, such as Lakehouse and Environment, will also be set automatically. Deployment rule is supported to overwrite the default binding of default Lakehouse and Additional Lakehouse. By providing the Lakehouse ID, Lakehouse name, and the ID of the workspace where the Lakehouse is located, you can specify which Lakehouse should be set as the default in the target stage. You need to run the deployment after updating the deployment rule to make it effective. To learn more about this, please refer to the documentation: Spark Job Definition deployment pipeline support. Row-level and Column-level security in Spark We are pleased to announce the introduction of row and column level security for Spark within Microsoft Fabric. This update significantly enhances data governance by incorporating fine-grained security controls within Spark. Access control policies are established in OneLake security by specifying limiting factors for rows and columns in conjunction with tables during role definition. Spark uses these roles associated with the user executing the code and applies row and column data filtering accordingly before presenting the data to the user's code. These enhancements offer greater flexibility, stronger compliance, and simplified access management across Fabric’s unified data ecosystem. Introducing Pylance language support for Fabric Notebook Python developers using Fabric Notebook can now take advantage of Pylance, a powerful and feature-rich language server, to enhance their coding experience. With context-aware completions, better error detection, and improved code insights, Pylance makes PySpark and Python development smoother and more productive. Key Improvements with Pylance Smarter Auto-Completion moves beyond basic keyword and variable suggestions to context-aware completions, helping users quickly find relevant variable names and functions. Before Pylance With Pylance Enhanced Lambda Expression Support: More accurate completions within inline lambda functions, improving readability and efficiency for functional programming. Parameter Completions: Intelligent suggestions based on type hints and type inference, streamlining function calls. Improved Hover Information: More detailed insights when hovering over variables and code elements. Better Docstring Rendering: Clearer formatting and presentation of documentation strings for better readability. Error Markers & Semantic Highlighting: Improved error detection and code visualization, making debugging more intuitive. With Pylance in Fabric Notebook, writing Python and PySpark code is faster, more accurate, and more efficient. To learn more about Pylance in Notebook: Develop, execute, and manage Microsoft Fabric notebooks. Environment sharing across workspaces (Preview) You can now attach Environments from different workspaces in your Notebooks and Spark job definitions! This is made effortless with a brand-new explorer! Easily manage and utilize resources across multiple workspaces by exploring Environments from the workspaces you own, have access to, or that are shared with you by others. This feature provides flexibility in managing Environment permissions. Workspace viewers can use the Environment for running jobs without access to edit contents, while roles above workspace viewer can update the contents. To ensure only authorized users can access or update Environments, you can now manage Environments in one workspace, grant access to different users with different roles, or share the Environment with others with Read/Reshare/Edit permissions. Note that using an Environment from a different workspace does not break the compute and security configurations set by the admins. When you attach an environment from another workspace, both workspaces must have the same capacity and network security settings. Although you can select environments from workspaces with different capacities or network security settings, the session will fail to start. Furthermore, the compute configuration in that environment is ignored. Instead, the pool and compute configurations will default to the settings of your current workspace. To learn more about across attaching Environments: Create, configure, and use an environment in Fabric. Shortcuts now supported in Lakehouse Git metadata representation and in Fabric Deployment pipelines Git and Deployment Pipelines support for Lakehouse objects is a top ask across all our customer base, and we are excited to announce that now Shortcuts definitions under the Tables and Files section of lakehouse are supported in the Fabric Git/ALM platform. This is an exciting milestone, allowing customers to version and orchestrate Shortcuts using Fabrics Application Lifecycle Management capabilities. Now, Shortcuts are automatically exported as JSON metadata to the git repository connected to the workspaces. Also, you can modify Shortcut properties directly in git using your favourite authoring tool and import changes directly to the workspace. The Fabric Deployment pipelines work as expected, Shortcuts are now deployed across the stages defined in the pipeline configuration. This is the first step, on the upcoming releases, we will incrementally add support to additional object types under the Lakehouse, such as Folders, Tables, Views and more. Find out more information about the feature in the Lakehouse deployment pipelines and git integration documentation. Announcing Fabric User Data Functions (Preview) Fabric User Data Functions is a serverless platform that gives app developers and data engineers the ability to easily write and run applications on Fabric. User Data Functions empowers you to implement custom logic wherever you need to in your Fabric ecosystem by leveraging native integrations with Fabric data sources, Fabric Notebooks and Data pipelines. You can use your functions to perform data engineering tasks such as data validation or data cleaning, create integrations with external systems, or create re-usable function libraries. Learn more about this feature in the Fabric User Data Functions documentation. Link to YouTube video In this update, we added features that will help you make the best of your functions from the comfort of your browser. Portal editor You can now create, modify, delete or edit your functions directly in your browser. This experience gives you powerful tools to add to your functions code with the convenience of using the Fabric website portal. The editor features Intellisense and Pylance functionality to help you write quality Python code, as well as common editing functionality such as edit history, find and replace, and more. Insert code samples One of the most convenient features in the portal editor is the Insert Samples function that allows you to input code to quickly get started developing common use case patterns such as reading and writing to a Fabric data source, performing data transformations, and more. Add your favorite PyPI libraries! Another new feature is the Library management experience, which allows you to use the browser to add PyPI libraries into your project. Think of this as your requirements.txt file. You can select the library from a dropdown menu of names and choose a version that best suits your needs. The versions will be filtered to the ones compatible with the supported Python environment. New use cases and data sources! User Data Functions are deeply integrated with the Fabric ecosystem. You can now invoke your functions from different kinds of Fabric items such as Fabric notebooks, Power BI reports and Data pipelines. In addition to this, you can connect to Fabric data sources such as warehouses, lakehouses, SQL Databases, and Mirrored Databases for all your data applications. Learn more about this feature in the Fabric User Data Functions documentation. OPTIMIZE FAST and FSCK commands in Fabric Runtime 1.3 for Apache Spark (Generally Available) The Spark SQL FSCK REPAIR TABLE command and fast OPTIMIZE for V-Order are now available on Fabric Runtime 1.3 (Spark 3.5 / Delta 3.2). FSCK is designed to safely remove missing parquet files from the Delta transaction log, to restore table read consistency. This is not data recovery functionality, the missing parquet files and the data contained in it are lost. The command removes the references so the table can be back to a readable state. The command can also be run with a DRY RUN evaluation mode and will list all files that are missing in storage but still referenced by the Delta transaction log, to help you assess issues with the table before moving forward. Delta Lake’s OPTIMIZE VORDER can now be run with idempotency, meaning that previously V-Ordered parquet files that are already within the target file size won’t be considered for bin compaction. This significantly improves the performance of the OPTIMIZE command. Enable it by setting parquet.vorder.fast.optimize.enabled to true in the Spark session configuration directly on Notebooks, Spark Jobs or using Environments. Find more information in the Fabric Runtime 1.3 (GA) documentation. Fabric Spark Monitoring APIs (Preview) We’ve received valuable customer feedback emphasizing the need for API support to automate Spark job submission and monitoring. In response, we’re excited to introduce the preview of Fabric Spark Monitoring APIs—a robust set of tools designed to enhance observability and streamline the monitoring and management of Spark applications within Microsoft Fabric. To improve the developer experience, monitoring APIs for Fabric Spark applications are essential for optimizing performance, debugging issues, and ensuring efficient workload management. These APIs enable customers to automate Spark job management and monitor Spark jobs programmatically using APIs and SDKs. Key Capabilities of Fabric Spark Monitoring APIs With these APIs, users can: List all Spark applications within a workspace. Retrieve Spark applications for specific items, including Notebooks, Spark Job Definitions, and Lakehouse. Access detailed Spark application metrics using Livy ID. Leverage Spark History Server APIs to obtain execution metrics and job event details for a single Spark application, including jobs, stages, tasks, executors, and event logs. These capabilities empower users with greater automation, improved visibility, and deeper insights into Spark workloads within Fabric. Stay tuned for further enhancements as we continue refining these APIs based on customer feedback! Notebook Integration with User Data Functions (UDFs) (Preview) Introducing the preview of Notebook integration with User Data Functions (UDFs)! This new capability allows you to define custom logic and calculations that can be reused across multiple Notebooks, helping streamline workflows and enhance code modularity. With NotebookUtils, you can now seamlessly access and invoke UDFs directly from your Notebook code, making it easier than ever to integrate reusable functions into your data processing and analysis. Key Features & Scenarios Here’s how you can take advantage of UDFs within your Notebooks: Invoking a Function IntelliSense and autocomplete support function names, improving usability. To help you better understand UDF functions, we’ve introduced a help method display(myFunction.functionDetails). This provides a clear view of function details, including parameters and return types, ensuring that you can invoke functions correctly. Supported Languages This integration is available for Python, PySpark, Scala, and R, making it accessible across various data science and engineering workflows. For more details, check out our documentation: NotebookUtils (former MSSparkUtils) for Fabric. https://youtu.be/ktqKB4Bj1LQ?si=KPJ83e5D_jN8jbGF Data Science Copilot in Notebooks Agentic and UX enhancements The Copilot in Notebooks Agentic enhancement has introduced several significant improvements aimed at enhancing the user experience and productivity. One of the key enhancements is the enhanced conversation history, which allows users to maintain context and continuity in their interactions. Additionally, there have been chat and natural language output enhancements through enhanced algorithms that ensure more accurate and relevant code generation. The improved code generation capabilities now offer advanced reasoning for complex problem-solving, making it easier for users to write, debug, and understand code within the notebook environment. We’ve also added a new interaction modal for Copilot, on-cell and quick actions. The Copilot in Notebooks on-cell and quick actions introduces powerful features designed to streamline and enhance the coding workflow. The On-Cell Copilot Button, conveniently positioned above each notebook cell, allows users to perform advanced data manipulation tasks such as pivoting tables, joining datasets, and aggregating data based on specific criteria. Additionally, the Quick Actions Button, located just below the cell, simplifies tedious tasks using AI, such as fixing code errors and adding code comments. These features not only improve the efficiency of coding tasks but also enhance the overall user experience by providing intuitive and accessible tools directly within the notebook environment. With these enhancements, users can achieve more accurate and efficient results, making their coding process smoother and more productive. Fabric data agent Since the launch of AI skill in August 2024, we’ve improved conversational abilities, support for multiple data sources and so much more. To better reflect these enhanced agentic capabilities, AI skill is now Fabric data agent! Copilot and AI capabilities now available across all paid SKUs We’re thrilled to announce that we are removing the SKU requirement to access Copilot and AI capabilities across all paid SKUs. This means that customers on F2 and above will be able to use Copilot and AI features, such as Copilot in Power BI and Fabric data agent, to streamline workflows, generate insights, and drive impactful decisions. Fabric data agent integration with Azure AI Agent Service (Preview) We are excited to launch the integration of data agents in Fabric with Azure AI Agent Service from Azure AI Foundry. A critical component of Azure AI Agent Service is the ability to securely ground AI agent outputs in enterprise knowledge, ensuring responses are accurate, relevant, and contextually aware. Data agents in Fabric can retrieve knowledge using several specialized query language tools that help AI to generate SQL, KQL and DAX. By combining Fabric’s sophisticated data analysis over enterprise data with Azure AI Foundry’s cutting-edge GenAI technology, businesses can create custom conversational AI agents leveraging domain expertise. This seamless integration enables organizations to develop agents that are not only based on unstructured data in Azure AI Search or SharePoint but also integrate with structured and semantic data in Microsoft OneLake, thereby enhancing data-driven decision-making. Fabric data agent SDK (Preview) We are pleased to announce the preview of the Fabric data agent Python SDK. The Fabric data agent Python SDK library is a powerful tool designed to streamline the development and prototyping of AI assistants on the Fabric platform. It is intended for users who are looking to create, manage, and utilize Fabric data agents programmatically. The library provides a set of simple APIs that facilitate various operations, such as managing Fabric data agents and integrating various data sources for enhanced analysis and insights. It also makes it easier for users to interact with the Fabric data agent using the OpenAI Assistants API. This enables users to quickly prototype and experiment with Fabric data agents to refine their solutions. With the Fabric data agent Python SDK, users can automate workflows and reduce manual effort. Users can seamlessly create, update, and delete Fabric data agent artifacts, optimize resource configurations, and gain valuable insights from their data. To get started, users can leverage the comprehensive documentation and sample code provided with the SDK. By automating experimentation and validation processes, the Fabric data agent Python SDK ensures that developers can efficiently meet customer needs and deliver high-quality solutions, making it an invaluable tool for working with data agents. INSTALL the Fabric data agent. Data Warehouse AI functions in Data Warehouse AI functions are now available in private preview for Data Warehouse and Lakehouse SQL Endpoint, making it easier to bring AI-driven insights directly into your SQL workflows. With these built-in functions, you can summarize content, translate text, extract key data, analyze sentiment, and more - right from T-SQL. This eliminates the need for external processing, helping you streamline analysis and make faster, more informed decisions within your data warehouse. Figure 1: A few examples of AI functions usage. Refer to the detailed blog Functions in Data Warehouse to learn more and sign up for preview. Fabric User Data Functions in Data Warehouse Scalar Fabric functions are now available in private preview for Data Warehouse and Lakehouse SQL Endpoint, giving you the flexibility to extend SQL capabilities beyond built-in functions. With this feature, you can write custom functions in Python (and soon other languages) and invoke them directly through T-SQL, just like regular scalar user-defined functions. This allows you to bring complex logic, custom transformations, and advanced computations closer to your data, reducing the need for external processing. Refer to the detailed blog Functions in Data Warehouse to learn more and sign up for preview. Scalar SQL User-defined Functions Scalar SQL User-Defined Functions (UDFs) are now available in private preview for Data Warehouse and SQL analytics endpoint. Scalar SQL User-Defined Functions (UDFs) are a cornerstone of T-SQL programming, widely recognized and utilized for their ability to encapsulate business rules and calculations into a reusable code. This feature offers an efficient solution for promoting code modularity across T-SQL queries while natively leveraging Fabric Warehouse distributed engine. In an example below, by using four (4) different functions we can easily apply data masking logic on our customer table. Refer to the detailed blog Functions in Data Warehouse to learn more and sign up for preview. Developer Experiences in Fabric Warehouse IntelliSense for Collate Clause The COLLATE clause in Microsoft Fabric Warehouse and the SQL Analytics Endpoint of Lakehouse (LH) is essential for managing text-based data processing, ensuring accurate sorting, filtering, and comparisons. Given that Warehouse, SQL Analytics Endpoint of LH and other items support variations of case-insensitive and case-sensitive configurations, collation settings provide users with precise control over text handling in their workloads. By explicitly defining collation for VARCHAR and CHAR fields in table definitions, schema modifications, and queries, users can ensure consistency across transformations. The support for DATABASE_DEFAULT collation further simplifies schema management, allowing tables to inherit database-level settings for ease of administration and alignment with organizational standards. The collation feature in Microsoft Fabric Warehouse and SQL Analytics Endpoint is enhanced with IntelliSense and syntax highlighting, providing a more intuitive and efficient development experience. IntelliSense offers real-time suggestions, validation for collation names, helping users avoid syntax errors and ensuring compatibility with supported collation settings. Syntax highlighting further improves readability by visually distinguishing collation clauses, making it easier to identify and manage collation settings in CREATE TABLE, ALTER TABLE, SELECT, and CTAS statements. These features streamline query development, reduce errors, and enhance productivity when working with case-sensitive and case-insensitive data configurations across Microsoft Fabric's SQL environments. A few examples are: CREATE TABLE [SampleData_CI_UTF8] ( [SampleID] INT NOT NULL, -- Unique identifier for each sample [SampleValue] VARCHAR(50) COLLATE Latin1_General_100_CI_AS_KS_WS_SC_UTF8, -- Sample value with specified collation [CreatedAt] DATETIME2(6) NOT NULL -- Timestamp for when the sample was created ); CREATE TABLE [SampleData_DB_Default] ( [SampleID] INT NOT NULL, -- Unique identifier for each sample [SampleValue] VARCHAR(50) COLLATE DATABASE_DEFAULT, -- Sample value with specified collation [CreatedAt] DATETIME2(6) NOT NULL -- Timestamp for when the sample was created ); INSERT INTO [SampleData_CI_UTF8] ([SampleID], [SampleValue], [CreatedAt]) VALUES (1, 'Sample1', GETDATE()), -- Inserting sample data (2, 'Sample2', GETDATE()); INSERT INTO [SampleData_DB_Default] ([SampleID], [SampleValue], [CreatedAt]) VALUES (1, 'Sample1', GETDATE()), -- Inserting sample data (2, 'Sample2', GETDATE()); -- Collate in Select Select [SampleValue] COLLATE Latin1_General_100_CI_AS_KS_WS_SC_UTF8 from [SampleData_DB_Default] Select [SampleValue] COLLATE DATABASE_DEFAULT from [SampleData_CI_UTF8] --Collate in CTAS CREATE TABLE SampleDataCreate AS Select [SampleValue] COLLATE Latin1_General_100_CI_AS_KS_WS_SC_UTF8 as SampleValue_CI_UTF8 from [SampleData_CI_UTF8] CREATE TABLE SampleDataCreate AS Select [SampleValue] COLLATE DATABASE_DEFAULT as SampleValue_CI_UTF8 from [SampleData_CI_UTF8] -- Collate in ALTER Table Add New Column ALTER TABLE SampleData_CI_UTF8 ADD Column4 VARCHAR(10) COLLATE Latin1_General_100_CI_AS_KS_WS_SC_UTF8 NULL; ALTER TABLE SampleData_DB_Default ADD Column4 VARCHAR(10) COLLATE DATABASE_DEFAULT NULL; JSON data in OPENROWSET These IntelliSense and grammar updates make working with OPENROWSET, JSON more seamless and efficient. With improved syntax highlighting, and query validation, SQL development is now faster and more error-free. Live templates Writing T-SQL queries efficiently is crucial for database developers. Live Templates are predefined code snippets that can be inserted into your T-SQL editor with minimal effort. They help reduce repetitive coding, enforce best practices, and improve developer productivity. Key benefits: Faster query development Standardized SQL formatting Reduced errors in repetitive tasks Expand and collapse objects properly in filter and search We’re making search and filter experiences more intuitive! Now, objects that meet your search and filter criteria will automatically expand, giving you instant visibility into relevant data. Moving forward, we’ll expand only what’s required—keeping your object explorer clean and efficient unless no matches are found. Artifact Status Bar The Git item status bar component offers a comparable experience to the status bar in the workspace. When accessing the item page, you can view the details of the connection between the workspace and the Git repository, such as: The name of the branch to which the workspace is connected The time of the last sync event between the workspace and the repository A hyperlink to the most recent commit on the branch. Cancel query on closing editor Handling long-running queries efficiently is crucial for a seamless warehouse experience. To improve user control, we’re introducing an enhanced query cancellation prompt that ensures users can make informed decisions when closing the editor while a query is still executing. How it works 1. Prompting users when closing an active query If a user attempts to close the editor while a query is running, they will see a confirmation message: ‘Do you want to cancel the query?’ Yes: the query is canceled, and the editor closes immediately. No: the editor closes, but the query continues running in the Queries section, ensuring users don’t lose progress. This helps prevent accidental cancellations while still allowing users to exit the editor seamlessly. 2. Customizing future prompts When a user chooses Yes to cancel a query for the first time, they will see an additional prompt: ‘Do you want to see this message next time?’ Yes: the prompt will continue appearing for future query cancellations. No: the editor will automatically cancel queries without showing the confirmation message moving forward. This setting is user-specific, meaning each user can customize their experience individually. Users who opt out will no longer be interrupted, making their workflow faster and more efficient. Why this matters: Prevents accidental query cancellation – Ensures users don’t unintentionally stop important queries. Reduces interruptions – Users can choose whether they want to see the prompt in the future, keeping their workflow smooth. Personalized experience – Every user gets the flexibility to decide how they handle active queries when closing the editor. Show query editor shortcuts Navigating your data warehouse just got faster! Our keyboard shortcuts UI enhance efficiency across key areas: Object explorer – Quickly browse and manage database objects. Ribbon – Access essential commands with a single keystroke. T-SQL editor – Speed up query writing and execution. Results grid – Seamlessly filter, copy, and analyze query results. Why use keyboard shortcuts? Faster navigation – Reduce mouse dependency and move through objects quickly. Increased productivity – Execute queries, format code, and manage results seamlessly. Streamlined workflow – Spend less time on repetitive actions and more on data insights. SQL audit logs (Preview) We are excited to announce that SQL audit logs are now in preview in Microsoft Fabric Data Warehouse! Audit Logs provide a detailed record of warehouse activity, capturing essential information such as when events occur, which triggered them, and the T-SQL statement behind the event. This feature is crucial for security and compliance, helping organizations monitor access patterns, detect anomalies, and meet regulatory requirements. Previously, tracking warehouse events manual effort making security audits and forensic investigations cumbersome. With native audit logging in Fabric Data Warehouse, organizations gain automated, tamper-resistant logging, simplifying security operations. Whether you need to investigate unauthorized access, analyze query execution trends, or ensure adherence to governance policies, SQL Audit Logs provide the transparency and control needed to safeguard your data. Fabric Data Warehouse item permissions We are thrilled to announce the introduction of enhanced sharing capabilities in Microsoft Fabric Data Warehouse! With these new updates, you can now grant additional permissions to monitor queries and audit activities, providing deeper visibility into warehouse operations. These enhancements allow organizations to delegate access more effectively, enabling security teams, auditors, and operations personnel to track query performance, analyze workloads, and audit activity with the appropriate level of control. By improving security, governance, and operational insights, these new capabilities help organizations maintain compliance while ensuring efficient data management. What Permissions Can Be Assigned to Users? When it comes to assigning permissions, it's important to understand the different types of permissions available and their implications. Here are some core and custom permissions that can be assigned to users: Read: Allows users to view the data. Write: Grants users the ability to modify the data. Reshare: Enables users to share data with others. Monitor: Provides users with the ability to monitor database activities and kill sessions. Audit: Allows users to configure and access audit logs. Restore: Permits users to perform in-place restores of data. How to Assign Permissions Assigning permissions can be done through user interfaces on the share dialog: After you click on the option, we will be able to see the options surfaced on the dialog menu: You can also validate the permissions on the Manage Permissions option on the share menu: Item permissions are a fundamental aspect of data management, providing the necessary controls to secure, comply, and collaborate effectively. By understanding and implementing permissions like Monitor, Reshare, and Audit, organizations can enhance their data security posture and foster a collaborative environment. OneLake Security for Lakehouse Analytics SQL Endpoints Private Preview As data governance becomes more central, we're thrilled to introduce OneLake Security for SQL Analytics Endpoints, now available in Private Preview! This powerful capability simplifies how security is enforced in Microsoft Fabric by letting you configure access once in OneLake, and have that security respected across your SQL workloads. With this release, organizations can now govern data at scale with a consistent, unified approach—whether you're implementing centralized security controls or need granular SQL-based permissions. OneLake Security empowers teams to secure, simplify, and scale access across your Lakehouse architecture. Two Flexible Access Modes to Match Your Needs OneLake Security introduces two distinct access modes for SQL Analytics Endpoints: 1. User Identity Mode In this mode, the SQL Endpoint uses the signed-in user’s identity to access data in OneLake. It fully honors the RLS (Row Level Security), CLS (Column Level Security), and OLS (Object Level Security) rules defined in OneLake. Great for: Organizations that want centralized control and alignment with data lake-level security. 2. Delegated Identity Mode Here, the SQL Endpoint uses the workspace or artifact owner's identity to connect to OneLake. This enables traditional SQL-based security management with full support for GRANT, custom roles, masking, and other advanced database security features. Great for: SQL administrators and advanced use cases needing fine-grained SQL access control. User Identity x Delegated Mode Capability User Identity Mode Delegated Identity Mode Access Context Signed-in User Datawarehouse Owner OneLake RLS/CLS/OLS Enforced Not Enforced SQL GRANT on Tables Not Allowed Allowed SQL GRANT on Views/Procedures Allowed Allowed Dynamic Data Masking Not Supported Supported Custom SQL Roles Not Supported Supported With OneLake Security for SQL Endpoints, Microsoft Fabric continues its mission to make data governance intuitive and scalable. Whether you're building a self-service analytics culture or enforcing strict compliance policies, OneLake Security gives you the tools to do both—with confidence. Real-Time Intelligence Eventstream sources: MQTT, Solace, ADX, weather & Azure Event Grid Eventstream is a powerful feature in Fabric Real-time Intelligence that allows users to ingest, transform, and route real-time data streams to various destinations within Fabric. We are excited to announce the addition of five new sources and additional sample data streams. These new sources enhance Eventstream's streaming capabilities, enabling seamless data ingestion and real-time transformation across various data streams. Let’s dive into the functionalities of each connector and explore how they can benefit your data processing needs. MQTT connector: Connect to an MQTT broker, subscribe to specific topics, and stream data from those topics into Eventstream. Solace PubSub+: Read messages from a Solace PubSub+ Event Broker cluster and stream them into Eventstream for real-time data processing. Azure Data Explorer: Streams data from an Azure Data Explorer database in real-time into Fabric. Real-time Weather: Ingest live weather data for a selected city into Eventstream, including temperature, humidity, and wind speed. Azure Event Grid namespace: Stream MQTT, IoT, or any messages from Azure Event Grid namespace to Eventstream. Sample data streams: Kickstart your streaming projects with additional pre-built sample streams, including real-time bus tracking data and S&P 500 stock market data. These new connectors open a world of possibilities for data integration and analytics. To learn more about real-time streaming and processing in Fabric Eventstream, be sure to check out the Fabric Eventstream overview documentation. Can’t find your data sources? Let us know! Send us an email at [email protected] or fill out our survey. Eventstream CI/CD & REST APIs (Generally Available) Collaborating on data streaming solutions can be challenging, especially when multiple developers work on the same Eventstream item. Version control challenges, deployment inefficiencies, and conflicts often slow down development. Since introducing Fabric CI/CD tools for Eventstream last year, many customers have streamlined their workflows, ensuring better source control and seamless versioning. Now, we’re excited to announce the general availability (GA) of Eventstream CI/CD and REST APIs—making these capabilities even more accessible and powerful for all users. Key benefits of leveraging CI/CD tools in Eventstream: Enhanced Collaboration: With Git integration, developers can use GitHub or Azure DevOps to sync with the Fabric workspace and work in parallel on the same Eventstream item without conflicts. Streamlined Deployments: The Deployment pipeline feature accelerates and standardizes Eventstream deployments to various stages, such as testing and production workspace, with minimal manual effort in the Fabric UI. This ensures a more efficient and reliable deployment process. Automated Workflows: The availability of Eventstream REST APIs allows developers to build fully automated CI/CD pipelines and integrate external applications. This capability ensures quality, reliability, and productivity for data streaming projects, reducing manual intervention and potential errors. Increased Productivity: By leveraging these powerful CI/CD tools, teams can focus more on transformation within Eventstream and less on managing conflicts and deployment issues. This ultimately boosts overall productivity and project success. Overall, the GA of CI/CD and REST APIs for Fabric Eventstream empowers users to achieve a more efficient, reliable, and collaborative development experience. To learn more about Eventstream’s CI/CD, check out: Eventstream CI/CD - Git Integration and Deployment pipeline Eventstream REST API Expanded regional availability for Eventstream's managed private endpoints (Secure Outbound) Managed Private Endpoint (MPE) is a Fabric platform security feature that allows Fabric items, such as Eventstream, to securely connect to data sources behind firewalls or protected networks. Since we introduced this integration last year, many customers have relied on it to establish secure outbound connections between Eventstream and their data sources. This feature ensures that your data is transmitted securely over a private network, allowing you to fully harness the power of real-time streaming and high-performance data processing in Eventstream. The diagram below shows a typical setup using MPE in Eventstream. Managed Private Endpoints are now available in even more regions, making it easier for organizations worldwide to build secure, scalable streaming solutions. The table below lists supported regions for Eventstream’s MPE: To learn more about Managed Private Endpoints, check out the Connect to Azure resources securely using MPE in Eventstream. Connect to Eventstream using Microsoft Entra ID authentication We’re excited to introduce Microsoft Entra ID authentication for Eventstream’s Custom Endpoint! This feature enhances security by eliminating the need for SAS keys or connection strings, reducing the risk of unauthorized access. Instead, Entra ID authentication ensures that user permissions are directly tied to Fabric workspace access, allowing only authorized users to send and fetch data from Eventstream. The screenshot demonstrates how this feature works in Eventstream’s Custom Endpoint! Additionally, if you’re using an Azure resource like Azure Logic Apps with a system-assigned or user-managed identity, you can now assign Fabric workspace permissions to that identity. This enables Azure Logic Apps to seamlessly connect to Eventstream using Managed Identity authentication. The screenshot demonstrates how to enable identity in the Azure Logic Apps and assigning permission in the Fabric workspace. To learn more about Entra ID authentication in Eventstream’s Custom Endpoint, refer to our documentation Connect to Eventstream using Microsoft Entra ID authentication. Preview Real-Time Data Streams for Apache Kafka, Confluent Cloud, Amazon MSK & Amazon Kinesis Transforming data in Eventstream requires an actual schema derived from incoming data, which can slow down development and troubleshooting. To simplify this process, we are excited to introduce Data preview, a major usability enhancement for third-party connectors in Fabric Eventstream, including Apache Kafka, Confluent Cloud, Amazon Managed Streaming for Apache Kafka (MSK) and Amazon Kinesis Data Streams. With this new capability, users can preview a snapshot of their source data directly within Eventstream Edit mode and process data with inferred schemas. Why Data preview matters The Data preview feature allows users to: Enable Eventstream to infer the schema from incoming data, making it easier to configure operators such as filtering and aggregation. Verify if an Eventstream source is properly configured. Preview of real-time data snapshots to confirm data is ingesting as expected. How It Works Using Data preview in Eventstream is simple: Select a source connector in Eventstream (e.g., Confluent Cloud). Click on ‘Data Preview’ tab to view a snapshot of the source data. Change and match the source data format for preview. The screenshot below shows a snapshot of the Confluent data streams in Eventstream Edit mode: With Data preview, teams can build, test, and deploy Eventstream items faster and with greater confidence. Get started today and experience the power of real-time processing for your third-party connectors in Eventstream! Continuous improvements to Eventhouse Get Data Experience There are a few different methods to get data into the Real-Time Intelligence workload, depending on your organization’s needs. Data can be pushed or pulled into an Eventstream from one of the many connectors and then landed in an eventhouse. Alternatively, there are several ways to directly ingest data into Eventhouse. Data can be directly from: Local files Azure storage Amazon S3 Event hub Eventstreams OneLake Get data in Eventhouse offers a step-by-step process, guiding you from importing the data, through inspecting the incoming data, creating or editing the destination table schema to exploration of the ingested result. Over the past few months, our team has been working tirelessly to bring new features to the Get Data wizard, creating a simpler interface, quicker navigation, and added automation, all aimed at delivering a better user experience and improved performance. The main changes introduced: Automated schema optimization: Since the Eventhouse engine is highly optimized for datetime and string operations, in certain cases mapping imported data to these data types can offer a significant boost in query performance. By introducing usage of the inferred schema plugin, in most applicable cases an optimal data mapping will be inferred and automatically applied to your data, enhancing Eventhouse’s query and indexing performance. For instance, it can detect columns storing Unix date-time values as long and convert them to datetime. Similarly, it might recognize that a column named ‘id’ using a long type should be converted to string. Any such automatic mapping changes are clearly reflected in the schema editor, with a lightbulb icon and a short explanation of the mapping applied. Changes can easily be manually reverted using the ‘Type’ dropdown menu, although this is usually not recommended. Simplified schema inspection: The Schema Inspection step allows users to preview the destination table schemas, modify the schema if any change is needed or extract the KQL commands for table and schema creation. Based on the feedback received from our customers, we have improved the schema Inspection experience to make schema preview and edit a seamless experience. Users can now easily switch between the schema preview; command viewer and schema edit modes with an intuitive switcher experience. We have also simplified the inference file, formats, mapping and nested JSON options to make them more accessible. 2. Real-Time data sampling: We have graduated Sample data option in the inspect step to real-time data sampling. This allows users to preview how the data would look like when ingested, even before finalizing the data schema. 3. Automatic detection of header row for CSV files: The Get Data wizard now seamlessly detects if a CSV has a header row and uses it for column names. The column data type is inferred based on the data in the remaining rows. This makes the process of schema definition, when your file has headers a painless process. Build event-driven workflows with Azure and Fabric Events (Generally Available) Azure and Fabric Events, a powerful capability that allows organizations to capture, process, and respond to events across Microsoft Fabric is now generally available. With these events, businesses can integrate event-driven solutions into their workflows, enabling seamless automation, enhanced observability, and faster decision-making. What are Azure and Fabric Events? Azure and Fabric Events offer a capability within Real-Time Intelligence that enables you to: Ingest events that are available in Microsoft Fabric like Onelake events, Azure blob storage events, Job events,Workspace item events Filter those events using rich filtering capabilities on event schema properties. Integrate those events to consumers in Microsoft Fabric like Activator for setting event-based alerts or Eventstream to stream events to other destinations. With Azure and Fabric Events, organizations can reduce latency, improve operational efficiency, and build scalable event-driven applications. To learn more, please go to Azure and Fabric Events documentation and for the full announcement, refer to the announcement blog. Eventhouse OneLake availability now supports backfill Eventhouse OneLake availability allows creating a delta parquet representation of data in Eventhouse. Previously, when you turned availability ON, only new data was made available in OneLake, with no backfill of existing data. This could cause inconsistencies between the data in Eventhouse and OneLake. Now, OneLake Availability supports backfill, making all existing and new data in Eventhouse available, regardless of when you turn it ON. This is the default behavior when you enable availability via the UI. Learn more about Eventhouse OneLake availability. Improved Activator alerts from Power BI We’ve made it easier than ever to create and manage Activator alerts on your Power BI reports. We’ve redesigned the Power BI ‘Set Alert’ experience so that you can conveniently manage your alerts entirely within your reports, without having to open Activator. We’ve also streamlined the experience so that you can set up an alert with fewer steps. To check out the new experience, open a Power BI report and select ‘Add Alert’ on a visual, or choose ‘Set Alert’ from the ribbon. Figure 2: The improved ‘Set Alert’ experience in Power BI makes it easier than ever to create and manage Activator alerts on your reports. End-to-end Real-Time Intelligence samples We are excited to announce a brand new RTI sample experience which allows you to create a fully working end-to-end RTI flow within seconds. The sample flow allows you to explore the main features of Real-Time Intelligence with sample data. It provides a comprehensive end-to-end solution, demonstrating how Real-Time Intelligence components work together to stream, analyse, and visualize real-time data in a real-world context. You can access the samples from the RTI workload home page. Select the sample scenario of your choice. Choice of Bike rentals or S&P 500 Stocks data. Create a sample solution with all RTI items within seconds. Learn more about End-to-end sample. Synapse Data Explorer to Eventhouse migration tooling (Preview) The next generation of Azure Synapse Data Explorer offering is evolving to become Eventhouse, part of Real-Time Intelligence in Microsoft Fabric. For customers looking to migrate to Eventhouse, we are providing a migration tooling that allows you to seamlessly migrate Synapse Data Explorer cluster to an Eventhouse in Fabric. The migration process is performed using Fabric REST API endpoints. The recommended steps for performing the migration are as follows: Validate: Use the Validate migration to Eventhouse endpoint to check whether the Azure Synapse Analytics Data Explorer cluster can be migrated to an eventhouse. Migrate: Use the Migrate to Eventhouse with the migrationSourceClusterUrl payload to create an eventhouse with the migration source cluster URL. The process runs asynchronously to create a new eventhouse and migrate all databases from the source cluster to the eventhouse. Monitor: Use the Monitor migration progress to track the progress of your migration. Verify: Verify the migration by checking the eventhouse state is Running, and that the migrated databases appear in the KQL database list. To learn more about how the API endpoints can be called directly or in an automated PowerShell script refer to our migration tool documentation. https://youtu.be/pluk-b8XVj4?si=WYcsvCJG4SIhzaHf Data Factory Enterprise readiness VNET Gateway support for Data pipelines Support for data pipeline functionality on the VNet data gateway is now available in preview. The VNet data gateway facilitates connections to data sources that are either behind firewalls or accessible within your virtual network. This feature enables the execution of data pipeline activities on the VNet data gateway, ensuring secure connections to data sources within the VNet. Unlike on-premises data gateways, VNet data gateways are managed by Microsoft, consistently updated, support auto-scaling, and deactivate when not in use, making it cost-effective. To learn more, refer to the documentation: What is a virtual network (VNet) data gateway? Best-in-class connectivity and enterprise data movement In the fast-evolving data integration landscape, Data Factory continues to enhance the existing connectors to provide a seamless, high-performance experience. With a focus on improving connector efficiency and expanding capabilities, recent updates have made significant advancements to Salesforce and Lakehouse connectors. These improvements not only boost performance but also enable more sophisticated data handling, ensuring that enterprises can extract, transform, and load data with greater accuracy and efficiency. Performance improvement in Salesforce connector in data pipelines Salesforce is a critical data source for many organizations, housing valuable customers and business data. To enhance data movement efficiency, Data Factory has introduced performance optimization in the Salesforce connector for pipelines. Optimization allows you to fetch the data concurrently from Salesforce by leveraging the parallelism capability, thus significantly reducing extraction times for large datasets. Lakehouse connector now supports deletion vector and column mapping for delta tables in data pipelines The Lakehouse connector in Data Factory has been upgraded to provide deeper integration with delta table. Two major new capabilities enhance data processing workflows: 1. Support for deletion of vector Delta table uses deletion vectors to track deleted records efficiently without physically removing them from storage. With this new feature in the Lakehouse connector, users can: Read Delta tables while respecting deletion vector, ensuring that deleted records are automatically excluded from queries. Improve performance by leveraging soft deletions instead of physical file modifications, making data updates and maintenance more efficient. Enable compliance with data retention policies by retaining historical data for auditability while ensuring deleted records are filtered out from active queries. 2. Column mapping support for delta tables Delta table's column mapping capability allows for more flexible schema evolution, ensuring that changes in table structure do not disrupt data workflows. With column mapping support in the Lakehouse connector, users can: Read from an existing delta Lake table with column mapping name/id mode enabled Write to existing delta lake table with column mapping name/id mode enabled Auto-create table with column mapping name mode enabled when sink table does not exist and source dataset columns contain special chars & whitespaces. Auto-create table with column mapping name mode enabled when table action is overwriting schema and source dataset columns contain special chars & whitespaces. These enhancements ensure that data engineers can work with delta tables more efficiently, improving data governance, performance, and maintainability. To learn more about how to Configure Lakehouse in a copy activity refer to our documentation. New and updated Certified Connectors for Power BI and Dataflows As a developer and data source owner, you can create connectors using the Power Query SDK and have them certify through the Data Factory Connector Certification Program. Certifying a Data Factory connector makes the connector available publicly, out-of-box, Microsoft Fabric Data Factory and Microsoft Power BI in the following experiences This month we are happy to list the newly updated certified connectors that are part of the Microsoft Data Factory Connector Certification Program. Be sure to check the documentation for each of these connectors so you can see what’s new with each of them. New connectors ADP Analytics Dynatrace Grail DQL Updated connectors Anaplan Asana BQE Core BuildingConnected Delta Sharing SolarWinds Service Desk Supermetrics Windsor Worksplace Analytics Zendesk Data Simplifying Data Ingestion with Copy Job Copy Job is making data ingestion simpler, faster, and more intuitive than ever and is now generally available. Whether you need batch or incremental data movement, Copy Job provides the flexibility to meet your needs while ensuring a seamless experience. Since its preview last September, Copy Job has rapidly evolved with several powerful enhancements. Let’s dive into what’s new! Public API & CICD support Fabric Data Factory now offers a robust Public API to automate and manage Copy Job efficiently. Plus, with Git Integration and Deployment pipelines, you can leverage your own Git repositories in Azure DevOps or GitHub and seamlessly deploy Copy Job with Fabric’s built-in CI/CD workflows. VNET gateway support Copy Job now supports the VNet data gateway in Preview! The VNet data gateway enables secure connections to data sources within your virtual network or behind firewalls. With this new capability, you can now execute Copy Job directly on the VNet data gateway, ensuring seamless and secure data movement. Upsert to Azure SQL Database & overwrite to Fabric Lakehouse By default, Copy Job appends data to ensure no changed data is lost. But now, you can also choose to upsert data directly into Azure SQL DB or SQL Server and overwrite data in Fabric Lakehouse tables. These options give you greater flexibility to tailor data ingestion to your specific needs. Enhanced usability & monitoring We’ve made Copy Job even more intuitive based on your feedback, with the following enhancements: Column mapping for simple data modification to storage as destination store. Data preview to help select the right incremental column. Search functionality to quickly find tables or columns. Real-time monitoring with an in-progress view of running Copy Jobs. Customizable update methods & schedules before job creation. More connectors, more possibilities! More source connections are now available, giving you greater flexibility for data ingestion with Copy Job. And we’re not stopping here—even more connectors are coming soon! What’s next? We’re committed to continuously improving Copy Job to make data ingestion simpler, smarter, and faster. Stay tuned for even more enhancements! Learn more about Copy Job in: What is Copy job in Data Factory Mirroring Mirroring for Azure SQL Database protected by a firewall (Preview) You now can mirror Azure SQL Databases protected by a firewall. Using either the VNet data gateway or the on-premises data gateway for mirroring is available. The data gateway facilitates secure connections to your source databases through a private endpoint or from a specific private network. Learn more about Mirroring for Azure SQL Database from Microsoft Fabric Mirrored Databases from Azure SQL Database. Mirroring for Azure Database for PostgreSQL Flexible Server (Preview) Database Mirroring now supports replication of your Azure Database for PostgreSQL Flexible Server into Fabric! Now you can continuously replicate data in near real-time from your Flexible Server instance to Fabric OneLake. This enables seamless data integration, allowing you to leverage Fabric’s analytics capabilities while ensuring your PostgreSQL data remains up to date. By mirroring your PostgreSQL data into Fabric, you can enhance reporting, analytics, and machine learning workflows without disrupting your operational database. To learn more, please reference the PostgreSQL mirroring preview blog. Open Mirroring UX improvements We’ve made improvements to our end-to-end in-product experience for Open Mirroring. With these changes, you can now create a Mirror DB and start uploading or dragging and dropping parquet and CSV files. It’s now easier than ever to get started with building your own Open Mirror source and allow you to test our replication technology before productionizing with APIs. Once your files are uploaded, you can also upload changes and updates to the data with the __rowMarker__ field specified to our change data capabilities. Transformations Save a new Dataflow Gen2 with CI/CD support from a Dataflow Gen1, Gen2, or Gen2 (CI/CD) Customers often would like to recreate an existing dataflow as a new dataflow Gen2 (CI/CD), getting all the benefits of the new GIT and CI/CD integration capabilities. Today, to accomplish this, they need to create the new Dataflow Gen2 (CI/CD) item from scratch and copy-paste their existing queries or leverage the Export/Import Power Query template capabilities. This, however, is not only inconvenient due to unnecessary steps, but it also does not carry over additional dataflow settings. Dataflows in Microsoft Fabric now includes a ‘Save as’ feature in preview, that in a single click lets you save an existing dataflow Gen1, Gen2 or Gen2 (CI/CD) as a new Dataflow Gen2 (CI/CD) item. Incremental Refresh for Dataflow Gen2 and new support to Lakehouse as destination with incremental refresh (Generally available) Incremental Refresh for Dataflow Gen2 is now generally available! Incremental Refresh for Dataflow Gen2 allows you to refresh only the buckets of data that have changed, rather than reloading the entire dataset on every dataflow refresh. This not only saves time but also reduces resource consumption, making your data operations more efficient and cost-effective. These new capabilities are designed to help you to be successful with your data integration needs and be as efficient as possible. Try it out today in your fabric workspace! Learn more about Incremental Refresh in Dataflow Gen2: Incremental refresh in Dataflow Gen2. Check ongoing validation status of a Dataflow Gen2 with CI/CD support When you click Save & run in Dataflow Gen2 with CI/CD support, the process that gets triggered is two-fold: Validation: it’s a background process where your Dataflow gets validated against a set of rules. If it passes all validations and no errors are returned, then it’ll be successfully saved. Run: Using the latest published version of the Dataflow, a refresh job gets triggered to run the Dataflow. If you only wish to trigger the validation process, you only need to click the ‘Save’ button. What if you want to check the status of the validation? You now have a new entry point in the home tab of the ribbon called Check validation which you can click at any time to give you information of the ongoing validation or the result of a previous validation run. Be sure to give this a try whenever you want to check the results of a save validation. Orchestration Apache Airflow Job (Generally Available) The Apache Airflow job in Microsoft Fabric is now generally available, providing a fully integrated Apache Airflow runtime for developing, scheduling, and monitoring Python-based data workflows using Directed Acyclic Graphs (DAGs). What's New: Introducing Fabric runtime versioning for Apache Airflow job – This includes Fabric runtime version 1.0, which comes with Apache Airflow 2.10.4 and Python 3 as the default runtime. Public API – APIs are now available to interact with Apache Airflow jobs for seamless management. Git Integration & Deployment pipeline support – Users can utilize their Git repositories (Azure DevOps/GitHub) and deploy with Fabric’s built-in CI/CD workflows. Diagnostic logs – Users can access Apache Airflow generated logs through the Apache Airflow job UI for enhanced observability. Learn more about Apache Airflow job in Microsoft Fabric in What is Apache Airflow job? OneLake file triggers for pipelines The Fabric Data Factory team is thrilled to announce that the pipeline trigger experience is now generally available (GA) and now includes access to files in OneLake! This exciting new improvement to pipeline triggers in Fabric Data Factory means that you can now automatically invoke your pipeline when files or folders have files that arrive, delete, or rename! We’ve previously supported Azure blob file events in Fabric Data Factory like ADF & Synapse but now that Fabric users are leveraging OneLake as the primary data hub, we’re excited to see the pipeline patterns that you’ll build using OneLake file triggers! Variable libraries for pipelines (Preview) One of the most requested features in Fabric Data Factory has been support for modifying values when deploying workspace changes between environments using Fabric CICD. To accommodate this, ask, we have integrated pipelines into the new Fabric platform feature called Variable Libraries. With Variable Libraries, you can assign variables to unique values based on different environments, i.e. dev, test, prod. Then when you promote your factory to high environments, you can use different values from the library providing the ability to change values when pipelines are promoted to new environments. This new preview feature will be super useful not just for CICD but also generically allows you to replace hardcoded values with variables anywhere in your pipelines to achieve the same functionality as global parameters in Azure Data Factory as well. Spark Job Definition pipeline activity parameter support The Spark Job Definition (SJD) activity in Data Factory allows you to create connections to your Spark Job Definitions and run them from your data pipeline. And we are excited to announce that parameterization is now supported in this activity! You will find this update in the Advanced settings where you can configure your SJD parameters and run your Spark Job Definitions with the parameter values that you set, allowing you to override your SJD artifact configurations. Azure Databricks jobs activity now supports parameters Parameterizing data pipelines to support generic reusable pipeline models is extremely common in the big data analytics world. Fabric Data Factory provides end-to-end support for these patterns and is now extending this capability to the Azure Databricks pipeline orchestration activity. Now when you select ‘Jobs’ as the source of your ADB action, you can send parameters to your ADF job allowing maximum flexibility and power of your orchestration jobs. User data functions in Data pipelines (Preview) User Data Functions are now available in preview within Data pipeline's Functions activity. This new feature is designed to enhance your data processing capabilities by allowing you to create and manage custom functions tailored to your specific needs. Key Highlights Custom functionality: User Data Functions enable you to define custom logic and calculations that can be reused across multiple Data pipelines. This allows for more flexible and efficient data processing. Integration in data pipelines: You can add User Data Functions as activities within your Data pipelines. This is done by selecting the Functions activity in the pipeline editor, choosing your User Data Functions as the type, and providing any necessary input parameters. Check out our documentation to learn more about how to User Data Functions in your data pipelines. Data Factory pipelines now support up to 120 activities We’ve increased the default activity limit from 80 activities to 120 activities! You can now utilize an additional 40 activities to build more complex pipelines for better error handling, branching, and other control flow capabilities. https://youtu.be/spe7ZMImHH0?si=uf7NmQ6CNp2iDJLO Dataflow Gen2 Dataflow Gen2 with CI/CD capabilities You can now add Dataflow refresh activities to your pipelines in Fabric Data Factory that include the new version of Dataflows: Check out our documentation on Dataflow Gen2 with CI/CD and Git integration to learn more. Data pipelines Data pipelines have supported CI/CD capabilities and REST APIs support is now generally available. The team just added Service Principal Name (SPN), and Variable libraries support for Data pipelines. Check out our documentation on CI/CD for Data pipelines and REST API capabilities for Data pipelines to learn more. Mounting ADF (Preview) CI/CD and REST APIs support is now available for the Azure Data Factory item (Mounting ADF). Mirrored database (Generally Available) The mirrored database’s CI/CD support is now Generally Available. Learn more from CI/CD for mirrored databases. The REST APIs support has been Generally Available including the SPN support. Check out our documentation on Mirroring Public REST APIs. Copy Job (Generally Available) The Copy job item’s CI/CD and APIs support is now Generally Availability. This includes SPN support for Copy job. Check out our documentation on CI/CD for Copy job to learn more. Parameterization Parameterized connections in Data pipelines Enhancing your Data Integration experience What are Parameterized Connections? Parameterization of data connections in Data pipelines allows you to specify values for connection placeholders dynamically. This means you can pre-create data connections for various sources, such as Azure Blob Storage, SQL Server or any other data source supported by data pipelines, and reference them through data pipeline’s dynamic expressions at runtime. This feature empowers you to create more flexible and adaptable data pipelines, capable of connecting to different instances of data connections of the same type, such as SQL Server, without altering the pipeline definition. Key benefits: Flexibility: Use the same data pipeline definition to dynamically connect to various instances of data connections. Efficiency: Minimize the need for multiple pipeline definitions, reducing complexity and maintenance effort. Scalability: Easily manage and scale your data integration processes by leveraging dynamic expressions to handle connection values. How it works: During the pipeline run, dynamic expressions within the data pipelines specify values for the connection placeholders, enabling seamless integration with pre-created data connections. This innovation ensures that your data pipelines are not only more efficient but also highly customizable to meet your specific requirements. We believe this new feature will significantly enhance your data processing capabilities and streamline your workflows. We can't wait for you to experience the benefits of parameterized connections in your data integration projects. Table Name parameter support for data destinations In Dataflow Gen2, you can create parameters, they serve as a way to easily store and manage a value that can be reused throughout your Dataflow. Major feedback that we’ve heard from our users is the lack of this capability in the Data destination experience for Dataflow Gen2. Thanks to the feedback, we’re now introducing the first support for parameters in the data destination experience where you can set a parameter to be used for the Table name of your destination. This is available to all destinations that support this field and we’re working on extending this support to other areas of the data destination experience. Try out this new capability and let us know what you think. AI-powered experiences Efficiently build and maintain your Data pipelines with enhanced capabilities for Copilot in Data Factory In November 2024, we announced the preview of 3 innovative capabilities in Copilot for Data Factory (Data pipeline). Today, we are excited to make these features generally available, with enhancements to make your data integration even more efficient and effortless. Check out the blog post on Efficiently build and maintain your Data pipelines with Copilot for Data Factory: new capabilities and experiences to learn more. Effortlessly generate your data pipelines: Understand your business intent and effortlessly translate it into data pipeline activities to build your data integration solutions. In the enhanced capability of Copilot, we can easily build more complex Data pipeline activities e.g. switch activity, metadata driven pipeline, etc. You can also update your pipeline settings and configurations in batches with multiple activities! Efficiently troubleshoot error messages in your data pipeline Copilot. Diagnose and resolve pipeline errors more intuitively by providing clear and actionable summary. Easily understand your complex data pipelines: Understand your complex pipeline configurations effortlessly by getting a clear and intuitive summary provided by Copilot. Data integration shared experiences Partner workloads We are thrilled to share the significant updates and newly released workloads for this month, showcasing the incredible efforts and innovative work by our amazing partners. Workloads (Generally Available) Osmos AI Data Wrangler Automate Data Ingestion with Osmos AI Data Wrangler for Microsoft Fabric As enterprises scale AI adoption, they must ensure all their data is AI-ready. However, enterprise data is often messy - semi-structured or unstructured, arriving in inconsistent formats from customers, partners, suppliers, and internal systems. Traditional ETL pipelines require constant engineering effort to adapt to schema drift, missing fields, and poor data quality, slowing down AI and analytics initiatives. Osmos AI Data Wrangler enables autonomous data transformation as a Workload on Microsoft Fabric. Osmos’ agentic AI automates data ingestion by intelligently cleaning, transforming, and validating your data, seamlessly normalizing messy bronze data into silver tables in your Lakehouse. Get it now: Osmos AI Data Wrangler for Microsoft Fabric Osmos AI Data Wrangler is now generally available! Osmos’ AI Data Wrangler is now generally available (GA), featuring self-configuration capabilities, featuring self-configuring Wrangler Context. This feature allows Wranglers to understand your business rules and apply it to data transformations. With the new Wrangler Context feature, businesses can auto-configure their Wranglers using existing documentation and code snippets, giving you easy-to-configure high performance data wranglers quickly. Why enterprises choose Osmos AI for data ingestion & transformation Osmos helps businesses across retail, manufacturing, finance, and audit unlock millions in savings while accelerating insights and new market opportunities. ✔ Unify clean data into Lakehouse - Normalize disparate data coming from multiple internal and external sources into clean, actionable SQL-ready data. ✔ Improve data quality - AI-powered validation ensures structured, clean, and accurate data for analytics and decision-making. ✔ Streamline workflows - Eliminate manual data cleanup by enabling Wranglers to autonomously learn from documentation and business rules. ✔ Enhance AI & analytics readiness - Deliver trusted, structured data ready for AI-driven insights and enterprise analytics. Check out the Osmos AI Data Wrangler with Wrangler Context video. Power BI Designer (Generally Available) Microsoft Fabric in collaboration with PowerBI.tips, are excited to share that Power BI Designer is now generally available! You won’t want to miss out on this time saving application. Say goodbye to bland, cookie-cutter reports and hello to dazzling, highly stylized masterpieces. What’s Power Designer All About? Power Designer is sleek, intuitive, and fun, making designing reports feel less like work and more like unleashing your inner artist. It’s packed with features that’ll have you saying, ‘Why didn’t I have this sooner?’ Get it now: Power Designer Workload. Let’s dive into the magic: Craft themes like a pro: create detailed theme files for your Power BI reports with ease. Customize colors, fonts, and styles to match your brand. Real-Time visual vibes: watch your Power BI visuals update live as you build your style. Multi-page: Add background images to each page with a snap, transforming your reports into polished, magazine-worthy layouts. AI-Powered: Let AI take the wheel with auto-placement of visuals in your multipage templates. Preview: Test your shiny new theme on reports already published in your workspaces with the preview feature. Now that Power Designer has officially been released, it’s time to jump in and start creating. Head to your Fabric Workspaces, fire up Power Designer, and let your imagination run wild. Ready, set, design! Let’s make some report magic happen! Learn more at PowerBI.tips Designer YouTube video: Introducing Power Designer: Unleash Your Inner Report Wizard! Newly released workloads Profisee MDM Workload Profisee has introduced the first native Master Data Management (MDM) workload within Microsoft Fabric, seamlessly integrating its MDM platform into Fabric’s environment. This breakthrough allows users to manage and unify enterprise data without leaving the familiar Fabric interface. Get it now: Profisee MDM Workload for Microsoft Fabric. By embedding MDM capabilities directly into Microsoft Fabric, Profisee empowers organizations to: Ensure data consistency – Align and integrate data from multiple sources while enforcing standardized data governance within Fabric’s OneLake. Enhance data quality – Leverage intelligent matching, merging, and standardization to create trusted ‘golden records’ for your most critical business data. Streamline workflows – Manage data within Fabric itself, eliminating the need for external tools and reducing context switching. Accelerate AI & analytics initiatives – Deliver high-quality, consumable data ready to fuel enterprise applications of generative AI and advanced analytics. This integration marks a major advancement in data management, offering a unified platform for data stewardship, modeling, and governance. The native MDM experience in Fabric simplifies the transformation of raw data into actionable insights— driving business outcomes faster than ever possible before. For organizations leveraging AI and advanced analytics, having trusted, consumable (aka ‘gold medallion’) data is critical. Profisee’s deep integration with Microsoft Fabric ensures businesses can confidently rely on their data to make informed decisions and drive innovation. This collaboration between Profisee and Microsoft represents a significant leap forward, enabling enterprises across industries to unlock insights, fuel opportunities, and finally bring their data into the age of AI to become data-driven at scale. YouTube Video – Welcome to Profisee's featured workload in Microsoft Fabric Figure: Profisee enables your medallion architecture to deliver consumable, trusted data for AI and analytics. Figure: Profisee can match, merge and standardize data from different sources. Lumel PowerTables Workload The Lumel Fabric workload is now in preview. PowerTable allows business users to build no-code, writeback-enabled table apps on Microsoft Fabric. It connects LIVE (with bi-directional sync) to your cloud data warehouse tables in platforms such as Fabric SQL, Fabric Data Warehouse, Azure SQL, Databricks, Snowflake, Amazon Redshift, Google BigQuery, and PostgreSQL. Get it now: Lumel PowerTables Workload. There are 3 key use cases for PowerTable: Build tabular apps for data typically maintained in Excel such as product price lists, contract trackers, project status updates, etc. Manage master data / reference data / meta data supporting your reporting and planning applications. PowerTable supports forward-looking master data (unlike your ERP and MDM platforms) such as prospective customers or products that are not yet launched. Build applications on top of your semantic models and facilitate user data input and data writeback. PowerTable delivers the following features: Bulk insert and bulk edit records Cell-level commenting and collaboration Change log with audit trail Support for slowly changing dimensions Row-level CRUD user permissions Field-level permissions Sequential, multi-level approvals Rule-based approvals Outlook and Teams notifications Workflow automation Triggers and cascading updates Webhook integration … and more Unlike products like Airtable or Smartsheet, which typically struggle to handle more than tens or hundreds of thousands of rows, PowerTable is highly scalable and can support millions of rows. This is possible because our architecture separates the user interface from storage and compute and uses pushdown SQL statements to perform all processing on the underlying database of your choice. Visit our website www.lumel.com to learn more. PowerTables Introduction in action video. SAS Decision Builder Workload (Preview) Announcing SAS Decision Builder in preview, a new workload from longtime Microsoft partner and analytics vendor SAS. This powerful SaaS solution is designed to help organizations automate, optimize, and scale their decision-making processes in real-time, whether they are managing complex business rules, integrating machine learning models, or using Python code. Get it now: SAS Decision Builder workload. There are numerous use cases across many industries, including financial services (loan approvals, financial products), manufacturing (product quality), and public sector (fraud identification, help with choosing a government service). Decision Flow in SAS Decision Builder Key features: Construct business rules and decisioning flows: Access and process data from all your sources to create decisions that align with business goals. Robust governance capabilities: Test, validate, schedule, run and monitor decisions within the unified Fabric environment. Adjust decisions as business needs evolve. Add value to your ML Models: Call your ML models within your Decision Flow to further enhance your decisions. Integrate Python code files into decisions: Bring advanced logic and flexibility into your automated processes. Partner workloads - SAS Video Demo Getting started: Access SAS Decision Builder from the Workloads tab within your Microsoft Fabric instance. Select the SAS Decision Builder Workload Hub page, select ‘Add Workload,’ and transact through the Azure Marketplace process. Once complete, you can start building your decisions with SAS Decision Builder. Striim SQL2 Fabric Workload Microsoft Fabric’s robust ecosystem empowers partners to integrate custom capabilities through the Fabric Workload Development Kit. Striim proudly introduces a Fabric workload designed for real-time data ingestion, processing, and AI-driven analytics—delivering seamless integration, scalability, and actionable insights. The Striim workload builds on the Open Mirroring capabilities provided by Fabric to provide users a managed copy of their SQL mirrored data in the Fabric environment. Get it now: SQL2Fabric-Mirroring Embedded workload integration: Striim embeds directly into the Microsoft Fabric Workload Hub, ensuring real-time data movement orchestration without leaving the Microsoft Fabric environment. Enterprises can leverage this integration for cohesive data workflows and enhanced discoverability. Real-Time data replication & streaming: Striim offers sub-second latency ingestion from SQL Server, Oracle, PostgreSQL, MongoDB, and Databricks. These structured pipelines optimize data for Azure OpenAI, Fabric Copilot, and Vector Search—delivering AI-ready insights. Enterprise security & performance: With end-to-end encryption, access control, and high-throughput event streaming via change data capture (CDC), Striim ensures secure, scalable, and low-latency performance across hybrid environments. Integrated workload with Fabric: Striim seamlessly embeds within the Microsoft Fabric Workload Hub, enabling enterprises to orchestrate and automate real-time data movement without leaving the Fabric ecosystem. AI-Optimized streaming: Striim delivers structured data pipelines tailored for Azure OpenAI, Fabric Copilot, and Vector Search. It ensures fresh, AI-ready data for machine learning models. Use cases Hybrid Cloud migration & replication: Continuously mirror on-premises SQL Server databases into Fabric OneLake for real-time analytics. Cross-Cloud data synchronization: Seamlessly integrate data across Azure, AWS, and GCP, reducing data silos. AI-Driven insights & automation: Provide fresh data pipelines for AI-powered recommendations, fraud detection, and predictive analytics. Fabric Workload automation: Utilize the Workload Hub to streamline AI-powered data transformations and event-driven automation. Closing We hope that you enjoy the update! Be sure to join the conversation in the Fabric Community and check out the Fabric documentation to get deeper into the technical details. As always, keep voting on Ideas to help us determine what to build next. We are looking forward to hearing from you!
