Welcome to the March Feature Summary!
From the innovative Variable library (Preview) to the powerful Service Principal support in the CI/CD features, there's a lot to explore. Dive in and discover how the new Partner Workloads in Fabric bring cutting-edge capabilities to your workspace. Plus, enhanced OneLake security ensures your data is protected. And don't miss out on the expanded regional availability for Eventstream's managed private endpoints, making it easier for organizations worldwide to build secure, scalable streaming solutions.
With FabCon kicking off today, the announcements are rolling in! Get ready to explore these features and more in the March 2025 updates for Fabric!
Contents
- Power BI
- Fabric Platform
- Variable library (Preview)
- New CI/CD features
- New Partner Workloads in Fabric
- Workload Development Kit improvements
- Drive data discovery & curation with tags (Generally Available)
- Fabric Domains
- OneLake catalog and Modern Get Data are now integrated into Excel for Windows
- Enhanced search on tables and columns in the OneLake catalog
- New quick action for seamless navigation in the OneLake catalog
- Purview Data Loss Prevention (DLP) policies for KQL and Mirrored DBs
- Multi-tenant organization (MTO) (Generally Available)
- OneLake
- OneLake security
- OneLake shared access signatures (SAS) (Generally Available)
- Data Engineering
- Write capabilities and PySpark support in Spark Connector for Fabric DW
- Esri’s ArcGIS GeoAnalytics integration with Microsoft Fabric Spark (Preview)
- Deployment pipeline inside Spark Job Definition (Preview)
- Row-level and Column-level security in Spark
- Introducing Pylance language support for Fabric Notebook
- Environment sharing across workspaces (Preview)
- Shortcuts now supported in Lakehouse Git metadata representation and in Fabric Deployment pipelines
- Announcing Fabric User Data Functions (Preview)
- OPTIMIZE FAST and FSCK commands in Fabric Runtime 1.3 for Apache Spark (Generally Available)
- Fabric Spark Monitoring APIs (Preview)
- Notebook Integration with User Data Functions (UDFs) (Preview)
- Data Science
- Copilot in Notebooks Agentic and UX enhancements
- Fabric data agent
- Copilot and AI capabilities now available across all paid SKUs
- Fabric data agent integration with Azure AI Agent Service (Preview)
- Fabric data agent SDK (Preview)
- Data Warehouse
- AI functions in Data Warehouse
- Fabric User Data Functions in Data Warehouse
- Scalar SQL User-defined Functions
- Developer Experiences in Fabric Warehouse
- JSON data in OPENROWSET
- SQL audit logs (Preview)
- Fabric Data Warehouse item permissions
- OneLake Security for Lakehouse Analytics SQL Endpoints Private Preview
- Real-Time Intelligence
- Eventstream sources: MQTT, Solace, ADX, weather & Azure Event Grid
- Eventstream CI/CD & REST APIs (Generally Available)
- Expanded regional availability for Eventstream's managed private endpoints (Secure Outbound)
- Connect to Eventstream using Microsoft Entra ID authentication
- Preview Real-Time Data Streams for Apache Kafka, Confluent Cloud, Amazon MSK & Amazon Kinesis
- Continuous improvements to Eventhouse Get Data Experience
- Build event-driven workflows with Azure and Fabric Events (Generally Available)
- Eventhouse OneLake availability now supports backfill
- Improved Activator alerts from Power BI
- End-to-end Real-Time Intelligence samples
- Synapse Data Explorer to Eventhouse migration tooling (Preview)
- Data Factory
- Enterprise readiness
- VNET Gateway support for Data pipelines
- New and updated Certified Connectors for Power BI and Dataflows
- Simplifying Data Ingestion with Copy Job
- Mirroring
- Mirroring for Azure SQL Database protected by a firewall(Preview)
- Mirroring for Azure Database for PostgreSQL Flexible Server (Preview)
- Open Mirroring UX improvements
- Transformations
- Save a new Dataflow Gen2 with CI/CD support from a Dataflow Gen1, Gen2, or Gen2 (CI/CD)
- Incremental Refresh for Dataflow Gen2 and new support to Lakehouse as destination with incremental r...
- Check ongoing validation status of a Dataflow Gen2 with CI/CD support
- Orchestration
- Apache Airflow Job (Generally Available)
- OneLake file triggers for pipelines
- Variable libraries for pipelines (Preview)
- Spark Job Definition pipeline activity parameter support
- Azure Databricks jobs activity now supports parameters
- User data functions in Data pipelines (Preview)
- Data Factory pipelines now support up to 120 activities
- Dataflow Gen2
- Dataflow Gen2 with CI/CD capabilities
- Data pipelines
- Mounting ADF (Preview)
- Mirrored database (Generally Available)
- Copy Job (Generally Available)
- Parameterization
- Parameterized connections in Data pipelines
- Table Name parameter support for data destinations
- AI-powered experiences
- Efficiently build and maintain your Data pipelines with enhanced capabilities for Copilot in Data Fa...
- Data integration shared experiences
- Partner workloads
- Workloads (Generally Available)
- Power BI Designer (Generally Available)
- Newly released workloads
- Profisee MDM Workload
- Lumel PowerTables Workload
- SAS Decision Builder Workload (Preview)
- Striim SQL2 Fabric Workload
- Closing
Power BI
This month, we're excited to introduce a range of new features and improvements that will elevate your data analysis and visualization experience. Among the highlights are the Copy report object name feature, which simplifies locating and identifying objects within the PBIR folder, and the better storytelling with Data annotations in Power BI for PowerPoint, allowing you to add descriptive text directly to visualizations in your presentations.Additionally, we've made significant enhancements to Reference Lines, enabling you to add shade areas for all reference line types and support reference lines on the Y-axis for Line and stacked column charts. The Category enhancements for new cards bring new styles for categories, including table style and cards style, with conditional formatting options. Dive in to explore these exciting features and see how they can help you make the most of your data.
To find out more about these features and more, head over to the Power BI March 2025 Feature Summary.
https://youtu.be/2ft7aZKnaXY?si=_5tz-qmlC6NiL4nF
Fabric Platform
Variable library (Preview)
We are excited to announce the upcoming preview of a new CI/CD feature - Variable library item in Microsoft Fabric. This feature is designed to provide a unified and centralized way to manage configurations, reducing the need for hardcoded values and simplifying your CI/CD processes, making it easier to manage configurations across different environments.What is the Variable Library?
The Variable library is a new item type in Microsoft Fabric that allows users to define and manage variables at the workspace level, so they could soon be used across various workspace items, such as data pipelines (already available!), notebooks, Shortcut for lakehouse and more.
Key features and benefits
AI-generated content may be incorrect.">Environment-specific configurations: With Variable library, you can define
different sets of values for your variables, e.g. one for each stage of your release pipeline. This means you can easily switch configurations based on the deployment environment, such as development, testing, and production.
- Centralized management: The Variable library provides a centralized location to manage all your configuration variables. This makes it easier to update and maintain configurations, ensuring consistency across your deployments.
Fabric_March_2025_Feature_Summary
AI-generated content may be incorrect.">2. Integration with CI/CD pipelines: The Variable library integrates seamlessly with your CI/CD practices – It’s a fabric item which is supported in Git integration and Deployment pipelines, and it has APIs to automate its management.
AI-generated content may be incorrect.">
3. Support for multiple Variable types: The Variable library supports various variable types, including boolean, integer, number, string, GUID, and DateTime. This flexibility allows you to define and use variables that best suit your needs.
Fabric items supporting Variable library
The Variable library is supported soon through various fabric items:
- Data pipeline – Where Variable library can be used in dynamic content fields.
- Notebook – Which will support using variables in Notebook code natively.
- Shortcut for Lakehouse – Where Variable library will be used to parameterize the shortcut configuration. This includes managing connections to data sources and defining paths.
The new CI/CD feature of the Variable library will be available in early April in Microsoft Fabric and requires admin approval. Try it out starting mid-April 2025 and be part of this exciting journey!
What is Fabric Variable library? To learn more, refer to the Variable library documentation.
New CI/CD features
In addition to Variable library, the CI/CD platform is releasing a few important updates to improve the developer experience when setting up your CI/CD process in Fabric.Service Principal support
The following set of APIs will start supporting Service Principal as well:
Calling Git APIs when working with Azure DevOps as your git provider is still being worked on and will be released in the upcoming few months. Please stay tuned and thank you for your patience!If you want to learn more about how to automate your CI/CD process in Fabric, you can use one of the following resources:
- Automate Git integration by using APIs
- Automate deployment pipeline by using Fabric APIs
- fabric-cicd
When working in Fabric using Source control, we recommend working on your own feature branch in an isolated environment. In Fabric, this means you need another workspace. We have made this process easy with the ability to ‘branch out’, landing you directly in a new workspace, already connected and synced to the new branch.
Now, we are making things even easier, as you can branch out to an existing workspace.
AI-generated content may be incorrect.">
If you have your own developer workspace, you don’t need to create another one to work on your next task. You can simply choose the same workspace, which already has all settings configured and data in place and continues working instantly after connecting it to the new branch.
AI-generated content may be incorrect." width="519" height="557">
New Partner Workloads in Fabric
Our amazing Fabric partners are delivering new capabilities fully integrated with Fabric as Workloads! This allows for the creation of new item types in shared workspaces for team collaboration.The Workload Hub is Fabric’s in-product marketplace for Partners that natively integrated with Fabric to provide our community with the ability to try and purchase leading data applications performing from data storage, transformation and connectivity tools to MDM platforms and visualization - all in the native Fabric experience we know and love!
Add a workload in the workload hub describes how customers can add and manage workloads that have been published as Fabric Workloads.
AI-generated content may be incorrect.">
Check out the Partner Workloads section in this blog to learn which workloads were released in the past month.
Workload Development Kit improvements
The Microsoft Fabric Workload Development Kit is designed to enhance the Microsoft Fabric experience by integrating custom capabilities into Fabric. It allows developers and Microsoft Partners to create and publish workloads providing a seamless user experience without leaving Fabric.By using the Workload Development Kit, developers can embed new capabilities in Microsoft Fabric, streamline analytics processes, and explore new avenues for revenue generation.
We are introducing several new functionalities that will empower our community to build more integrated workloads for Fabric independently.
OneLake integration
Workloads can now leverage the new OneLake integration, which allows for storing both structured and unstructured data directly as part of the partner workload item. This integration enables customers to access the data through standard OneLake APIs and expose it as a data item in the OneLake catalog. Importantly, all customer data is stored and protected within the customer tenant.
Enhanced navigation experience
We have improved the navigation experience over the Workload Development Kit. The community can now build workloads that open new tabs and navigate directly to other items within the workspace, providing a smoother and more intuitive user experience.
Promoting Workload Solutions
In response to requests from workload developers, we are excited to introduce support for embedding videos on the workload page. Additionally, we have rolled out new Fabric monetization guidelines that the community can utilize as part of the Fabric UX system.
Real-Time Intelligence integration
For workload developers, we have extended our example to include Real-Time Intelligence. Partners can now use the Event House selector in their workloads to offer customers rich real-time experiences. We have also included an example of how to use the real-time APIs and execute queries against the Event House.
There are several additional changes that are helping the community build new workloads. Be sure to check out the Workload Development Kit - Announcing OneLake support and Developer Experience enhancements to learn more.
Drive data discovery & curation with tags (Generally Available)
Tags in Fabric enable flexibility in how you structure and manage your data estate and are now generally available.By providing the ability to apply additional metadata to items in Fabric, tags help admins and data owners categorize the data, enhancing the searchability and boosts success rates and efficiency for end users.
AI-generated content may be incorrect.">
To learn more on Tags in Microsoft Fabric refer to our documentation.
In the OneLake catalog, the tagging experience has been further refined with context-aware applied tags. Now, when users filter data by tags, they will only see relevant tags applicable to their current context instead of browsing through the entire organization’s tag collection. This enhancement reduces clutter and improves efficiency when searching for tagged assets.
Fabric Domains
Microsoft Fabric’s data mesh architecture supports organizing data into domains & sub domains helping admins to manage and govern the data per business context with various delegated settings. Domains & sub domains structure enables data consumers to filter and discover content from the area most relevant to them.We have improved the visibility of the selected domain within OneLake catalog and enriched the domain image gallery with new, vivid imagery.
Now, when users filter by domain in OneLake catalog they'll see the domain's cover image displayed in the background. This will create more clarity for users in their current context as they browse the catalog.
AI-generated content may be incorrect.">
Coming soon - create Tags in domains
Domain admins will soon be able to create a list of tags in their domain.
Item owners will be able to apply these tags to their items within the domain and data consumers will be able to use them to filter and search relevant data.
OneLake catalog and Modern Get Data are now integrated into Excel for Windows
It’s now even easier than ever to work with your Fabric data in Excel! The OneLake catalog is now integrated into the Modern Get Data experience in Excel, allowing users to effortlessly discover and connect to their Lakehouse or Warehouse assets. With just a few clicks, you can bring Fabric data into Excel for analysis and decision-making. This integration is currently available to customers enrolled in the M365 Insiders program on the Beta Channel (Insiders Fast), with plans for a broader rollout soon.For more information, refer to Announcing a new modern data connectivity and discovery experience in Dataflows.
AI-generated content may be incorrect.">
Enhanced search on tables and columns in the OneLake catalog
Navigating through your data just got more efficient! Our new enhanced search capability allows users to search for sub-items such as tables, columns, and measures directly within their data items. This search functionality is available for users after they click on a specific item, ensuring quick access to relevant information without unnecessary navigation.
AI-generated content may be incorrect.">
AI-generated content may be incorrect.">
New quick action for seamless navigation in the OneLake catalog
To optimize workflows, you can now quickly open an item editor or viewer with a single click through a dedicated quick action instead of navigating to the Item Details page first. This improvement speeds up access to frequently used items and enhances productivity for all users.
AI-generated content may be incorrect.">
Purview Data Loss Prevention (DLP) policies for KQL and Mirrored DBs
Security teams can use DLP policies to meet security and compliance requirements for sensitive data in the cloud. These policies leverage content scan to automatically detect the upload of sensitive information, and to trigger risk remediation actions (such as policy tips, audit logs and alerts) in semantic models and lakehouses. DLP policies support KQL DBs and Mirrored DBs (including Snowflake and Azure DBs).
AI-generated content may be incorrect.">
DLP coverage with this enhancement:
- KQL Database
- Mirrored Azure Cosmos DB
- Mirrored Azure DB for PostgreSQL
- Mirrored Azure SQL Database
- Mirrored Azure SQL Managed Instance
- Mirrored database
- Mirrored Snowflake
- Mirrored SQL Server Database
- Lakehouse (previously supported)
- Semantic model (previously supported)
DLP policies restrict access action for lakehouses
DLP Policies in Fabric help organizations detect sensitive information within their tabular data and surface it to end users and security administrators through policy tips, audit logs and alerts.
The Restrict Access Action allows further control over data items once sensitive information has been discovered, by enabling security admins to define who can access the item upon DLP detection.
This announcement means that once sensitive information is found within Fabric Lakehouse, unauthorized users will be blocked from accessing it until the data is removed. Items can be blocked from all users (excluding the data owners who always maintain access) or from guest users in the tenant.
Learn more about Restrict Access in DLP in Fabric.
AI-generated content may be incorrect.">
Multi-tenant organization (MTO) (Generally Available)
Support for multi-tenant organizations in Fabric is now generally available (GA). Entra ID users of type external member are supported across the Fabric platform. Users can authenticate, bring their own licenses from their home tenants and use Fabric workloads for development and consumption. There are some limitations when using Fabric with an external user.For more information refer to the Distribute Power BI content to external guest users with Microsoft Entra B2B documentation.
https://youtu.be/qilDzEjPig4?si=qmQxTeaeBhjoPJIL
OneLake
OneLake security
Managing granular data security across multiple applications and analytics engines is complex, often leading to either excessive restrictions or accidental exposure. That’s why we’re introducing OneLake security as a breakthrough in data protection.With OneLake security, you define access once, and Fabric enforces it consistently across all engines. Data owners can create security roles, grant precise permissions, and control access at the row and column level—for example, restricting Personally Identifiable Information (PII) while keeping other data available. This security propagates automatically, ensuring that whether users query via SQL or build Power BI reports, they only see what they’re authorized to access. OneLake security replaces the existing OneLake data access roles preview feature.
Users start by creating OneLake security roles that grant access to specific data in a lakehouse. In addition to selecting tables and folders, OneLake security also allows for row and column level security to be defined. Using T-SQL, table access can be restricted to only specific rows where the T-SQL statement is true. To secure entire columns, roles can contain column level security definitions that block access to the sensitive columns. Assign members to your role to grant them access to only the allowed items in that role.
With the role created, users can use any Fabric engine to query the data and see consistent results. Any queries through a Spark notebook are secured with OneLake security. The SQL Analytics Endpoint now uses the OneLake security definition to secure data when running in user’s identity mode. Semantic models can use Direct Lake mode to secure data using the security from OneLake. Even if users access the data in OneLake directly through API calls or OneLake file explorer, users are always restricted by the relevant OneLake security roles.
AI-generated content may be incorrect." width="511" height="611">
OneLake security will be preview in the coming months, sign up for early access.
Head over to OneLake security documentation for more information.
External data sharing enhancements
We have recently released several much-anticipated enhancements to the external data sharing feature. External data sharing allows in-place sharing of OneLake data across tenant boundaries. These updates include support for sharing multiple tables and folders, as well as entire Lakehouse schemas. Changes made to a shared lakehouse schema are automatically and immediately reflected in the consumer’s Lakehouse. Additionally, externally shared tables can now be consumed via the lakehouse’s SQL Analytics Endpoint and Semantic model, enabling seamless integration with Power BI reports.
We have also expanded the types of data that can be shared to include KQL and SQL databases and introduced service principal support in the external data sharing APIs for automated management. For more details, check out the full announcement of external data sharing
AI-generated content may be incorrect." width="377" height="615">
OneLake shared access signatures (SAS) (Generally Available)
OneLake shared access signatures (SAS) are now generally available (GA)! OneLake SAS tokens provide secure-short-term, delegated access to your resources in OneLake, helping you share or distribute data through scoped-down SAS tokens. SAS tokens and user delegation keys are always backed by a Microsoft Entra identity and always limited to a 1-hour lifetime, ensuring that all access to your data is through an approved identity and for a limited period.You can learn more about how to expand your data estate with OneLake SAS in the OneLake documentation: What is a OneLake shared access signature.
https://youtu.be/qilDzEjPig4?si=qmQxTeaeBhjoPJIL
Data Engineering
Write capabilities and PySpark support in Spark Connector for Fabric DW
We are pleased to announce the addition of writing capabilities with the Fabric Spark connector for Fabric Data Warehouse (DW) in the Fabric Spark runtime. This connector utilizes a two-phase write process to a Fabric DW table. Initially, it stages the Spark dataframe data into intermediate storage, followed by the COPY INTO command to ingest the data into the Fabric DW table. This approach ensures scalability with increasing data volumes and supports multiple modes for writing data to a DW table.Additionally, we are excited to announce PySpark support for this connector. This means you no longer need to use a workaround to utilize this connector in PySpark, as it is now available as a native capability in PySpark. The connector will be included as a default library within the Fabric Runtime, eliminating the need for separate installation.
To learn more about Spark Connector for Fabric Data Warehouse (DW), please refer to the documentation: Spark connector for Fabric Data Warehouse.
Esri’s ArcGIS GeoAnalytics integration with Microsoft Fabric Spark (Preview)
Esri is recognized as the global market leader in geographic information system (GIS) technology, location intelligence, and mapping, primarily through its flagship software, ArcGIS. Esri empowers businesses, governments, and communities to tackle the world's most pressing challenges through spatial analysis and location insight.We are pleased to share that Microsoft and Esri have partnered to bring spatial analytics into Microsoft Fabric and have launched public preview. Our collaboration with Esri introduces cutting-edge visual spatial analytics right within Microsoft Fabric Spark notebooks and Spark job definitions (across both Data Engineering and Data Science experiences).
With its integrated product experience, it empowers Spark developers or data scientists to natively use ArcGIS capabilities to run GeoAnalytics functions and tools within Fabric Spark for transformation, enrichment, and pattern / trend analysis of data across different use cases without any need for separate installation and configuration.
Example - How to transform the data with ArcGIS spatial function to uncover the pattern of interest, for instance summarizing the total number of policies of insured properties by hexagonal bins:
Description automatically generated">
Example - Understand the impact of natural hazards or current events on insured properties by bringing a dataset with probabilities of hurricane force winds and spatially joining it with insured properties. Spatial join links insured properties with wind speed probabilities, and with that for each property we would know the likelihood of hurricane force winds and can run predictive models to assess potential insurance claims.
Description automatically generated">
To learn more about this integration and capabilities, please refer to the documentation: ArcGIS GeoAnalytics for Microsoft Fabric (Preview).
Deployment pipeline inside Spark Job Definition (Preview)
The Spark Job Definition now supports the Deployment Pipeline. With this update, you can easily deploy your SJD (Spark Job Definition) item across different stages (Development, Testing, Production) and ensure that the proper state of the SJD item is synchronized across these stages. You can also customize the deployment with deployment rules to specify the default lakehouse and additional lakehouse of the SJD.Before triggering the deployment, you can verify the detail difference with the ‘Compare’ view.
AI-generated content may be incorrect.">
After the deployment is done, in the target stage/workspace, a new SJD item will be created based on the state from the source stage/workspace, and the association with other artifacts, such as Lakehouse and Environment, will also be set automatically.
AI-generated content may be incorrect.">
Deployment rule is supported to overwrite the default binding of default Lakehouse and Additional Lakehouse.
AI-generated content may be incorrect.">
By providing the Lakehouse ID, Lakehouse name, and the ID of the workspace where the Lakehouse is located, you can specify which Lakehouse should be set as the default in the target stage. You need to run the deployment after updating the deployment rule to make it effective.
To learn more about this, please refer to the documentation: Spark Job Definition deployment pipeline support.
Row-level and Column-level security in Spark
We are pleased to announce the introduction of row and column level security for Spark within Microsoft Fabric. This update significantly enhances data governance by incorporating fine-grained security controls within Spark.Access control policies are established in OneLake security by specifying limiting factors for rows and columns in conjunction with tables during role definition. Spark uses these roles associated with the user executing the code and applies row and column data filtering accordingly before presenting the data to the user's code.
These enhancements offer greater flexibility, stronger compliance, and simplified access management across Fabric’s unified data ecosystem.
Introducing Pylance language support for Fabric Notebook
Python developers using Fabric Notebook can now take advantage of Pylance, a powerful and feature-rich language server, to enhance their coding experience. With context-aware completions, better error detection, and improved code insights, Pylance makes PySpark and Python development smoother and more productive.Key Improvements with Pylance
Smarter Auto-Completion moves beyond basic keyword and variable suggestions to context-aware completions, helping users quickly find relevant variable names and functions.
Before Pylance
AI-generated content may be incorrect.">
With Pylance
AI-generated content may be incorrect.">
Enhanced Lambda Expression Support: More accurate completions within inline lambda functions, improving readability and efficiency for functional programming.
Parameter Completions: Intelligent suggestions based on type hints and type inference, streamlining function calls.
Improved Hover Information: More detailed insights when hovering over variables and code elements.
Better Docstring Rendering: Clearer formatting and presentation of documentation strings for better readability.
Error Markers & Semantic Highlighting: Improved error detection and code visualization, making debugging more intuitive.
With Pylance in Fabric Notebook, writing Python and PySpark code is faster, more accurate, and more efficient.
To learn more about Pylance in Notebook: Develop, execute, and manage Microsoft Fabric notebooks.
Environment sharing across workspaces (Preview)
You can now attach Environments from different workspaces in your Notebooks and Spark job definitions! This is made effortless with a brand-new explorer! Easily manage and utilize resources across multiple workspaces by exploring Environments from the workspaces you own, have access to, or that are shared with you by others.
AI-generated content may be incorrect.">
This feature provides flexibility in managing Environment permissions. Workspace viewers can use the Environment for running jobs without access to edit contents, while roles above workspace viewer can update the contents. To ensure only authorized users can access or update Environments, you can now manage Environments in one workspace, grant access to different users with different roles, or share the Environment with others with Read/Reshare/Edit permissions.
Note that using an Environment from a different workspace does not break the compute and security configurations set by the admins.
When you attach an environment from another workspace, both workspaces must have the same capacity and network security settings. Although you can select environments from workspaces with different capacities or network security settings, the session will fail to start. Furthermore, the compute configuration in that environment is ignored. Instead, the pool and compute configurations will default to the settings of your current workspace.
To learn more about across attaching Environments: Create, configure, and use an environment in Fabric.
Shortcuts now supported in Lakehouse Git metadata representation and in Fabric Deployment pipelines
Git and Deployment Pipelines support for Lakehouse objects is a top ask across all our customer base, and we are excited to announce that now Shortcuts definitions under the Tables and Files section of lakehouse are supported in the Fabric Git/ALM platform. This is an exciting milestone, allowing customers to version and orchestrate Shortcuts using Fabrics Application Lifecycle Management capabilities.Now, Shortcuts are automatically exported as JSON metadata to the git repository connected to the workspaces. Also, you can modify Shortcut properties directly in git using your favourite authoring tool and import changes directly to the workspace. The Fabric Deployment pipelines work as expected, Shortcuts are now deployed across the stages defined in the pipeline configuration.
This is the first step, on the upcoming releases, we will incrementally add support to additional object types under the Lakehouse, such as Folders, Tables, Views and more.
Find out more information about the feature in the Lakehouse deployment pipelines and git integration documentation.
Announcing Fabric User Data Functions (Preview)
Fabric User Data Functions is a serverless platform that gives app developers and data engineers the ability to easily write and run applications on Fabric. User Data Functions empowers you to implement custom logic wherever you need to in your Fabric ecosystem by leveraging native integrations with Fabric data sources, Fabric Notebooks and Data pipelines.You can use your functions to perform data engineering tasks such as data validation or data cleaning, create integrations with external systems, or create re-usable function libraries.
Learn more about this feature in the Fabric User Data Functions documentation.
Link to YouTube video
In this update, we added features that will help you make the best of your functions from the comfort of your browser.
Portal editor
AI-generated content may be incorrect.">
You can now create, modify, delete or edit your functions directly in your browser. This experience gives you powerful tools to add to your functions code with the convenience of using the Fabric website portal. The editor features Intellisense and Pylance functionality to help you write quality Python code, as well as common editing functionality such as edit history, find and replace, and more.
Insert code samples
One of the most convenient features in the portal editor is the Insert Samples function that allows you to input code to quickly get started developing common use case patterns such as reading and writing to a Fabric data source, performing data transformations, and more.
AI-generated content may be incorrect.">
Add your favorite PyPI libraries!
Another new feature is the Library management experience, which allows you to use the browser to add PyPI libraries into your project. Think of this as your requirements.txt file. You can select the library from a dropdown menu of names and choose a version that best suits your needs. The versions will be filtered to the ones compatible with the supported Python environment.
AI-generated content may be incorrect.">
New use cases and data sources!
User Data Functions are deeply integrated with the Fabric ecosystem. You can now invoke your functions from different kinds of Fabric items such as Fabric notebooks, Power BI reports and Data pipelines. In addition to this, you can connect to Fabric data sources such as warehouses, lakehouses, SQL Databases, and Mirrored Databases for all your data applications.
Learn more about this feature in the Fabric User Data Functions documentation.
OPTIMIZE FAST and FSCK commands in Fabric Runtime 1.3 for Apache Spark (Generally Available)
The Spark SQL FSCK REPAIR TABLE command and fast OPTIMIZE for V-Order are now available on Fabric Runtime 1.3 (Spark 3.5 / Delta 3.2).FSCK is designed to safely remove missing parquet files from the Delta transaction log, to restore table read consistency. This is not data recovery functionality, the missing parquet files and the data contained in it are lost. The command removes the references so the table can be back to a readable state. The command can also be run with a DRY RUN evaluation mode and will list all files that are missing in storage but still referenced by the Delta transaction log, to help you assess issues with the table before moving forward.
Delta Lake’s OPTIMIZE VORDER can now be run with idempotency, meaning that previously V-Ordered parquet files that are already within the target file size won’t be considered for bin compaction. This significantly improves the performance of the OPTIMIZE command. Enable it by setting parquet.vorder.fast.optimize.enabled to true in the Spark session configuration directly on Notebooks, Spark Jobs or using Environments.
Find more information in the Fabric Runtime 1.3 (GA) documentation.
Fabric Spark Monitoring APIs (Preview)
We’ve received valuable customer feedback emphasizing the need for API support to automate Spark job submission and monitoring. In response, we’re excited to introduce the preview of Fabric Spark Monitoring APIs—a robust set of tools designed to enhance observability and streamline the monitoring and management of Spark applications within Microsoft Fabric.To improve the developer experience, monitoring APIs for Fabric Spark applications are essential for optimizing performance, debugging issues, and ensuring efficient workload management. These APIs enable customers to automate Spark job management and monitor Spark jobs programmatically using APIs and SDKs.
Key Capabilities of Fabric Spark Monitoring APIs
With these APIs, users can:- List all Spark applications within a workspace.
- Retrieve Spark applications for specific items, including Notebooks, Spark Job Definitions, and Lakehouse.
- Access detailed Spark application metrics using Livy ID.
- Leverage Spark History Server APIs to obtain execution metrics and job event details for a single Spark application, including jobs, stages, tasks, executors, and event logs.
Stay tuned for further enhancements as we continue refining these APIs based on customer feedback!
Notebook Integration with User Data Functions (UDFs) (Preview)
Introducing the preview of Notebook integration with User Data Functions (UDFs)! This new capability allows you to define custom logic and calculations that can be reused across multiple Notebooks, helping streamline workflows and enhance code modularity.With NotebookUtils, you can now seamlessly access and invoke UDFs directly from your Notebook code, making it easier than ever to integrate reusable functions into your data processing and analysis.
Key Features & Scenarios
Here’s how you can take advantage of UDFs within your Notebooks:
AI-generated content may be incorrect.">
Invoking a Function
- IntelliSense and autocomplete support function names, improving usability.
- To help you better understand UDF functions, we’ve introduced a help method display(myFunction.functionDetails). This provides a clear view of function details, including parameters and return types, ensuring that you can invoke functions correctly.
This integration is available for Python, PySpark, Scala, and R, making it accessible across various data science and engineering workflows.
For more details, check out our documentation: NotebookUtils (former MSSparkUtils) for Fabric.
https://youtu.be/ktqKB4Bj1LQ?si=KPJ83e5D_jN8jbGF
Data Science
Copilot in Notebooks Agentic and UX enhancements
The Copilot in Notebooks Agentic enhancement has introduced several significant improvements aimed at enhancing the user experience and productivity. One of the key enhancements is the enhanced conversation history, which allows users to maintain context and continuity in their interactions. Additionally, there have been chat and natural language output enhancements through enhanced algorithms that ensure more accurate and relevant code generation. The improved code generation capabilities now offer advanced reasoning for complex problem-solving, making it easier for users to write, debug, and understand code within the notebook environment.We’ve also added a new interaction modal for Copilot, on-cell and quick actions. The Copilot in Notebooks on-cell and quick actions introduces powerful features designed to streamline and enhance the coding workflow. The On-Cell Copilot Button, conveniently positioned above each notebook cell, allows users to perform advanced data manipulation tasks such as pivoting tables, joining datasets, and aggregating data based on specific criteria.
AI-generated content may be incorrect.">
Additionally, the Quick Actions Button, located just below the cell, simplifies tedious tasks using AI, such as fixing code errors and adding code comments.
AI-generated content may be incorrect.">
These features not only improve the efficiency of coding tasks but also enhance the overall user experience by providing intuitive and accessible tools directly within the notebook environment. With these enhancements, users can achieve more accurate and efficient results, making their coding process smoother and more productive.
Fabric data agent
Since the launch of AI skill in August 2024, we’ve improved conversational abilities, support for multiple data sources and so much more. To better reflect these enhanced agentic capabilities, AI skill is now Fabric data agent!Copilot and AI capabilities now available across all paid SKUs
We’re thrilled to announce that we are removing the SKU requirement to access Copilot and AI capabilities across all paid SKUs. This means that customers on F2 and above will be able to use Copilot and AI features, such as Copilot in Power BI and Fabric data agent, to streamline workflows, generate insights, and drive impactful decisions.Fabric data agent integration with Azure AI Agent Service (Preview)
We are excited to launch the integration of data agents in Fabric with Azure AI Agent Service from Azure AI Foundry. A critical component of Azure AI Agent Service is the ability to securely ground AI agent outputs in enterprise knowledge, ensuring responses are accurate, relevant, and contextually aware. Data agents in Fabric can retrieve knowledge using several specialized query language tools that help AI to generate SQL, KQL and DAX.By combining Fabric’s sophisticated data analysis over enterprise data with Azure AI Foundry’s cutting-edge GenAI technology, businesses can create custom conversational AI agents leveraging domain expertise. This seamless integration enables organizations to develop agents that are not only based on unstructured data in Azure AI Search or SharePoint but also integrate with structured and semantic data in Microsoft OneLake, thereby enhancing data-driven decision-making.
Fabric data agent SDK (Preview)
We are pleased to announce the preview of the Fabric data agent Python SDK. The Fabric data agent Python SDK library is a powerful tool designed to streamline the development and prototyping of AI assistants on the Fabric platform. It is intended for users who are looking to create, manage, and utilize Fabric data agents programmatically. The library provides a set of simple APIs that facilitate various operations, such as managing Fabric data agents and integrating various data sources for enhanced analysis and insights. It also makes it easier for users to interact with the Fabric data agent using the OpenAI Assistants API. This enables users to quickly prototype and experiment with Fabric data agents to refine their solutions.
AI-generated content may be incorrect.">
With the Fabric data agent Python SDK, users can automate workflows and reduce manual effort. Users can seamlessly create, update, and delete Fabric data agent artifacts, optimize resource configurations, and gain valuable insights from their data. To get started, users can leverage the comprehensive documentation and sample code provided with the SDK. By automating experimentation and validation processes, the
Fabric data agent Python SDK ensures that developers can efficiently meet customer needs and deliver high-quality solutions, making it an invaluable tool for working with data agents.
INSTALL the Fabric data agent.
Data Warehouse
AI functions in Data Warehouse
AI functions are now available in private preview for Data Warehouse and Lakehouse SQL Endpoint, making it easier to bring AI-driven insights directly into your SQL workflows. With these built-in functions, you can summarize content, translate text, extract key data, analyze sentiment, and more - right from T-SQL. This eliminates the need for external processing, helping you streamline analysis and make faster, more informed decisions within your data warehouse.
AI-generated content may be incorrect.">
Figure 1: A few examples of AI functions usage.
Refer to the detailed blog Functions in Data Warehouse to learn more and sign up for preview.
Fabric User Data Functions in Data Warehouse
Scalar Fabric functions are now available in private preview for Data Warehouse and Lakehouse SQL Endpoint, giving you the flexibility to extend SQL capabilities beyond built-in functions. With this feature, you can write custom functions in Python (and soon other languages) and invoke them directly through T-SQL, just like regular scalar user-defined functions. This allows you to bring complex logic, custom transformations, and advanced computations closer to your data, reducing the need for external processing.
AI-generated content may be incorrect.">
Refer to the detailed blog Functions in Data Warehouse to learn more and sign up for preview.
Scalar SQL User-defined Functions
Scalar SQL User-Defined Functions (UDFs) are now available in private preview for Data Warehouse and SQL analytics endpoint.Scalar SQL User-Defined Functions (UDFs) are a cornerstone of T-SQL programming, widely recognized and utilized for their ability to encapsulate business rules and calculations into a reusable code. This feature offers an efficient solution for promoting code modularity across T-SQL queries while natively leveraging Fabric Warehouse distributed engine.
In an example below, by using four (4) different functions we can easily apply data masking logic on our customer table.
AI-generated content may be incorrect.">
Refer to the detailed blog Functions in Data Warehouse to learn more and sign up for preview.
Developer Experiences in Fabric Warehouse
IntelliSense for Collate ClauseThe COLLATE clause in Microsoft Fabric Warehouse and the SQL Analytics Endpoint of Lakehouse (LH) is essential for managing text-based data processing, ensuring accurate sorting, filtering, and comparisons. Given that Warehouse, SQL Analytics Endpoint of LH and other items support variations of case-insensitive and case-sensitive configurations, collation settings provide users with precise control over text handling in their workloads. By explicitly defining collation for VARCHAR and CHAR fields in table definitions, schema modifications, and queries, users can ensure consistency across transformations. The support for DATABASE_DEFAULT collation further simplifies schema management, allowing tables to inherit database-level settings for ease of administration and alignment with organizational standards.
The collation feature in Microsoft Fabric Warehouse and SQL Analytics Endpoint is enhanced with IntelliSense and syntax highlighting, providing a more intuitive and efficient development experience. IntelliSense offers real-time suggestions, validation for collation names, helping users avoid syntax errors and ensuring compatibility with supported collation settings. Syntax highlighting further improves readability by visually distinguishing collation clauses, making it easier to identify and manage collation settings in CREATE TABLE, ALTER TABLE, SELECT, and CTAS statements. These features streamline query development, reduce errors, and enhance productivity when working with case-sensitive and case-insensitive data configurations across Microsoft Fabric's SQL environments.
A few examples are:
CREATE TABLE [SampleData_CI_UTF8] (
[SampleID] INT NOT NULL, -- Unique identifier for each sample
[SampleValue] VARCHAR(50) COLLATE Latin1_General_100_CI_AS_KS_WS_SC_UTF8, -- Sample value with specified collation
[CreatedAt] DATETIME2(6) NOT NULL -- Timestamp for when the sample was created
);
CREATE TABLE [SampleData_DB_Default] (
[SampleID] INT NOT NULL, -- Unique identifier for each sample
[SampleValue] VARCHAR(50) COLLATE DATABASE_DEFAULT, -- Sample value with specified collation
[CreatedAt] DATETIME2(6) NOT NULL -- Timestamp for when the sample was created
);
INSERT INTO [SampleData_CI_UTF8] ([SampleID], [SampleValue], [CreatedAt])
VALUES (1, 'Sample1', GETDATE()), -- Inserting sample data
(2, 'Sample2', GETDATE());
INSERT INTO [SampleData_DB_Default] ([SampleID], [SampleValue], [CreatedAt])
VALUES (1, 'Sample1', GETDATE()), -- Inserting sample data
(2, 'Sample2', GETDATE());
-- Collate in Select
Select [SampleValue] COLLATE Latin1_General_100_CI_AS_KS_WS_SC_UTF8
from [SampleData_DB_Default]
Select [SampleValue] COLLATE DATABASE_DEFAULT
from [SampleData_CI_UTF8]
--Collate in CTAS
CREATE TABLE SampleDataCreate AS
Select [SampleValue] COLLATE Latin1_General_100_CI_AS_KS_WS_SC_UTF8 as SampleValue_CI_UTF8
from [SampleData_CI_UTF8]
CREATE TABLE SampleDataCreate AS
Select [SampleValue] COLLATE DATABASE_DEFAULT as SampleValue_CI_UTF8
from [SampleData_CI_UTF8]
-- Collate in ALTER Table Add New Column
ALTER TABLE SampleData_CI_UTF8
ADD Column4 VARCHAR(10) COLLATE Latin1_General_100_CI_AS_KS_WS_SC_UTF8 NULL;
ALTER TABLE SampleData_DB_Default
ADD Column4 VARCHAR(10) COLLATE DATABASE_DEFAULT NULL;
JSON data in OPENROWSET
These IntelliSense and grammar updates make working with OPENROWSET, JSON more seamless and efficient. With improved syntax highlighting, and query validation, SQL development is now faster and more error-free.Live templates
Writing T-SQL queries efficiently is crucial for database developers. Live Templates are predefined code snippets that can be inserted into your T-SQL editor with minimal effort. They help reduce repetitive coding, enforce best practices, and improve developer productivity.
Key benefits:
- Faster query development
- Standardized SQL formatting
- Reduced errors in repetitive tasks
Fabric_March_2025_Feature_Summary
AI-generated content may be incorrect.">
Expand and collapse objects properly in filter and search
We’re making search and filter experiences more intuitive! Now, objects that meet your search and filter criteria will automatically expand, giving you instant visibility into relevant data. Moving forward, we’ll expand only what’s required—keeping your object explorer clean and efficient unless no matches are found.
Artifact Status Bar
The Git item status bar component offers a comparable experience to the status bar in the workspace. When accessing the item page, you can view the details of the connection between the workspace and the Git repository, such as:
- The name of the branch to which the workspace is connected
- The time of the last sync event between the workspace and the repository
- A hyperlink to the most recent commit on the branch.
Fabric_March_2025_Feature_Summary
AI-generated content may be incorrect.">
Cancel query on closing editor
Handling long-running queries efficiently is crucial for a seamless warehouse experience. To improve user control, we’re introducing an enhanced query cancellation prompt that ensures users can make informed decisions when closing the editor while a query is still executing.
AI-generated content may be incorrect."> How it works
1. Prompting users when closing an active query
If a user attempts to close the editor while a query is running, they will see a confirmation message:
‘Do you want to cancel the query?’
- Yes: the query is canceled, and the editor closes immediately.
- No: the editor closes, but the query continues running in the Queries section, ensuring users don’t lose progress.
2. Customizing future prompts
When a user chooses Yes to cancel a query for the first time, they will see an additional prompt:
‘Do you want to see this message next time?’
- Yes: the prompt will continue appearing for future query cancellations.
- No: the editor will automatically cancel queries without showing the confirmation message moving forward.
Why this matters:
- Prevents accidental query cancellation – Ensures users don’t unintentionally stop important queries.
- Reduces interruptions – Users can choose whether they want to see the prompt in the future, keeping their workflow smooth.
- Personalized experience – Every user gets the flexibility to decide how they handle active queries when closing the editor.
Navigating your data warehouse just got faster! Our keyboard shortcuts UI enhance efficiency across key areas:
- Object explorer – Quickly browse and manage database objects.
- Ribbon – Access essential commands with a single keystroke.
- T-SQL editor – Speed up query writing and execution.
- Results grid – Seamlessly filter, copy, and analyze query results.
Fabric_March_2025_Feature_Summary
AI-generated content may be incorrect.">
Why use keyboard shortcuts?
- Faster navigation – Reduce mouse dependency and move through objects quickly.
- Increased productivity – Execute queries, format code, and manage results seamlessly.
- Streamlined workflow – Spend less time on repetitive actions and more on data insights.
SQL audit logs (Preview)
We are excited to announce that SQL audit logs are now in preview in Microsoft Fabric Data Warehouse! Audit Logs provide a detailed record of warehouse activity, capturing essential information such as when events occur, which triggered them, and the T-SQL statement behind the event. This feature is crucial for security and compliance, helping organizations monitor access patterns, detect anomalies, and meet regulatory requirements.
AI-generated content may be incorrect.">
Previously, tracking warehouse events manual effort making security audits and forensic investigations cumbersome. With native audit logging in Fabric Data Warehouse, organizations gain automated, tamper-resistant logging, simplifying security operations. Whether you need to investigate unauthorized access, analyze query execution trends, or ensure adherence to governance policies, SQL Audit Logs provide the transparency and control needed to safeguard your data.
Fabric Data Warehouse item permissions
We are thrilled to announce the introduction of enhanced sharing capabilities in Microsoft Fabric Data Warehouse! With these new updates, you can now grant additional permissions to monitor queries and audit activities, providing deeper visibility into warehouse operations. These enhancements allow organizations to delegate access more effectively, enabling security teams, auditors, and operations personnel to track query performance, analyze workloads, and audit activity with the appropriate level of control.By improving security, governance, and operational insights, these new capabilities help organizations maintain compliance while ensuring efficient data management.
What Permissions Can Be Assigned to Users?
When it comes to assigning permissions, it's important to understand the different types of permissions available and their implications. Here are some core and custom permissions that can be assigned to users:
- Read: Allows users to view the data.
- Write: Grants users the ability to modify the data.
- Reshare: Enables users to share data with others.
- Monitor: Provides users with the ability to monitor database activities and kill sessions.
- Audit: Allows users to configure and access audit logs.
- Restore: Permits users to perform in-place restores of data.
Assigning permissions can be done through user interfaces on the share dialog:
AI-generated content may be incorrect.">
After you click on the option, we will be able to see the options surfaced on the dialog menu:
AI-generated content may be incorrect.">
You can also validate the permissions on the Manage Permissions option on the share menu:
AI-generated content may be incorrect.">
Item permissions are a fundamental aspect of data management, providing the necessary controls to secure, comply, and collaborate effectively. By understanding and implementing permissions like Monitor, Reshare, and Audit, organizations can enhance their data security posture and foster a collaborative environment.
OneLake Security for Lakehouse Analytics SQL Endpoints Private Preview
As data governance becomes more central, we're thrilled to introduce OneLake Security for SQL Analytics Endpoints, now available in Private Preview! This powerful capability simplifies how security is enforced in Microsoft Fabric by letting you configure access once in OneLake, and have that security respected across your SQL workloads.With this release, organizations can now govern data at scale with a consistent, unified approach—whether you're implementing centralized security controls or need granular SQL-based permissions. OneLake Security empowers teams to secure, simplify, and scale access across your Lakehouse architecture.
Two Flexible Access Modes to Match Your Needs
OneLake Security introduces two distinct access modes for SQL Analytics Endpoints:
1. User Identity Mode
In this mode, the SQL Endpoint uses the signed-in user’s identity to access data in OneLake. It fully honors the RLS (Row Level Security), CLS (Column Level Security), and OLS (Object Level Security) rules defined in OneLake.
Great for: Organizations that want centralized control and alignment with data lake-level security.
AI-generated content may be incorrect.">
2. Delegated Identity Mode
Here, the SQL Endpoint uses the workspace or artifact owner's identity to connect to OneLake. This enables traditional SQL-based security management with full support for GRANT, custom roles, masking, and other advanced database security features.
Great for: SQL administrators and advanced use cases needing fine-grained SQL access control.
AI-generated content may be incorrect.">
User Identity x Delegated Mode
| Capability | User Identity Mode | Delegated Identity Mode |
| Access Context | Signed-in User | Datawarehouse Owner |
| OneLake RLS/CLS/OLS | Enforced | Not Enforced |
| SQL GRANT on Tables | Not Allowed | Allowed |
| SQL GRANT on Views/Procedures | Allowed | Allowed |
| Dynamic Data Masking | Not Supported | Supported |
| Custom SQL Roles | Not Supported | Supported |
Real-Time Intelligence
Eventstream sources: MQTT, Solace, ADX, weather & Azure Event Grid
Eventstream is a powerful feature in Fabric Real-time Intelligence that allows users to ingest, transform, and route real-time data streams to various destinations within Fabric. We are excited to announce the addition of five new sources and additional sample data streams. These new sources enhance Eventstream's streaming capabilities, enabling seamless data ingestion and real-time transformation across various data streams.
AI-generated content may be incorrect.">
Let’s dive into the functionalities of each connector and explore how they can benefit your data processing needs.
- MQTT connector: Connect to an MQTT broker, subscribe to specific topics, and stream data from those topics into Eventstream.
- Solace PubSub+: Read messages from a Solace PubSub+ Event Broker cluster and stream them into Eventstream for real-time data processing.
- Azure Data Explorer: Streams data from an Azure Data Explorer database in real-time into Fabric.
- Real-time Weather: Ingest live weather data for a selected city into Eventstream, including temperature, humidity, and wind speed.
- Azure Event Grid namespace: Stream MQTT, IoT, or any messages from Azure Event Grid namespace to Eventstream.
- Sample data streams: Kickstart your streaming projects with additional pre-built sample streams, including real-time bus tracking data and S&P 500 stock market data.
Can’t find your data sources?
Let us know! Send us an email at askeventstreams@microsoft.com or fill out our survey.
Eventstream CI/CD & REST APIs (Generally Available)
Collaborating on data streaming solutions can be challenging, especially when multiple developers work on the same Eventstream item. Version control challenges, deployment inefficiencies, and conflicts often slow down development. Since introducing Fabric CI/CD tools for Eventstream last year, many customers have streamlined their workflows, ensuring better source control and seamless versioning. Now, we’re excited to announce the general availability (GA) of Eventstream CI/CD and REST APIs—making these capabilities even more accessible and powerful for all users.Key benefits of leveraging CI/CD tools in Eventstream:
- Enhanced Collaboration: With Git integration, developers can use GitHub or Azure DevOps to sync with the Fabric workspace and work in parallel on the same Eventstream item without conflicts.
- Streamlined Deployments: The Deployment pipeline feature accelerates and standardizes Eventstream deployments to various stages, such as testing and production workspace, with minimal manual effort in the Fabric UI. This ensures a more efficient and reliable deployment process.
- Automated Workflows: The availability of Eventstream REST APIs allows developers to build fully automated CI/CD pipelines and integrate external applications. This capability ensures quality, reliability, and productivity for data streaming projects, reducing manual intervention and potential errors.
- Increased Productivity: By leveraging these powerful CI/CD tools, teams can focus more on transformation within Eventstream and less on managing conflicts and deployment issues. This ultimately boosts overall productivity and project success.
Expanded regional availability for Eventstream's managed private endpoints (Secure Outbound)
Managed Private Endpoint (MPE) is a Fabric platform security feature that allows Fabric items, such as Eventstream, to securely connect to data sources behind firewalls or protected networks. Since we introduced this integration last year, many customers have relied on it to establish secure outbound connections between Eventstream and their data sources.This feature ensures that your data is transmitted securely over a private network, allowing you to fully harness the power of real-time streaming and high-performance data processing in Eventstream. The diagram below shows a typical setup using MPE in Eventstream.
AI-generated content may be incorrect.">
Managed Private Endpoints are now available in even more regions, making it easier for organizations worldwide to build secure, scalable streaming solutions. The table below lists supported regions for Eventstream’s MPE:
AI-generated content may be incorrect.">
To learn more about Managed Private Endpoints, check out the Connect to Azure resources securely using MPE in Eventstream.
Connect to Eventstream using Microsoft Entra ID authentication
We’re excited to introduce Microsoft Entra ID authentication for Eventstream’s Custom Endpoint! This feature enhances security by eliminating the need for SAS keys or connection strings, reducing the risk of unauthorized access. Instead, Entra ID authentication ensures that user permissions are directly tied to Fabric workspace access, allowing only authorized users to send and fetch data from Eventstream.The screenshot demonstrates how this feature works in Eventstream’s Custom Endpoint!
AI-generated content may be incorrect.">
Additionally, if you’re using an Azure resource like Azure Logic Apps with a system-assigned or user-managed identity, you can now assign Fabric workspace permissions to that identity. This enables Azure Logic Apps to seamlessly connect to Eventstream using Managed Identity authentication.
The screenshot demonstrates how to enable identity in the Azure Logic Apps and assigning permission in the Fabric workspace.
AI-generated content may be incorrect.">
To learn more about Entra ID authentication in Eventstream’s Custom Endpoint, refer to our documentation Connect to Eventstream using Microsoft Entra ID authentication.
Preview Real-Time Data Streams for Apache Kafka, Confluent Cloud, Amazon MSK & Amazon Kinesis
Transforming data in Eventstream requires an actual schema derived from incoming data, which can slow down development and troubleshooting. To simplify this process, we are excited to introduce Data preview, a major usability enhancement for third-party connectors in Fabric Eventstream, including Apache Kafka, Confluent Cloud, Amazon Managed Streaming for Apache Kafka (MSK) and Amazon Kinesis Data Streams. With this new capability, users can preview a snapshot of their source data directly within Eventstream Edit mode and process data with inferred schemas.Why Data preview matters
The Data preview feature allows users to:
- Enable Eventstream to infer the schema from incoming data, making it easier to configure operators such as filtering and aggregation.
- Verify if an Eventstream source is properly configured.
- Preview of real-time data snapshots to confirm data is ingesting as expected.
Using Data preview in Eventstream is simple:
- Select a source connector in Eventstream (e.g., Confluent Cloud).
- Click on ‘Data Preview’ tab to view a snapshot of the source data.
- Change and match the source data format for preview.
Edit mode:
AI-generated content may be incorrect.">
With Data preview, teams can build, test, and deploy Eventstream items faster and with greater confidence. Get started today and experience the power of real-time processing for your third-party connectors in Eventstream!
Continuous improvements to Eventhouse Get Data Experience
There are a few different methods to get data into the Real-Time Intelligence workload, depending on your organization’s needs. Data can be pushed or pulled into an Eventstream from one of the many connectors and then landed in an eventhouse. Alternatively, there are several ways to directly ingest data into Eventhouse.Data can be directly from:
- Local files
- Azure storage
- Amazon S3
- Event hub
- Eventstreams
- OneLake
Over the past few months, our team has been working tirelessly to bring new features to the Get Data wizard, creating a simpler interface, quicker navigation, and added automation, all aimed at delivering a better user experience and improved performance.
The main changes introduced:
- Automated schema optimization: Since the Eventhouse engine is highly optimized for datetime and string operations, in certain cases mapping imported data to these data types can offer a significant boost in query performance. By introducing usage of the inferred schema plugin, in most applicable cases an optimal data mapping will be inferred and automatically applied to your data, enhancing Eventhouse’s query and indexing performance. For instance, it can detect columns storing Unix date-time values as long and convert them to datetime. Similarly, it might recognize that a column named ‘id’ using a long type should be converted to string. Any such automatic mapping changes are clearly reflected in the schema editor, with a lightbulb icon and a short explanation of the mapping applied. Changes can easily be manually reverted using the ‘Type’ dropdown menu, although this is usually not recommended.
Fabric_March_2025_Feature_Summary
AI-generated content may be incorrect.">
- Simplified schema inspection: The Schema Inspection step allows users to preview the destination table schemas, modify the schema if any change is needed or extract the KQL commands for table and schema creation. Based on the feedback received from our customers, we have improved the schema Inspection experience to make schema preview and edit a seamless experience. Users can now easily switch between the schema preview; command viewer and schema edit modes with an intuitive switcher experience. We have also simplified the inference file, formats, mapping and nested JSON options to make them more accessible.
We have graduated Sample data option in the inspect step to real-time data sampling. This allows users to preview how the data would look like when ingested, even before finalizing the data schema.
AI-generated content may be incorrect.">
3. Automatic detection of header row for CSV files:
The Get Data wizard now seamlessly detects if a CSV has a header row and uses it for column names. The column data type is inferred based on the data in the remaining rows. This makes the process of schema definition, when your file has headers a painless process.
AI-generated content may be incorrect.">
Build event-driven workflows with Azure and Fabric Events (Generally Available)
Azure and Fabric Events, a powerful capability that allows organizations to capture, process, and respond to events across Microsoft Fabric is now generally available. With these events, businesses can integrate event-driven solutions into their workflows, enabling seamless automation, enhanced observability, and faster decision-making.
AI-generated content may be incorrect.">
What are Azure and Fabric Events?
Azure and Fabric Events offer a capability within Real-Time Intelligence that enables you to:
- Ingest events that are available in Microsoft Fabric like Onelake events, Azure blob storage events, Job events,Workspace item events
- Filter those events using rich filtering capabilities on event schema properties.
- Integrate those events to consumers in Microsoft Fabric like Activator for setting event-based alerts or Eventstream to stream events to other destinations.
To learn more, please go to Azure and Fabric Events documentation and for the full announcement, refer to the announcement blog.
Eventhouse OneLake availability now supports backfill
Eventhouse OneLake availability allows creating a delta parquet representation of data in Eventhouse. Previously, when you turned availability ON, only new data was made available in OneLake, with no backfill of existing data. This could cause inconsistencies between the data in Eventhouse and OneLake.Now, OneLake Availability supports backfill, making all existing and new data in Eventhouse available, regardless of when you turn it ON. This is the default behavior when you enable availability via the UI.
Learn more about Eventhouse OneLake availability.
AI-generated content may be incorrect.">
Improved Activator alerts from Power BI
We’ve made it easier than ever to create and manage Activator alerts on your Power BI reports. We’ve redesigned the Power BI ‘Set Alert’ experience so that you can conveniently manage your alerts entirely within your reports, without having to open Activator. We’ve also streamlined the experience so that you can set up an alert with fewer steps.To check out the new experience, open a Power BI report and select ‘Add Alert’ on a visual, or choose ‘Set Alert’ from the ribbon.
AI-generated content may be incorrect.">
Figure 2: The improved ‘Set Alert’ experience in Power BI makes it easier than ever to create and manage Activator alerts on your reports.
End-to-end Real-Time Intelligence samples
We are excited to announce a brand new RTI sample experience which allows you to create a fully working end-to-end RTI flow within seconds. The sample flow allows you to explore the main features of Real-Time Intelligence with sample data. It provides a comprehensive end-to-end solution, demonstrating how Real-Time Intelligence components work together to stream, analyse, and visualize real-time data in a real-world context.- You can access the samples from the RTI workload home page.
- Select the sample scenario of your choice. Choice of Bike rentals or S&P 500 Stocks data.
- Create a sample solution with all RTI items within seconds.
Fabric_March_2025_Feature_Summary
AI-generated content may be incorrect.">
Learn more about End-to-end sample.
Synapse Data Explorer to Eventhouse migration tooling (Preview)
The next generation of Azure Synapse Data Explorer offering is evolving to become Eventhouse, part of Real-Time Intelligence in Microsoft Fabric. For customers looking to migrate to Eventhouse, we are providing a migration tooling that allows you to seamlessly migrate Synapse Data Explorer cluster to an Eventhouse in Fabric.The migration process is performed using Fabric REST API endpoints. The recommended steps for performing the migration are as follows:
- Validate: Use the Validate migration to Eventhouse endpoint to check whether the Azure Synapse Analytics Data Explorer cluster can be migrated to an eventhouse.
- Migrate: Use the Migrate to Eventhouse with the migrationSourceClusterUrl payload to create an eventhouse with the migration source cluster URL. The process runs asynchronously to create a new eventhouse and migrate all databases from the source cluster to the eventhouse.
- Monitor: Use the Monitor migration progress to track the progress of your migration.
- Verify: Verify the migration by checking the eventhouse state is Running, and that the migrated databases appear in the KQL database list.
https://youtu.be/pluk-b8XVj4?si=WYcsvCJG4SIhzaHf
Data Factory
Enterprise readiness
VNET Gateway support for Data pipelines
Support for data pipeline functionality on the VNet data gateway is now available in preview. The VNet data gateway facilitates connections to data sources that are either behind firewalls or accessible within your virtual network. This feature enables the execution of data pipeline activities on the VNet data gateway, ensuring secure connections to data sources within the VNet. Unlike on-premises data gateways, VNet data gateways are managed by Microsoft, consistently updated, support auto-scaling, and deactivate when not in use, making it cost-effective.
AI-generated content may be incorrect.">
To learn more, refer to the documentation: What is a virtual network (VNet) data gateway?
Best-in-class connectivity and enterprise data movement
In the fast-evolving data integration landscape, Data Factory continues to enhance the existing connectors to provide a seamless, high-performance experience. With a focus on improving connector efficiency and expanding capabilities, recent updates have made significant advancements to Salesforce and Lakehouse connectors. These improvements not only boost performance but also enable more sophisticated data handling, ensuring that enterprises can extract, transform, and load data with greater accuracy and efficiency.
Performance improvement in Salesforce connector in data pipelines
Salesforce is a critical data source for many organizations, housing valuable customers and business data. To enhance data movement efficiency, Data Factory has introduced performance optimization in the Salesforce connector for pipelines. Optimization allows you to fetch the data concurrently from Salesforce by leveraging the parallelism capability, thus significantly reducing extraction times for large datasets.
Lakehouse connector now supports deletion vector and column mapping for delta tables in data pipelines
The Lakehouse connector in Data Factory has been upgraded to provide deeper integration with delta table. Two major new capabilities enhance data processing workflows:
1. Support for deletion of vector
Delta table uses deletion vectors to track deleted records efficiently without physically removing them from storage. With this new feature in the Lakehouse connector, users can:
- Read Delta tables while respecting deletion vector, ensuring that deleted records are automatically excluded from queries.
- Improve performance by leveraging soft deletions instead of physical file modifications, making data updates and maintenance more efficient.
- Enable compliance with data retention policies by retaining historical data for auditability while ensuring deleted records are filtered out from active queries.
Delta table's column mapping capability allows for more flexible schema evolution, ensuring that changes in table structure do not disrupt data workflows. With column mapping support in the Lakehouse connector, users can:
- Read from an existing delta Lake table with column mapping name/id mode enabled
- Write to existing delta lake table with column mapping name/id mode enabled
- Auto-create table with column mapping name mode enabled when sink table does not exist and source dataset columns contain special chars & whitespaces.
- Auto-create table with column mapping name mode enabled when table action is overwriting schema and source dataset columns contain special chars & whitespaces.
To learn more about how to Configure Lakehouse in a copy activity refer to our documentation.
New and updated Certified Connectors for Power BI and Dataflows
As a developer and data source owner, you can create connectors using the Power Query SDK and have them certify through the Data Factory Connector Certification Program. Certifying a Data Factory connector makes the connector available publicly, out-of-box, Microsoft Fabric Data Factory and Microsoft Power BI in the following experiencesThis month we are happy to list the newly updated certified connectors that are part of the Microsoft Data Factory Connector Certification Program. Be sure to check the documentation for each of these connectors so you can see what’s new with each of them.
New connectors
Updated connectors- Anaplan
- Asana
- BQE Core
- BuildingConnected
- Delta Sharing
- SolarWinds Service Desk
- Supermetrics
- Windsor
- Worksplace Analytics
- Zendesk Data
Simplifying Data Ingestion with Copy Job
Copy Job is making data ingestion simpler, faster, and more intuitive than ever and is now generally available. Whether you need batch or incremental data movement, Copy Job provides the flexibility to meet your needs while ensuring a seamless experience.Since its preview last September, Copy Job has rapidly evolved with several powerful enhancements. Let’s dive into what’s new!
Public API & CICD support
Fabric Data Factory now offers a robust Public API to automate and manage Copy Job efficiently. Plus, with Git Integration and Deployment pipelines, you can leverage your own Git repositories in Azure DevOps or GitHub and seamlessly deploy Copy Job with Fabric’s built-in CI/CD workflows.
AI-generated content may be incorrect.">
VNET gateway support
Copy Job now supports the VNet data gateway in Preview! The VNet data gateway enables secure connections to data sources within your virtual network or behind firewalls. With this new capability, you can now execute Copy Job directly on the VNet data gateway, ensuring seamless and secure data movement.
Upsert to Azure SQL Database & overwrite to Fabric Lakehouse
By default, Copy Job appends data to ensure no changed data is lost. But now, you can also choose to upsert data directly into Azure SQL DB or SQL Server and overwrite data in Fabric Lakehouse tables. These options give you greater flexibility to tailor data ingestion to your specific needs.
AI-generated content may be incorrect.">
Enhanced usability & monitoring
We’ve made Copy Job even more intuitive based on your feedback, with the following enhancements:
- Column mapping for simple data modification to storage as destination store.
- Data preview to help select the right incremental column.
- Search functionality to quickly find tables or columns.
- Real-time monitoring with an in-progress view of running Copy Jobs.
- Customizable update methods & schedules before job creation.
Fabric_March_2025_Feature_Summary
AI-generated content may be incorrect.">
More connectors, more possibilities!
More source connections are now available, giving you greater flexibility for data ingestion with Copy Job. And we’re not stopping here—even more connectors are coming soon!
AI-generated content may be incorrect.">
What’s next?
We’re committed to continuously improving Copy Job to make data ingestion simpler, smarter, and faster. Stay tuned for even more enhancements!
Learn more about Copy Job in: What is Copy job in Data Factory
Mirroring
Mirroring for Azure SQL Database protected by a firewall (Preview)
You now can mirror Azure SQL Databases protected by a firewall. Using either the VNet data gateway or the on-premises data gateway for mirroring is available. The data gateway facilitates secure connections to your source databases through a private endpoint or from a specific private network.Learn more about Mirroring for Azure SQL Database from Microsoft Fabric Mirrored Databases from Azure SQL Database.
AI-generated content may be incorrect.">
Mirroring for Azure Database for PostgreSQL Flexible Server (Preview)
Database Mirroring now supports replication of your Azure Database for PostgreSQL Flexible Server into Fabric! Now you can continuously replicate data in near real-time from your Flexible Server instance to Fabric OneLake. This enables seamless data integration, allowing you to leverage Fabric’s analytics capabilities while ensuring your PostgreSQL data remains up to date. By mirroring your PostgreSQL data into Fabric, you can enhance reporting, analytics, and machine learning workflows without disrupting your operational database.To learn more, please reference the PostgreSQL mirroring preview blog.
Open Mirroring UX improvements
We’ve made improvements to our end-to-end in-product experience for Open Mirroring. With these changes, you can now create a Mirror DB and start uploading or dragging and dropping parquet and CSV files. It’s now easier than ever to get started with building your own Open Mirror source and allow you to test our replication technology before productionizing with APIs. Once your files are uploaded, you can also upload changes and updates to the data with the __rowMarker__ field specified to our change data capabilities.
AI-generated content may be incorrect.">
Transformations
Save a new Dataflow Gen2 with CI/CD support from a Dataflow Gen1, Gen2, or Gen2 (CI/CD)
Customers often would like to recreate an existing dataflow as a new dataflow Gen2 (CI/CD), getting all the benefits of the new GIT and CI/CD integration capabilities. Today, to accomplish this, they need to create the new Dataflow Gen2 (CI/CD) item from scratch and copy-paste their existing queries or leverage the Export/Import Power Query template capabilities. This, however, is not only inconvenient due to unnecessary steps, but it also does not carry over additional dataflow settings.Dataflows in Microsoft Fabric now includes a ‘Save as’ feature in preview, that in a single click lets you save an existing dataflow Gen1, Gen2 or Gen2 (CI/CD) as a new Dataflow Gen2 (CI/CD) item.
AI-generated content may be incorrect.">
Incremental Refresh for Dataflow Gen2 and new support to Lakehouse as destination with incremental refresh (Generally available)
Incremental Refresh for Dataflow Gen2 is now generally available!Incremental Refresh for Dataflow Gen2 allows you to refresh only the buckets of data that have changed, rather than reloading the entire dataset on every dataflow refresh. This not only saves time but also reduces resource consumption, making your data operations more efficient and cost-effective.
These new capabilities are designed to help you to be successful with your data integration needs and be as efficient as possible. Try it out today in your fabric workspace!
Learn more about Incremental Refresh in Dataflow Gen2: Incremental refresh in Dataflow Gen2.
Check ongoing validation status of a Dataflow Gen2 with CI/CD support
When you click Save & run in Dataflow Gen2 with CI/CD support, the process that gets triggered is two-fold:- Validation: it’s a background process where your Dataflow gets validated against a set of rules. If it passes all validations and no errors are returned, then it’ll be successfully saved.
- Run: Using the latest published version of the Dataflow, a refresh job gets triggered to run the Dataflow.
What if you want to check the status of the validation? You now have a new entry point in the home tab of the ribbon called Check validation which you can click at any time to give you information of the ongoing validation or the result of a previous validation run.
Description automatically generated">
Be sure to give this a try whenever you want to check the results of a save validation.
Orchestration
Apache Airflow Job (Generally Available)
The Apache Airflow job in Microsoft Fabric is now generally available, providing a fully integrated Apache Airflow runtime for developing, scheduling, and monitoring Python-based data workflows using Directed Acyclic Graphs (DAGs).What's New:
- Introducing Fabric runtime versioning for Apache Airflow job – This includes Fabric runtime version 1.0, which comes with Apache Airflow 2.10.4 and Python 3 as the default runtime.
- Public API – APIs are now available to interact with Apache Airflow jobs for seamless management.
- Git Integration & Deployment pipeline support – Users can utilize their Git repositories (Azure DevOps/GitHub) and deploy with Fabric’s built-in CI/CD workflows.
- Diagnostic logs – Users can access Apache Airflow generated logs through the Apache Airflow job UI for enhanced observability.
Fabric_March_2025_Feature_Summary
AI-generated content may be incorrect.">
Learn more about Apache Airflow job in Microsoft Fabric in What is Apache Airflow job?
OneLake file triggers for pipelines
The Fabric Data Factory team is thrilled to announce that the pipeline trigger experience is now generally available (GA) and now includes access to files in OneLake!
Description automatically generated">
This exciting new improvement to pipeline triggers in Fabric Data Factory means that you can now automatically invoke your pipeline when files or folders have files that arrive, delete, or rename!
Description automatically generated">
We’ve previously supported Azure blob file events in Fabric Data Factory like ADF & Synapse but now that Fabric users are leveraging OneLake as the primary data hub, we’re excited to see the pipeline patterns that you’ll build using OneLake file triggers!
Variable libraries for pipelines (Preview)
One of the most requested features in Fabric Data Factory has been support for modifying values when deploying workspace changes between environments using Fabric CICD. To accommodate this, ask, we have integrated pipelines into the new Fabric platform feature called Variable Libraries.
AI-generated content may be incorrect.">
With Variable Libraries, you can assign variables to unique values based on different environments, i.e. dev, test, prod. Then when you promote your factory to high environments, you can use different values from the library providing the ability to change values when pipelines are promoted to new environments.
This new preview feature will be super useful not just for CICD but also generically allows you to replace hardcoded values with variables anywhere in your pipelines to achieve the same functionality as global parameters in Azure Data Factory as well.
Spark Job Definition pipeline activity parameter support
The Spark Job Definition (SJD) activity in Data Factory allows you to create connections to your Spark Job Definitions and run them from your data pipeline.And we are excited to announce that parameterization is now supported in this activity!
You will find this update in the Advanced settings where you can configure your SJD parameters and run your Spark Job Definitions with the parameter values that you set, allowing you to override your SJD artifact configurations.
AI-generated content may be incorrect.">
Azure Databricks jobs activity now supports parameters
Parameterizing data pipelines to support generic reusable pipeline models is extremely common in the big data analytics world. Fabric Data Factory provides end-to-end support for these patterns and is now extending this capability to the Azure Databricks pipeline orchestration activity. Now when you select ‘Jobs’ as the source of your ADB action, you can send parameters to your ADF job allowing maximum flexibility and power of your orchestration jobs.
AI-generated content may be incorrect.">
User data functions in Data pipelines (Preview)
User Data Functions are now available in preview within Data pipeline's Functions activity. This new feature is designed to enhance your data processing capabilities by allowing you to create and manage custom functions tailored to your specific needs.
Description automatically generated">
Key Highlights
- Custom functionality: User Data Functions enable you to define custom logic and calculations that can be reused across multiple Data pipelines. This allows for more flexible and efficient data processing.
- Integration in data pipelines: You can add User Data Functions as activities within your Data pipelines. This is done by selecting the Functions activity in the pipeline editor, choosing your User Data Functions as the type, and providing any necessary input parameters.
Data Factory pipelines now support up to 120 activities
We’ve increased the default activity limit from 80 activities to 120 activities!You can now utilize an additional 40 activities to build more complex pipelines for better error handling, branching, and other control flow capabilities.
Description automatically generated">
https://youtu.be/spe7ZMImHH0?si=uf7NmQ6CNp2iDJLO
Dataflow Gen2
Dataflow Gen2 with CI/CD capabilities
You can now add Dataflow refresh activities to your pipelines in Fabric Data Factory that include the new version of Dataflows:
AI-generated content may be incorrect.">
Check out our documentation on Dataflow Gen2 with CI/CD and Git integration to learn more.
Data pipelines
Data pipelines have supported CI/CD capabilities and REST APIs support is now generally available. The team just added Service Principal Name (SPN), and Variable libraries support for Data pipelines.Check out our documentation on CI/CD for Data pipelines and REST API capabilities for Data pipelines to learn more.
Mounting ADF (Preview)
CI/CD and REST APIs support is now available for the Azure Data Factory item (Mounting ADF).Mirrored database (Generally Available)
The mirrored database’s CI/CD support is now Generally Available.Learn more from CI/CD for mirrored databases.
The REST APIs support has been Generally Available including the SPN support. Check out our documentation on Mirroring Public REST APIs.
Copy Job (Generally Available)
The Copy job item’s CI/CD and APIs support is now Generally Availability. This includes SPN support for Copy job.Check out our documentation on CI/CD for Copy job to learn more.
Parameterization
Parameterized connections in Data pipelines
Enhancing your Data Integration experienceWhat are Parameterized Connections?
Parameterization of data connections in Data pipelines allows you to specify values for connection placeholders dynamically. This means you can pre-create data connections for various sources, such as Azure Blob Storage, SQL Server or any other data source supported by data pipelines, and reference them through data pipeline’s dynamic expressions at runtime. This feature empowers you to create more flexible and adaptable data pipelines, capable of connecting to different instances of data connections of the same type, such as SQL Server, without altering the pipeline definition.
Key benefits:
- Flexibility: Use the same data pipeline definition to dynamically connect to various instances of data connections.
- Efficiency: Minimize the need for multiple pipeline definitions, reducing complexity and maintenance effort.
- Scalability: Easily manage and scale your data integration processes by leveraging dynamic expressions to handle connection values.
AI-generated content may be incorrect.">
During the pipeline run, dynamic expressions within the data pipelines specify values for the connection placeholders, enabling seamless integration with pre-created data connections. This innovation ensures that your data pipelines are not only more efficient but also highly customizable to meet your specific requirements.
We believe this new feature will significantly enhance your data processing capabilities and streamline your workflows. We can't wait for you to experience the benefits of parameterized connections in your data integration projects.
Table Name parameter support for data destinations
In Dataflow Gen2, you can create parameters, they serve as a way to easily store and manage a value that can be reused throughout your Dataflow.Major feedback that we’ve heard from our users is the lack of this capability in the Data destination experience for Dataflow Gen2. Thanks to the feedback, we’re now introducing the first support for parameters in the data destination experience where you can set a parameter to be used for the Table name of your destination.
Description automatically generated">
This is available to all destinations that support this field and we’re working on extending this support to other areas of the data destination experience. Try out this new capability and let us know what you think.
AI-powered experiences
Efficiently build and maintain your Data pipelines with enhanced capabilities for Copilot in Data Factory
In November 2024, we announced the preview of 3 innovative capabilities in Copilot for Data Factory (Data pipeline). Today, we are excited to make these features generally available, with enhancements to make your data integration even more efficient and effortless.Check out the blog post on Efficiently build and maintain your Data pipelines with Copilot for Data Factory: new capabilities a... to learn more.
Effortlessly generate your data pipelines: Understand your business intent and effortlessly translate it into data pipeline activities to build your data integration solutions. In the enhanced capability of Copilot, we can easily build more complex Data pipeline activities e.g. switch activity, metadata driven pipeline, etc. You can also update your pipeline settings and configurations in batches with multiple activities!
AI-generated content may be incorrect.">
Efficiently troubleshoot error messages in your data pipeline Copilot. Diagnose and resolve pipeline errors more intuitively by providing clear and actionable summary.
AI-generated content may be incorrect.">
Easily understand your complex data pipelines: Understand your complex pipeline configurations effortlessly by getting a clear and intuitive summary provided by Copilot.
AI-generated content may be incorrect.">
Data integration shared experiences
Partner workloads
We are thrilled to share the significant updates and newly released workloads for this month, showcasing the incredible efforts and innovative work by our amazing partners.Workloads (Generally Available)
Osmos AI Data WranglerAutomate Data Ingestion with Osmos AI Data Wrangler for Microsoft Fabric
As enterprises scale AI adoption, they must ensure all their data is AI-ready. However, enterprise data is often messy - semi-structured or unstructured, arriving in inconsistent formats from customers, partners, suppliers, and internal systems. Traditional ETL pipelines require constant engineering effort to adapt to schema drift, missing fields, and poor data quality, slowing down AI and analytics initiatives.
Osmos AI Data Wrangler enables autonomous data transformation as a Workload on Microsoft Fabric. Osmos’ agentic AI automates data ingestion by intelligently cleaning, transforming, and validating your data, seamlessly normalizing messy bronze data into silver tables in your Lakehouse.
Get it now: Osmos AI Data Wrangler for Microsoft Fabric
AI-generated content may be incorrect.">
Osmos AI Data Wrangler is now generally available!
Osmos’ AI Data Wrangler is now generally available (GA), featuring self-configuration capabilities, featuring self-configuring Wrangler Context. This feature allows Wranglers to understand your business rules and apply it to data transformations. With the new Wrangler Context feature, businesses can auto-configure their Wranglers using existing documentation and code snippets, giving you easy-to-configure high performance data wranglers quickly.
AI-generated content may be incorrect.">
AI-generated content may be incorrect.">
Why enterprises choose Osmos AI for data ingestion & transformation
Osmos helps businesses across retail, manufacturing, finance, and audit unlock millions in savings while accelerating insights and new market opportunities.
✔ Unify clean data into Lakehouse - Normalize disparate data coming from multiple internal and external sources into clean, actionable SQL-ready data.
✔ Improve data quality - AI-powered validation ensures structured, clean, and accurate data for analytics and decision-making.
✔ Streamline workflows - Eliminate manual data cleanup by enabling Wranglers to autonomously learn from documentation and business rules.
✔ Enhance AI & analytics readiness - Deliver trusted, structured data ready for AI-driven insights and enterprise analytics.
Check out the Osmos AI Data Wrangler with Wrangler Context video.
Power BI Designer (Generally Available)
Microsoft Fabric in collaboration with PowerBI.tips, are excited to share that Power BI Designer is now generally available! You won’t want to miss out on this time saving application. Say goodbye to bland, cookie-cutter reports and hello to dazzling, highly stylized masterpieces.What’s Power Designer All About?
Power Designer is sleek, intuitive, and fun, making designing reports feel less like work and more like unleashing your inner artist. It’s packed with features that’ll have you saying, ‘Why didn’t I have this sooner?’
Get it now: Power Designer Workload.
Let’s dive into the magic:
- Craft themes like a pro: create detailed theme files for your Power BI reports with ease. Customize colors, fonts, and styles to match your brand.
Fabric_March_2025_Feature_Summary
Description automatically generated">
- Real-Time visual vibes: watch your Power BI visuals update live as you build your style.
Fabric_March_2025_Feature_Summary
Description automatically generated">
- Multi-page: Add background images to each page with a snap, transforming your reports into polished, magazine-worthy layouts.
- AI-Powered: Let AI take the wheel with auto-placement of visuals in your multipage templates.
- Preview: Test your shiny new theme on reports already published in your workspaces with the preview feature.
Ready, set, design! Let’s make some report magic happen!
Learn more at PowerBI.tips Designer
YouTube video: Introducing Power Designer: Unleash Your Inner Report Wizard!
Newly released workloads
Profisee MDM Workload
Profisee has introduced the first native Master Data Management (MDM) workload within Microsoft Fabric, seamlessly integrating its MDM platform into Fabric’s environment. This breakthrough allows users to manage and unify enterprise data without leaving the familiar Fabric interface.Get it now: Profisee MDM Workload for Microsoft Fabric.
By embedding MDM capabilities directly into Microsoft Fabric, Profisee empowers organizations to:
- Ensure data consistency – Align and integrate data from multiple sources while enforcing standardized data governance within Fabric’s OneLake.
- Enhance data quality – Leverage intelligent matching, merging, and standardization to create trusted ‘golden records’ for your most critical business data.
- Streamline workflows – Manage data within Fabric itself, eliminating the need for external tools and reducing context switching.
- Accelerate AI & analytics initiatives – Deliver high-quality, consumable data ready to fuel enterprise applications of generative AI and advanced analytics.
For organizations leveraging AI and advanced analytics, having trusted, consumable (aka ‘gold medallion’) data is critical. Profisee’s deep integration with Microsoft Fabric ensures businesses can confidently rely on their data to make informed decisions and drive innovation.
This collaboration between Profisee and Microsoft represents a significant leap forward, enabling enterprises across industries to unlock insights, fuel opportunities, and finally bring their data into the age of AI to become data-driven at scale.
YouTube Video – Welcome to Profisee's featured workload in Microsoft Fabric
Figure: Profisee enables your medallion architecture to deliver consumable, trusted data for AI and analytics.
Figure: Profisee can match, merge and standardize data from different sources.
Lumel PowerTables Workload
The Lumel Fabric workload is now in preview. PowerTable allows business users to build no-code, writeback-enabled table apps on Microsoft Fabric. It connects LIVE (with bi-directional sync) to your cloud data warehouse tables in platforms such as Fabric SQL, Fabric Data Warehouse, Azure SQL, Databricks, Snowflake, Amazon Redshift, Google BigQuery, and PostgreSQL.Get it now: Lumel PowerTables Workload.
AI-generated content may be incorrect.">
There are 3 key use cases for PowerTable:
- Build tabular apps for data typically maintained in Excel such as product price lists, contract trackers, project status updates, etc.
- Manage master data / reference data / meta data supporting your reporting and planning applications. PowerTable supports forward-looking master data (unlike your ERP and MDM platforms) such as prospective customers or products that are not yet launched.
- Build applications on top of your semantic models and facilitate user data input and data writeback.
- Bulk insert and bulk edit records
- Cell-level commenting and collaboration
- Change log with audit trail
- Support for slowly changing dimensions
- Row-level CRUD user permissions
- Field-level permissions
- Sequential, multi-level approvals
- Rule-based approvals
- Outlook and Teams notifications
- Workflow automation
- Triggers and cascading updates
- Webhook integration
- … and more
Fabric_March_2025_Feature_Summary
AI-generated content may be incorrect.">
Unlike products like Airtable or Smartsheet, which typically struggle to handle more than tens or hundreds of thousands of rows, PowerTable is highly scalable and can support millions of rows. This is possible because our architecture separates the user interface from storage and compute and uses pushdown SQL statements to perform all processing on the underlying database of your choice.
Visit our website www.lumel.com to learn more.
PowerTables Introduction in action video.
SAS Decision Builder Workload (Preview)
Announcing SAS Decision Builder in preview, a new workload from longtime Microsoft partner and analytics vendor SAS. This powerful SaaS solution is designed to help organizations automate, optimize, and scale their decision-making processes in real-time, whether they are managing complex business rules, integrating machine learning models, or using Python code.Get it now: SAS Decision Builder workload.
There are numerous use cases across many industries, including financial services (loan approvals, financial products), manufacturing (product quality), and public sector (fraud identification, help with choosing a government service).
Decision Flow in SAS Decision Builder
Key features:
- Construct business rules and decisioning flows: Access and process data from all your sources to create decisions that align with business goals.
- Robust governance capabilities: Test, validate, schedule, run and monitor decisions within the unified Fabric environment. Adjust decisions as business needs evolve.
- Add value to your ML Models: Call your ML models within your Decision Flow to further enhance your decisions.
- Integrate Python code files into decisions: Bring advanced logic and flexibility into your automated processes.
Getting started:
Access SAS Decision Builder from the Workloads tab within your Microsoft Fabric instance. Select the SAS Decision Builder Workload Hub page, select ‘Add Workload,’ and transact through the Azure Marketplace process. Once complete, you can start building your decisions with SAS Decision Builder.
Striim SQL2 Fabric Workload
Microsoft Fabric’s robust ecosystem empowers partners to integrate custom capabilities through the Fabric Workload Development Kit. Striim proudly introduces a Fabric workload designed for real-time data ingestion, processing, and AI-driven analytics—delivering seamless integration, scalability, and actionable insights. The Striim workload builds on the Open Mirroring capabilities provided by Fabric to provide users a managed copy of their SQL mirrored data in the Fabric environment.Get it now: SQL2Fabric-Mirroring
Embedded workload integration: Striim embeds directly into the Microsoft Fabric Workload Hub, ensuring real-time data movement orchestration without leaving the Microsoft Fabric environment. Enterprises can leverage this integration for cohesive data workflows and enhanced discoverability.
Real-Time data replication & streaming: Striim offers sub-second latency ingestion from SQL Server, Oracle, PostgreSQL, MongoDB, and Databricks. These structured pipelines optimize data for Azure OpenAI, Fabric Copilot, and Vector Search—delivering AI-ready insights.
Enterprise security & performance: With end-to-end encryption, access control, and high-throughput event streaming via change data capture (CDC), Striim ensures secure, scalable, and low-latency performance across hybrid environments.
Integrated workload with Fabric: Striim seamlessly embeds within the Microsoft Fabric Workload Hub, enabling enterprises to orchestrate and automate real-time data movement without leaving the Fabric ecosystem.
AI-Optimized streaming: Striim delivers structured data pipelines tailored for Azure OpenAI, Fabric Copilot, and Vector Search. It ensures fresh, AI-ready data for machine learning models.
Use cases
- Hybrid Cloud migration & replication: Continuously mirror on-premises SQL Server databases into Fabric OneLake for real-time analytics.
- Cross-Cloud data synchronization: Seamlessly integrate data across Azure, AWS, and GCP, reducing data silos.
- AI-Driven insights & automation: Provide fresh data pipelines for AI-powered recommendations, fraud detection, and predictive analytics.
- Fabric Workload automation: Utilize the Workload Hub to streamline AI-powered data transformations and event-driven automation.