This month's update delivers key advancements across Microsoft Fabric, including enhanced security with Outbound Access Protection and Workspace-Level Private Link, smarter data engineering features like Adaptive Target File Size, and new integrations such as Data Agent in Lakehouse. Together, these improvements streamline workflows and strengthen data governance for users.
Contents
- Events & Announcements
- Fabric Data Days starts next week!
- FabCon is back from March 16-20, 2026, in Atlanta, GA
- Fabric Platform
- Keyboard shortcut support for horizontal tabs and object explorer
- Use Focus Mode to maximize editing space in Fabric
- Outbound Access Protection for Spark (Generally Available)
- Workspace-Level Private Link (Generally Available)
- Data Engineering
- Introducing Adaptive Target File Size
- Spark Connector for SQL databases (Preview)
- Spark Executor Rolling Logs: Easier Access for Large and Long Jobs
- Table Deep Links in Lakehouse Explorer
- Data Agent integration in Lakehouse
- Data Science
- New authoring experience for Data Agent Creators
- Introducing the Markdown Editor for Data Agent Creators
- Data Warehouse
- Reading and ingesting JSONL files
- Data source in OPENROWSET function
- Improving Concurrency in Fabric Data Warehouse - Compaction Preemption
- Real-time Intelligence
- Native Graph for Connected Data Insights that Power Intelligent Agents
- New Eventstream Source: MongoDB CDC
- Eventstream supports sourcing events with schema from an EventHub Source (Preview)
- Eventstream Supports Pause/Resume for Derived Stream
- Data Factory
- More file formats support in Copy job, including ORC, Excel, Avro, and XML
- Copy job CSV format now supports quote characters, escape characters, and encoding options
- Improved experience for Variable libraries in Dataflow Gen2
- Export Query Results (Preview)
Events & Announcements
Fabric Data Days starts next week!
Supercharge your career with 50+ days of data & AI. Join us for 2 months of learning, contests, live sessions, discount exam vouchers and community connection.
Designed for data professionals at every level, you’ll sharpen your real-world skills, gain a competitive edge in today’s market, and connect with a global network of peers and experts. It all kicks off on November 4th.
FabCon is back from March 16-20, 2026, in Atlanta, GA
Join us for the ultimate Power BI, Microsoft Fabric SQL, Real-Time Intelligence, AI, and Databases community-led event. The third annual FabCon Americas will feature sessions from your favorite Microsoft and community speakers, keynotes, more opportunities to Ask the Experts for 1:1 support, an engaging community lounge with opportunities to network and connect with your peers, a dedicated partner pre-day, a packed expo hall, Power Hour, Data Viz World Championship, and a can’t-miss attendee party at the Georgia Aquarium.
Register with code FABCOMM to save $200.
Fabric Platform
Keyboard shortcut support for horizontal tabs and object explorer
We’re enhancing the developer experience by adding keyboard shortcut support to the new multitasking experience with Horizontal Tabbing and Object Explorer. This allows developers to switch tabs and items efficiently using keyboard shortcuts, like professional IDEs.
To learn more, refer to the Keyboard shortcuts for Object Explorer and Horizontal Tabs documentation.
Use Focus Mode to maximize editing space in Fabric
When developers are editing an item, they often need to maximize the available screen space for the editor pane. Focus Mode allows you to hide navigation sidebar and the object explorer, giving you a distraction-free surface to concentrate on editing.
Select Focus Mode from the global header. The navigation sidebar and the object explorer (if you have it pinned) collapse, leaving only the editor pane maximized.
Refer to the Maximize editing space with focus mode documentation to learn more.
Outbound Access Protection for Spark (Generally Available)
With Outbound Access Protection for Spark, organizations can govern outbound connections from their Spark workloads in Fabric workspaces to external destinations and other Fabric workspaces within the same tenant providing organizations with granular controls to protect against data exfiltration. We are actively working to expand OAP support for additional experiences and plan to add support for Data Factory artifacts soon.
AI-generated content may be incorrect." />
Refer to the Workspace outbound access protection overview documentation to learn more.
Workspace-Level Private Link (Generally Available)
Private link for Fabric workspaces empowers organizations to secure their Fabric workspaces with fine-grained network isolation, allowing private, secure access from their virtual networks — without traversing the public internet.
AI-generated content may be incorrect." />
Refer to the Workspace outbound access protection documentation to learn more.
Data Engineering
Introducing Adaptive Target File Size
Data teams have long struggled with the File Size Dilemma: the ideal file size for a 10GB table can be disastrous when used for a 10TB table. The new Adaptive Target File Size feature uses Delta table telemetry to estimate the ideal target file size based on heuristics like the table's current size, and then automatically updates that target as the table grows over time.
You enable it once: SET spark.microsoft.delta.targetFileSize.adaptive.enabled = True
and the Fabric Spark Runtime handles the rest. For instance, a table that starts at 1GB might get 128MB target files, but as it grows to 10TB over the next five years, the target will automatically and incrementally adapt to be 1GB.
- Enhanced Data Skipping: Smaller, right-sized files in smaller tables enable up to 8x more possible file skipping during query execution.
- Reduced Update Costs: For operations like and , smaller target files mean the operations touch and rewrite less data, drastically reducing write amplification.
- Optimized Parallelism: The system ensures your files are not too big (limiting parallelism) or too small (overwhelming the scheduler), maximizing both read and write throughput.
The performance impact is substantial: a customer benchmark saw a 30% reduction in overall ELT cycle time, and during TPC-DS testing, the compaction phase ran nearly 1.6x faster, leading to the query phase running 1.2x faster.
AI-generated content may be incorrect." />
Learn more about this feature in our Adaptive target file size documentation and Adaptive Target File Size Management in Fabric Spark blog post.
Fast Optimize & Auto Compaction: End Write Amplification and Automate Maintenance
Compaction is necessary, but the hidden costs of traditional maintenance—like write amplification, constant manual intervention, and performance degradation between maintenance windows—have been a major pain point. We're tackling these costs with two complementary features.
Fast Optimize intelligently analyzes file bins and short-circuits compaction operations that aren’t estimated to provide a meaningful performance improvement.
- The Check: It evaluates whether the compaction job is estimated to produce files that meet the minimum target size and if there are enough small files to justify the work. If not, the operation is skipped, or its scope is reduced.
- The Benefit: This dramatically improves the idempotency of your jobs, meaning you can run them more frequently without wasting compute resources. In a study, Fast Optimize reduced the time spent doing compaction by 80% by over 200 ELT cycles by simply avoiding the long-term impact of write amplification.
AI-generated content may be incorrect." />
Enable it with: spark.conf.set('spark.microsoft.delta.optimize.fast.enabled', True)
Auto Compaction: Fix Small Files Before They Hurt
We’ve revamped the OSS Delta implementation of Auto Compaction with critical bug fixes, making it the recommended hands-off approach for table maintenance.
- Smart Triggering: It monitors your table’s file distribution as part of every write operation and automatically triggers compaction when small file accumulation crosses a smart threshold.
- Why it Matters: Instead of suffering the ‘sawtooth pattern’ of performance drops between scheduled jobs, your tables maintain optimal performance automatically. This has a direct cost benefit: one customer saw a near ~40% reduction in Spark Capacity Unit (CU) consumption by simply enabling this feature, stabilizing their production environment. In a study measuring the impact of Auto Compaction when running typical ELT pipelines, it resulted in 5x faster performance after 200 iterations:
AI-generated content may be incorrect." /> Auto Compaction handles the synchronous, immediate small file problem, while Fast Optimize makes your scheduled command efficient and idempotent. Together, these features bring you closer to a truly maintenance-free Lakehouse.
Learn more about these features in the Compacting Delta tables documentation and Introducing Optimized Compaction in Fabric Spark blog post.
Spark Connector for SQL databases (Preview)
Fabric Spark connector for SQL databases (Azure SQL databases, Azure SQL Managed Instances, Fabric SQL databases and SQL Server in Azure VM) in the Fabric Spark runtime is now available. This connector enables Spark developers and data scientists to access and work with data from SQL database engines using a simplified Spark API. The connector will be included as a default library within the Fabric Runtime, eliminating the need for separate installation.
AI-generated content may be incorrect." />
Operation Support
The Spark connector for Microsoft Fabric Spark Runtime is a high-performance library that enables you to read from and write to SQL databases. The connector offers the following capabilities:
- Use Spark to perform large write and read operations on the following SQL products: Azure SQL Database, Azure SQL Managed Instance, SQL Server on Azure VM, Fabric SQL databases.
- The connector comes preinstalled within the Fabric runtime, which eliminates the need for separate installations.
- While you’re accessing a table or a view, the connector upholds security models defined at the SQL engine level. These models include object-level security (OLS), row-level security (RLS), and column-level security (CLS).
Language Support
We are also introducing PySpark support for this connector, in addition to Scala. This means that you no longer need to use a workaround to utilize this connector in PySpark, as it is now available as a native capability.
Authentication Support
This connector supports multiple authentication methods which you can choose based on your need and setup.
To learn more about the Spark connector for SQL databases, please refer to the Spark connector for SQL databases documentation.
Spark Executor Rolling Logs: Easier Access for Large and Long Jobs
As Spark applications continue to grow in scale and duration, efficient log management and analysis have become increasingly critical. To address these evolving needs, we’ve introduced enhancements to the Spark History Server (for completed applications) and Spark UI (for running applications), enabling executor rolling logs for Spark 3.4 and above.
With this enhancement, when an executor log exceeds 16MB or the Spark job runs for more than one hour, the system automatically splits logs into hourly segments. This makes it easier to navigate, view, and download logs without dealing with extremely large files.
- View logs by hour to quickly pinpoint specific execution windows.
- Access the latest active logs while the job is still running.
- Download individual hourly logs or all logs together as needed.
This feature empowers users to locate and analyze logs from a particular timepoint with ease, while avoiding the hassle of downloading or opening a massive single log file.
Example: Executor Rolling Logs view.
AI-generated content may be incorrect." />
To learn more, refer to the Use extended Apache Spark history server to debug and diagnose Apache Spark applications documentation.
Table Deep Links in Lakehouse Explorer
When opening a lakehouse in Lakehouse Explorer, users see a preview of the first table by default. Now, users can generate a unique URL for any table in the lakehouse, allowing them to preview that specific table directly. By copying and sharing this URL with others who have access, recipients can open Lakehouse Explorer with the chosen table already previewed.
Simply click on the ‘…’ beside the table and select ‘Copy URL’. You can use this link to open the lakehouse in Lakehouse Explorer, where you’ll see a preview of the chosen table.
AI-generated content may be incorrect." />
To learn more, refer to the Navigate the Fabric Lakehouse explorer documentation.
Data Agent integration in Lakehouse
We’ve created a new integration for Data Agent directly within Lakehouse experiences. This enhancement is designed to streamline and accelerate your journey from raw data to actionable intelligence just with a few clicks.
How it works:
- Start in your Lakehouse.
- Select the ‘Add to data agent’ button in the ribbon.
- Select create a new Data Agent (auto-linked to your data source) or select an existing one.
- You’re taken directly to your Data Agent, with your data source ready to query.
- Use multitasking navigation to switch between Data Agent and your data source.
- Success, errors, and status updates appear in the notification hub.
AI-generated content may be incorrect." />
Try out Data Agent with your Lakehouse data now for quick, actionable, and intuitive AI-driven insights.
Data Science
New authoring experience for Data Agent Creators
The new UI introduces two dedicated tabs—Data and Setup—making it easier than ever to design, test, and refine your agent. Under the Setup tab, you can see and apply all the relevant settings for your data sources, while seamlessly switching back to the Data tab to chat with your agent and test how your configurations affect its behavior—without losing context.
This side-by-side view lets you explore your schemas while authoring instructions and example queries, improving speed and accuracy. And because all configurations are integrated with CI/CD and Git, tracking changes over time is effortless. This update gives creators the flexibility to multi-task and iterates quickly, setting a new standard for building powerful, tailored Data Agents.
AI-generated content may be incorrect." />
To learn more about how to configure your Data Agent, refer to the Data agent configurations documentation.
Introducing the Markdown Editor for Data Agent Creators
Creators can now write their agent and data source instructions directly in Markdown—a clean, structured, and highly readable format that’s perfect for organizing context. Markdown makes it simple to format instructions with headers, lists, and code blocks, helping you clearly express complex logic or multi-step processes.
It’s also a natural way to pass context to the Data Agent—LLMs are optimized to understand Markdown, which means your instructions are more likely to be interpreted accurately and consistently. This update not only improves the quality of responses but also enhances your authoring workflow, making it easier to iterate and maintain well-structured instructions over time.
AI-generated content may be incorrect." />
To learn more about how to author agent and data source instructions, refer to the Best practices for configuring your data agent documentation.
Data Warehouse
Reading and ingesting JSONL files
Data Warehouse enables you to efficiently read and ingest JSONL (JSON Lines) files by leveraging the OPENROWSET function. With the OPENROWSET function, you can reference one or many JSONL files and read their content as a set of rows.
SELECT *
FROM OPENROWSET(BULK '/Files/jsonl/year=*/month=*/tweets-*.jsonl' )
Each property found in the JSON documents is returned as a column, while each value becomes a cell in the resulting dataset. This makes it straightforward to transform semi-structured JSONL data into a tabular format for analysis or further processing.
In addition to reading flat JSONL files, the OPENROWSET function also enables you to read complex JSON structures that contain nested JSON sub-objects. By using the ‘WITH clause, you can explicitly define all output columns and their data types, mapping each column to the corresponding JSON path within the source files.
AI-generated content may be incorrect." />
The ‘WITH’ clause allows you to easily extract and transform deeply nested data into a tabular format, making it accessible for further analysis and reporting.
Beyond just reading data, the OPENROWSET function also supports data ingestion. You can ingest the rows returned by OPENROWSET directly into existing tables using the INSERT statement. Alternatively, a new table can be created and populated from the results of the OPENROWSET function used in the CREATE TABLE AS SELECT (CTAS) statement. This streamlined approach simplifies the process of bringing external JSONL data into your data warehouse environment, making it easy to work with large volumes of semi-structured data using T-SQL syntax.
Learn more about the reading and ingesting JSONL files in the Query and ingest JSONL files in Data Warehouse and SQL Analytics Endpoint for Lakehouse (Generally A... blog post.
Data source in OPENROWSET function
The data_source option in the OPENROWSET function allows you to simplify data access to files in the lake by using a relative file path and referencing a data source that points to a root folder location. You can create an external data source with a location that refers to the location of the lakehouse folder:
CREATE EXTERNAL DATA SOURCE MyLakehouse
WITH ( LOCATION = 'https://onelake.dfs.fabric.microsoft.com/{wsid}/{lhid}' );
After creating an external data source and set its location to the root lakehouse URI, you can reference this data source by name in your OPENROWSET queries.
AI-generated content may be incorrect." />
The data source option enables you to use relative file paths, making your queries cleaner and easier to maintain.
Additionally, when running OPENROWSET queries on SQL endpoints for Lakehouse, if you don't specify the data source option, the SQL endpoint automatically defaults to the lakehouse root location. This means you can use shorter relative paths in your queries to access lakehouse files, streamlining your workflow and reducing the need for fully qualified paths.
Find more information about the DATA_SOURCE in OPENROWSET in the Simplifying file access in OPENROWSET using data sources and relative paths (Preview) blog post.
Improving Concurrency in Fabric Data Warehouse - Compaction Preemption
As part of our ongoing investments to enhance concurrency and reliability in Fabric Data Warehouse, we’re introducing Compaction Preemption, a smart new capability that intelligently prevents write-write conflicts caused by background compaction tasks. These tasks optimize storage by rewriting small, inefficient parquet files into fewer, larger files, but can interfere with user operations like UPDATE, DELETE or MERGE.
With this enhancement, the system detects active user workloads and preempts compaction tasks before they commit, reducing the likelihood of query failures during concurrent operations. This ensures smoother experiences for workloads with frequent updates and deletes, especially in high-throughput environments.
We will continue to evolve our conflict avoidance strategies for Compaction Preemption. Explore the latest blog post to gain insights into how compaction preemption works in Microsoft Fabric Data Warehouse, along with best practices to avoid conflicts Resolving Write Conflicts in Microsoft Fabric Data Warehouse.
Real-time Intelligence
Native Graph for Connected Data Insights that Power Intelligent Agents
Fabric Graph is Microsoft’s first horizontally scalable, native graph data management, analytics, and visualization service. Inspired by LinkedIn's graph engine that efficiently maps over a billion interconnected entities, Fabric Graph helps organizations uncover and act on insights hidden in their most complex, interwoven data. It is a foundation for the next generation of data-rich, reasoning-capable agents.
AI-generated content may be incorrect." />
To learn more about Explore capabilities, use cases, and getting-started guides, refer to the Graph in Microsoft Fabric (preview) documentation.
New Eventstream Source: MongoDB CDC
Fabric Eventstream now supports MongoDB Change Data Capture (CDC) as a fully managed connector, expanding its rich set of streaming data sources. With this connector, you can stream CDC events from MongoDB to Eventstream for real-time processing and analytics.
Highlights of MongoDB CDC Connector
- Compatible with all MongoDB deployments, including MongoDB Atlas, cloud-hosted, and on-premises.
- Captures real-time database changes and streams them into Eventstream for immediate processing.
- Built on Debezium, a widely adopted open-source CDC framework with strong community support, ensuring flexibility and reliability
You’ll find this new connector in the Real-Time Hub.
AI-generated content may be incorrect." />
The following example demonstrates how Eventstream using MongoDB Atlas as the source, processing CDC events in real time, and routing them to an Eventhouse destination.
AI-generated content may be incorrect." />
To learn more, and guidance getting started, refer to the Add MongoDB CDC source to an eventstream documentation.
Eventstream supports sourcing events with schema from an EventHub Source (Preview)
Fabric Eventstream now supports sourcing events from an EventHub source while enforcing schema on the payloads. This integration enables data engineers and developers to build governed, type-safe event streaming pipelines using header-based schema validation. This integration leverages Fabric's Schema Registry to validate events at ingestion, eliminating the need for downstream defensive coding and data quality checks.
- Use Schema Registry to validate events from Azure EventHub across multiple event types flowing through a single EventHub namespace.
- Support for header-based schema matching, where UTF-8 string headers deterministically map events to registered Avro schemas.
- While events flow through the Eventstream, the connector validates payload structure, data types, and required fields before forwarding to downstream destinations like Eventhouse.
Header-Based Routing
The integration uses custom EventHub properties (headers) to route events to the correct schema for validation. This approach supports multiple patterns:
- One-to-one mapping: A single header key-value pair uniquely identifies one schema.
- Multi-schema support: When multiple types of events flow through a single EventHub, each validated against its registered schema.
- Validation at the gate: Malformed events are rejected at ingestion and logged in Fabric Diagnostics, preventing data quality issues downstream.
To set up header-based routing, enable schema-mode when adding your EventHub as a source, select ‘Dynamic schema via headers’, and provide unique header-value pairs to enable Fabric Eventstreams to automatically select and apply schemas to the incoming events.
AI-generated content may be incorrect." />
To learn more about sourcing schema-driven events from EventHub, please refer to the Add Azure Event Hubs source to an eventstream documentation.
Eventstream Supports Pause/Resume for Derived Stream
Eventstream customers can now pause/resume their traffic for their derived stream. This capability gives customers greater operational control and flexibility over their event-driven workflows.
- Cost Optimization: Pause traffic during maintenance windows or low-demand periods without deleting or recreating streams.
- Operational Continuity: Resume traffic instantly when ready, reducing downtime and avoiding complex reconfigurations.
- Resource Efficiency: Dynamically manage throughput based on business needs, improving overall system efficiencies.
With pause/resume, customers can maintain agility, reliability, and cost-effectiveness in managing real-time pipelines.
Data Factory
More file formats support in Copy job, including ORC, Excel, Avro, and XML
Copy job supports even more file formats—including ORC, Excel, Avro, and XML—so you can move data from any source to any destination with greater flexibility. Whether you're working with structured tables or semi-structured files, Copy job lets you seamlessly ingest, and distribute your data in the format that best fits your business and analytics needs.
AI-generated content may be incorrect." />
For more information about Copy job, refer to the What is Copy job in Data Factory documentation.
Copy job CSV format now supports quote characters, escape characters, and encoding options
It is now possible to set the Quote character, Escape character, and encoding options when transferring data in CSV format through a Copy job.
AI-generated content may be incorrect." />
With these options, you can:
- Precisely control how text values and delimiters are handled, ensuring accurate data parsing.
- Escape quotes or delimiters within text fields to avoid broken records.
- Choose from a wide range of encoding formats—from UTF-8 and UTF-16 to regional encodings like BIG5, GB18030, and SHIFT-JIS—to ensure compatibility across global data sources and destinations.
This enhancement gives you the flexibility to handle any CSV structure, maintain data fidelity, and enable seamless cross-system integration.
For more information about Copy job, refer to the What is Copy job in Data Factory documentation.
Improved experience for Variable libraries in Dataflow Gen2
In September we unveiled the new integration between Variables libraries and Dataflow Gen2. Improvements have since been made to this experience.
- Gateway support: evaluations running through a Gateway can now leverage variable libraries during run time. The minimum required gateway version is 3000.282.
- Power Query editor support: when creating a solution using the Dataflow editor, your variables will be able to get evaluated in the Power Query editor.
AI-generated content may be incorrect." />
We’ve heard your feedback about how you wish to get more guidance on how to better take advantage of this integration and to that end we’ve created a full solution architecture guide and a thorough tutorial on how you could implement your own solution using Variable libraries in Dataflow Gen2.
Feel free to share your comments, inquiries, feedback and suggestions about this experience through our Data Factory community forum.
Check out the tutorial on using Fabric variable libraries with Dataflow Gen2: Variable references in Dataflow.
Export Query Results (Preview)
Export Query Results in Power BI Desktop unlocks seamless data portability and collaboration across Microsoft Fabric. This feature empowers analysts to take transformed, cleaned data from Power Query and export it directly to trusted destinations - such as Dataflows Gen2, Lakehouses, and other cloud stores - without relying on complex workarounds or third-party tools. By meeting users where they work (Power BI Desktop) it reduces duplication of effort, accelerates data sharing, and drives interoperability across Fabric workloads.
To enable this feature, select it in the Preview feature settings in Power BI Desktop.
Users can select export query results from the PQ ribbon.
Select an online destination.
And export their queries to this destination, where they can choose to modify the Dataflow. 
The result is improved data gravity in OneLake, streamlined refresh and monitoring capabilities, and a minimal-click experience that supports both new and existing destinations. Ultimately, this feature bridges the gap between desktop analytics and enterprise-scale data integration, enabling organizations to maximize the value of their data investments.
To learn more about how to use this feature, refer to the Export Query Results in Power BI Desktop documentation.