Modernize data integration strategy with Microsoft Fabric when using Snowflake and Databricks
Challenges solved by Microsoft Fabric Microsoft Fabric is an end-to-end analytics platform that brings together data integration, data engineering, warehousing, real-time analytics, business intelligence, and AI-related experiences. It has grown tremendously within the community because it is solving a key problem: lowering the integration tax across data and analytics systems. To understand the integration tax, consider a mid-sized or large organization that wants to perform analytics and data processing. It may use several source systems, an analytics platform such as a warehouse or lakehouse, and yet another platform or vendor for orchestration. To generate insights, the organization must integrate these different systems (hence the term integration tax). Fabric brings together a broad set of analytics capabilities within one platform, which can reduce the number of separate integrations an organization needs to design and operate. You can bring data into Fabric, store it in a warehouse or lakehouse, query and analyze it with Spark or T-SQL, visualize insights with Power BI, and support real-time scenarios with Real-Time Intelligence. Fabric also interoperates with platforms such as Snowflake and Databricks, helping organizations unify data and analytics across technologies while minimizing data movement and duplication. The following diagram shows how these experiences come together as part of the Microsoft Fabric platform. OneLake serves as Fabric's unified logical data layer. Fabric-managed data resides in OneLake, while shortcuts and some mirroring scenarios can reference data that remains outside OneLake. If one team owns the data and another team in the organization needs access, the second team does not need to duplicate it. Instead, it can create a Fabric shortcut and work with the same copy of the data. Beyond storage, OneLake also enables data-sharing and governance patterns across domains and workspaces. Organizations can expose data through shortcuts while maintaining ownership within the producing team, enabling data mesh architecture without duplicating datasets. Integration options We have discussed shortcuts briefly. Now let's look at how they work and at the other options available for making data accessible in Fabric: Shortcuts: Fabric lets you create a pointer to data in its existing location and access it without moving it. A shortcut is similar to a symbolic link or a Windows shortcut: the data source can differ from the location where users access it. For example, data can remain in Amazon S3 while a Fabric shortcut provides access to the required data. To users, the data appears to reside in Fabric even though it is retrieved from S3. Shortcut transformations also support selected transformations on shortcut data. Mirroring: For supported data sources, mirroring provides you with a low latency option to bring your data into Fabric. Depending on the source system, mirroring either continuously replicates data into OneLake or exposes the source through metadata mirroring and shortcuts. This approach reduces the need to manage pipelines or infrastructure and can offload analytical workloads from the source system. For example, if source data resides in an on-premises SQL Server database, you can create a SQL Server mirror in Fabric and analyze the latest available data there. Open Mirroring: This is an extension of mirroring for any data sources that are specific to your organization. Open Mirroring enables applications or partners to write changes directly into an open mirroring landing zone, after which Fabric converts and manages the data as Delta tables. Pipelines: are an option when a solution requires explicit data movement, multi-step orchestration, complex transformations, or integration across multiple sources. The following diagram illustrates how mirroring and shortcuts complement Fabric compute experiences by either bringing data into Fabric or making externally stored data available without traditional ingestion. Decision guide If you are using Snowflake and/or Databricks in your organization and if you are deciding which option best fits your use case, the following guide compares the available approaches and highlights when each one is most appropriate. Source platform Fabric integration Data movement What happens Best for Snowflake on Azure Mirroring: managed tables and views Replicated Table and view data is replicated into OneLake for Fabric workloads. Fabric-native analytics, BI, engineering and data science Mirroring: Iceberg No data copy Metadata is replicated; shortcuts access the existing data, which OneLake virtualizes as Delta. Existing Snowflake Iceberg estate Snowflake Iceberg in OneLake Single copy Snowflake writes Iceberg data to OneLake so both platforms use the same copy. Strategic shared-data architecture DirectQuery / Import Query / Optional copy Power BI queries Snowflake directly or imports the data Existing BI Data Factory / Pipelines Copied Traditional ingestion / ETL Migration and transformation Snowflake on AWS Mirroring: managed tables & views Replicated Snowflake data is replicated cross-cloud into OneLake Fabric-native analytics Mirroring: Iceberg No table-data copy Metadata is mirrored; shortcuts access the existing Iceberg data. Existing Iceberg architecture Iceberg Shortcut No copy Fabric references external Iceberg tables and virtualizes them as Delta. Open-format interoperability DirectQuery / Import Query / Optional copy Query Snowflake directly or import Existing BI Data Factory / Pipelines Copied Traditional ingestion / ETL Migration and transformation Azure Databricks + Unity Catalog Catalog Mirroring No copy Fabric mirrors the catalog structure; underlying data is accessed through shortcuts rather than replicated. Broad Unity Catalog integration with Fabric Delta Shortcut from ADLS No copy OneLake shortcuts can expose Delta data from external storage without copying it. Existing Databricks Delta lakehouse Iceberg Shortcut from ADLS No copy OneLake can shortcut compatible Iceberg tables and virtualize their metadata as Delta for Fabric workloads. Databricks Iceberg / open-format interoperability Data Factory / Pipelines Copied Traditional ingestion / ETL Migration and transformation Databricks on AWS / S3 Delta Shortcut from S3 No copy Fabric references open-format Delta data in S3 through OneLake shortcuts. Existing Databricks Delta lakehouse on AWS Iceberg Shortcut from S3 No copy Fabric shortcuts the Iceberg table in S3 and OneLake generates virtual Delta metadata so Fabric workloads can consume it. Databricks Iceberg and cross-platform interoperability DirectQuery / Connector Query Fabric/Power BI queries the Databricks serving layer Existing BI Data Factory / Pipelines Copied Traditional ingestion / ETL into Fabric Migration and transformation Figure: Fabric integration options by source platform, data-movement pattern, and use case. Integration recommendations The following options may provide a starting point. The appropriate design depends on source support, data residency, freshness, transformation, governance, security, networking, cost, and operational requirements. If you're using... Option to consider Why Snowflake managed tables / views Mirroring Replicate into OneLake for Fabric-native analytics Snowflake + existing Iceberg Catalog Mirroring Access Iceberg without copying the underlying table data Snowflake on Azure + shared lake architecture Snowflake Iceberg in OneLake Snowflake and Fabric can operate around a shared copy Azure Databricks + Unity Catalog Catalog Mirroring Bring the catalog into Fabric with no underlying data replication. Azure Databricks + Delta on ADLS Delta Shortcut Access Delta data in place without copying it. Azure Databricks + Iceberg on ADLS Iceberg Shortcut Access Iceberg in place and virtualize it for Fabric workloads. Databricks AWS + Delta on S3 Delta Shortcut to S3 Access the existing Delta data without copying it into OneLake. Databricks AWS + Iceberg on S3 Iceberg Shortcut to S3 Access existing Iceberg data and expose it to Fabric workloads through metadata virtualization. Need traditional ETL / migration Data Factory / Pipelines Explicit data movement and transformation Figure: Recommended starting points for common Snowflake and Databricks integration scenarios. User layer: AI Foundry and Copilot Studio with data agents Making enterprise data accessible in Fabric is only the first step. Once that data is available through a governed integration pattern, organizations can build AI experiences that allow users and applications to interact with it using natural language. Microsoft Fabric Data Agents allow users to interact with governed data using natural language and can generate and execute queries across supported Fabric experiences. Data Agents make it possible to gain insights directly from data without building an agent from scratch. After creating a Data Agent, you can publish it to applications or other consumer experiences. As shown in the AI Agent Decision Tree, several consumer-layer options are available. This article focuses on two: Copilot Studio and Microsoft AI Foundry. Copilot Studio Copilot Studio is a low-code platform for building agents that combine generative AI with enterprise data and system actions. It is a practical option for employee or customer self-service, support automation, conversational access to Fabric data, and workflows where an agent needs to retrieve information or invoke an existing business process. After publishing a Fabric Data Agent, you can connect it to Copilot Studio to build a custom agent that also uses other data sources. You can then publish the agent to Microsoft 365 experiences such as Teams. This option works well for common, out-of-the-box scenarios that a large language model can support without extensive customization. Microsoft AI Foundry Microsoft AI Foundry is where you would go when you need more control over how an AI application or agent is built and operated. It brings models, agents, tools, evaluation, and monitoring into one place, while still letting developers choose how much code and customization they want. In the context of Fabric, a Foundry agent can use a Fabric data agent as one of its tools, so it can work with governed enterprise data alongside other knowledge sources and business systems. This option is well suited to use cases that require greater control over model behavior, orchestration, evaluation, and customization. Microsoft AI Foundry is typically best suited for professional developer scenarios. Next steps Microsoft Fabric provides multiple approaches for connecting existing data estates to analytics and AI workloads. Start by identifying whether your scenario requires data replication, virtualization, or explicit data movement, and then use the decision guide to choose the appropriate starting point. To explore these capabilities in more detail: Learn how OneLake shortcuts make external data available to Fabric without traditional data movement. Explore Mirroring in Microsoft Fabric for supported databases and data platforms. Learn how Microsoft Fabric Data Factory pipelines support orchestration and data movement. Explore Fabric data agents for natural-language interaction with governed Fabric data. Learn how to connect a Fabric data agent to Copilot Studio or Microsoft AI Foundry.
Fabric Policies: Granular governance at scale (Preview)
As Microsoft Fabric grows across teams, workspaces, and capacities, governance becomes harder to manage with broad tenant-wide switches alone because governance models are not one-size-fits-all. Fabric Policies provide granular controls that can fit your governance model by defining what actions are allowed, to whom and where. The result is governance that scales with your Fabric estate. Teams keep the flexibility they need to work; capacity admins can protect the resources they manage, and central administrators gain consistent controls and visibility across the organization. Fabric Policies help organizations: Match controls to organizational boundaries - Apply organization-wide rules at tenant scope and resource-specific rules at capacity scope. Protect sensitive actions without slowing routine work - Keep approved self-service available while restricting higher-risk actions to the right users and context. Apply consistent guardrails at scale - Manage policies centrally through Policy Sets, Policies Center, auditing, and public APIs. What you can govern Fabric Policies currently provide controls for item creation, external data sharing (coming soon), and selected security-sensitive workspace settings. Control item creation by capacity Capacity admins can set a policy for any Fabric item type in workspaces assigned to their capacity (Power BI item creation is not yet enforced). An admin can allow the Data Engineering security group to create Dataflow Gen2 items in selected data platform workspaces and the AI Builders security group to create Data agent items in selected AI workspaces. Matching requests proceed; requests that do not match a complete Allow rule are blocked. This control supports a gradual Fabric rollout. Organizations can make specific item types available to approved teams and workspaces as their governance requirements are met, rather than enabling every item type everywhere at once. Item creation requires both controls to allow the request: the applicable Fabric item creation setting and the active Item creation policy. Figure: Advanced Item creation policy with separate Allow rules for Dataflow Gen2 and Data agent. Protect security-sensitive workspace settings Fabric admins can protect selected settings without removing broader workspace administration capabilities. An admin can allow the Central IT team to edit Outbound access protection, Inbound access protection, and Apply customer-managed keys (CMK) in sensitive workspaces. Other workspace admins can continue to edit the rest of the settings. The policy refines what existing workspace admins can change; it does not grant the Workspace Admin role. Govern external data sharing (Coming Soon) Fabric admins can support partner collaboration while controlling who can share, which workspaces the data comes from, its sensitivity label, and the recipient email domain. An admin can allow the Finance security group to share items labeled Confidential from Finance workspaces with recipients at CALDOVA.com. Sharing proceeds only when all configured conditions match; other requests are blocked. External sharing can be allowed by either the existing External data sharing tenant setting or a matching and active policy rule. How Policies work On Policy Set creation, an administrative scope is selected, such as the tenant or a capacity. This defines the available policy types and the administrator role required to activate the Policy Set for that scope. For each policy, administrators choose Allow all, Block all, or Advanced. Advanced policies use an allow-based model: All conditions within a rule must match. A match on any complete rule allows the action. If no rule matches, the action is blocked. Once the Policy Set is activated, Fabric evaluates its policies whenever a governed action occurs within that scope. Only one Policy Set can be active for a tenant or an individual capacity. Activating a replacement changes which definitions govern that scope. Existing access requirements remain in effect. Policies do not grant Fabric roles, workspace access, or item permissions. A user must first have the access required for the action; the policy then adds a governance decision for that action and context. Fabric Policies are available in preview for supported Fabric capacities and workspaces. Standard Fabric capacity and workspace requirements apply. Get started with Fabric Policies In Admin portal > Tenant settings, enable Policies (Preview). In a Fabric-capacity-backed workspace, create a Policy Set and choose Tenant or Capacity scope. Configure a policy with Allow all, Block all, or Advanced Allow rules, then review the users, actions, and workspaces covered. Activate the Policy Set as an administrator of the target scope. For capacity scope, select a capacity you administer. Review the related Fabric setting and policy together, then verify the active result in OneLake Catalog > Policies Center. For repeatable deployment across environments or capacities, use Fabric REST APIs to automate Policy Set and policy management at scale. API operations and activation remain subject to the required item permissions and administrator role. Policies best practices and management Design Advanced policies around every allowed scenario. List every user, item type, and workspace combination that must continue to work. Because Advanced policies are allow-based, a request is blocked when no complete rule matches. Align capacity policies with capacity ownership. Item creation policies govern workspaces assigned to the selected capacity. Each capacity can have its own active Policy Set, giving capacity admins control within their resource boundary. Review policy coverage whenever workspaces move between capacities. Use a dedicated management workspace. A Policy Set can be hosted in one supported Fabric-capacity-backed workspace and activated for a different capacity. A dedicated workspace can simplify access, ownership, and lifecycle management across multiple capacities. Separate editing from activation. People with Read, or the workspace Viewer role, can view a Policy Set. People with Read and Write, or a workspace Contributor, Member, or Admin role, can edit it. Activation still requires the administrator role for the target scope: Fabric admin for tenant scope or capacity admin for capacity scope. Validate before broad rollout. Start with representative users and workspaces and review each allowed scenario. Monitor and refine. Use Policies Center to review active policies and the Microsoft 365 Audit Log to follow Policy Set and rule lifecycle operations. Conclusion As Microsoft Fabric adoption scales across teams, workspaces, and capacities, governance must scale with it. Fabric Policies provide the flexibility to apply the right controls in the right contexts while maintaining a consistent governance model across the organization. By combining centralized oversight with granular policy enforcement, organizations can confidently expand their Fabric footprint without sacrificing security or control. Learn more about policies.Workspace Outbound Access Protection (OAP) for Operations Agent and Fabric Maps (Preview)
Co-authors: Andre Terceros and Bodhisatva Gautam Workspace Outbound Access Protection (OAP) in Microsoft Fabric helps WS admins secure outbound connections from workspace items to external resources. Administrators can control outbound access by blocking unwanted connections by default and allowing only approved connections through configured rules. As organizations adopt AI-powered operations at scale, governance and security remain at critical requirements. With this preview release, Microsoft Fabric introduces Outbound Access Protection (OAP) for Operations Agent and Fabric Maps, enabling workspace administrators to control the outbound actions an agent can perform. OAP for Operations Agent When OAP is enabled, Operations Agent continues to perform core functions including reasoning, recommendation generation, rule evaluation, and telemetry collection. However, outbound actions are governed by the workspace's configured access policies. Administrators gain greater visibility through in-product notifications, Teams messaging experiences, and the Operations Agent Activity Log, making it easier to identify and troubleshoot blocked actions. Figure: Outbound Access Protection helps workspace administrators govern outbound Operations Agent actions while providing clear visibility when actions are blocked. What's new with Operations Agent and OAP? With Outbound Access Protection support for Operations Agent, workspace administrators gain more control over how agent-initiated actions interact with external services and resources. The following updates improve governance, visibility, and operational oversight while helping organizations continue to automate with confidence. Govern outbound agent actions through workspace-level OAP policies. Control whether Operations Agent can send Teams notifications based on allowed connections. Prevent unauthorized cross-workspace actions when OAP policies restrict outbound access. Receive clear visibility when actions are blocked through in-product notifications and Teams messaging experiences. Monitor agent activity and OAP-related outcomes through the Operations Agent Activity Log. Figure: Activity Log Operations Details show the steps the agent completed and each action’s status, including when an action is blocked. OAP for Maps Fabric Maps can connect to a variety of data sources, including Lakehouse across Fabric workspaces, Kusto databases (KQL), Ontologies as well as external geospatial services such as Web Map Services (WMS), Web Map Tile Services (WMTS), and Web Feature Services (WFS). Workspace Outbound Access Protection helps organizations maintain security and compliance by governing outbound connections and requiring explicit approval before a map can access external resources. With OAP enabled, Fabric Maps follows a "default deny" model, ensuring only approved destinations can be accessed. How it Works When Workspace Outbound Access Protection is enabled on a workspace, Fabric Maps evaluates outbound connectivity at multiple stages: Save and Load Operations Whenever a map is created, updated, or loaded, Fabric evaluates all referenced data sources against the workspace policy. If a data source isn't permitted: References are blocked Disallowed sources are redacted when the map is opened Attempts to save unsupported references are rejected Runtime Data Access Map visuals continuously retrieve data such as: Map tiles Geospatial features Query results Before each outbound request is executed, Fabric validates the destination against the workspace's outbound access protection policy. External Service Validation External geospatial services such as WMS, WMTS, and WFS are validated through Data Movement and Transformation Services (DMTS). DMTS acts as a policy enforcement layer and ensures only explicitly approved external endpoints can be reached. Supported Connection Scenarios Lakehouse Lakehouse connectivity receives the most flexibility under the current release. Lakehouse within the same workspace are always allowed. Lakehouse in other workspaces can be allowed using Data Connection Rules. Administrators maintain granular control over cross-workspace access. External Geospatial Services Organizations can securely connect to approved WMS, WMTS, and WFS services. By using Data Connection Rules with the Geospatial Web Services connection type, administrators can create an allow list of approved endpoints. This enables scenarios such as: Accessing enterprise GIS systems Consuming approved mapping services Integrating with external geospatial data providers Kusto Databases and Ontologies Connections to Kusto databases and Ontologies within the same workspace continue to function normally. Cross-workspace connectivity for these sources is currently blocked when OAP is enabled. Future releases will introduce additional policy controls for these connection types. Configuring Fabric Maps with Workspace OAP Enabling protection is straightforward: Enable Workspace Outbound Access Protection for the workspace. Create Data Connection Rules that define approved destinations. Add approved external geospatial services using the Fabric connection experience. Configure Geospatial Web Services rules to authorize required endpoints. Once configured, Fabric Maps can access only the approved destinations specified by policy. Security by Design Workspace OAP for Fabric Maps follows several key security principles: Explicit Allow Lists: Only approved destinations can be reached. All other outbound connectivity is automatically blocked. Consistent Enforcement: Policies are applied during authoring, loading, and runtime execution, ensuring continuous protection throughout the lifecycle of a map. Fail-Closed Protection: If policy validation services become unavailable, Fabric denies cross-workspace connections by default. This fail-closed behavior helps prevent unintended data exposure during service disruptions. What’s next? We are actively working to expand OAP support for additional experiences and plan to add support for Power BI Semantic Models and Reports soon in GA soon. Your feedback is essential! Let us know how we can make Fabric even more secure and flexible for your workloads by sharing your feedback at Fabric Ideas – Microsoft Fabric Community Learn more Workspace outbound access protection overview Outbound Access Protection for Operations Agent (Preview) Workspace Outbound Access Protection for Fabric Maps
Capacity Operation Events in Real-Time Hub (Preview)
Capacity Operation Events in Real-Time Hub is now available as a preview, giving Microsoft Fabric capacity administrators access to detailed, operation-level telemetry in near real time. Capacity Operation Events build on Capacity Overview Events by helping administrators move beyond understanding overall capacity health to identifying the specific operations, workspaces, and artifacts contributing to capacity consumption, throttling, and performance issues. From capacity health to root cause analysis Capacity administrators have long relied on tools such as the Capacity Metrics App to understand utilization patterns and investigate capacity-related issues. This augments those capabilities by providing direct, near real-time access to the underlying operational signals. It exposes granular telemetry for individual operations running on a Fabric capacity, enabling organizations to: Identify the operations driving capacity consumption. Investigate throttling delays and performance bottlenecks. Understand which workspaces and artifacts contribute to capacity pressure. Correlate workload activity with capacity utilization trends. Build custom monitoring, governance, and automation solutions using Fabric Real-Time Intelligence, including the use of Microsoft Activator for designing automated triggers with no code. Together with Capacity Overview Events, administrators now gain both a high-level and detailed view of capacity behavior: Capacity Overview Events help answer: What is happening to my capacity? Capacity Operation Events help answer: Which operations are causing it? Deep operational visibility Capacity Operation Events provide detailed telemetry for individual operations executed on a Fabric capacity. Each event includes information such as workspace and item details; capacity unit consumption; operation duration; operation start time and smoothing window attribution. This enables administrators to quickly answer questions such as Which refreshes consumed the most capacity yesterday? Which operations experienced throttling delays? Which operations are contributing most to capacity pressure? Administrators can in turn translate these insights into actions: Create alerts when high-cost operations exceed defined thresholds. Detect and investigate throttled workloads in near real time. Trigger automated notification and/or remediation workflows. Stream operational telemetry into Eventhouse for historical analysis and retention. Build custom dashboards tailored to your monitoring and governance needs. Integrate capacity telemetry into existing operational platforms and workflows. Getting started For complete schema and event details for operation events, see the Explore Fabric capacity operation events in Fabric Real-Time hub documentation. You can also learn more about overview events in the Explore Fabric capacity overview events in Fabric Real-Time hub documentation. The Capacity Events Accelerator GitHub repository is also available as an even easier way to get started with capacity events.Choosing your medallion pattern in Fabric Data Warehouse
Coauthor: Artur Vieira Part one of a series on medallion architecture with Fabric Data Warehouse. Medallion architecture is one of the most common patterns for organizing data in Microsoft Fabric, but successful implementations require a series of design decisions — from choosing the right architecture pattern to securing, governing, and optimizing your workloads. In this five-part series, we'll walk through the practical choices that shape a modern medallion implementation in Fabric Data Warehouse (DW), sharing recommendations, tradeoffs, and real-world guidance along the way. In this installment, we'll focus on the first and most important decision: choosing the right medallion pattern for your workload. Why this matters Most teams designing a medallion architecture in Microsoft Fabric start with the wrong question: "Should I use a Lakehouse or a Warehouse?" The better question is, "How much Spark do I actually need?" Your answer will shape everything from development workflows to security and long-term maintenance. Quick level-set: medallion organizes data into three layers — Bronze (raw), Silver (enriched), and Gold (curated) — to progressively improve data quality and structure. That’s the whole definition you need. The interesting part is how you map those layers onto Fabric. The key decision: Data Warehouse, Lakehouse, or both? Fabric gives you two major analytics storage options on OneLake: Lakehouse and Data Warehouse. Because Fabric DW is an enterprise-scale SQL warehouse built on the open Delta Lake format in OneLake — combining a relational SQL engine with lake storage, so data is stored as Delta Parquet with ACID transactions and time travel — you don’t have to choose between “warehouse” and “lake.” You’re really choosing how much of the pipeline you run in T-SQL versus Spark. That leads to two patterns worth recommending: an all-in-one or hybrid approach. Pattern A: All-in-One Data Warehouse Use Fabric DW for Bronze, Silver, and Gold, separated by schemas (for example Bronze.*, Silver.*, Gold.*) or by separate warehouses, with data flowing raw → curated entirely within the warehouse using T-SQL or Data Factory pipelines. Best when: most of your data is structured (or can be structured on load) and your team prefers SQL-centric development. Why it’s nice: one engine, one skill set, transactions and views for every hop. Pattern B: Lakehouse + Warehouse Hybrid Land raw data in a Lakehouse for Bronze (and optionally Silver), where you can use Spark for complex prep, then implement Gold in the Warehouse as the SQL serving layer for BI. This works seamlessly because OneLake underpins both — Silver or Gold data can be materialized to Delta tables and queried by the Warehouse’s SQL endpoint without copying. Best when: You have unstructured or semi-structured data, or need heavy data engineering in Python or Scala. Why it’s nice: The right engine for each layer — Spark for heavy prep, T-SQL for serving — with no data copying, since OneLake underpins both Which should you pick? Here’s the rule I’d give a customer: Rule of thumb: For most SQL-focused analytics teams, an all-in-one Fabric Data Warehouse is the pattern I’d recommend starting with. If you have a mix of unstructured data or need heavy data engineering, land and refine raw data in a Lakehouse, then serve the final Gold layer from the Warehouse. Fabric’s architecture lets you evolve, so you’re not locked in: start all-in-warehouse and add a Lakehouse later when a new source needs Spark, or vice versa. All-in-One DW Lakehouse + DW Hybrid Primary skill T-SQL Spark / Python + T-SQL Best for data Structured / relational Unstructured, semi-structured, streaming Bronze lives in Warehouse staging tables Lakehouse (raw files, delta tables or both) Gold lives in Warehouse Warehouse Pick it for Simplicity, one engine Flexibility, heavy engineering One layout rule, whichever pattern you choose Keep layer separation clear. Microsoft recommends isolating layers into different workspaces, or at least different Fabric items, for better control and governance. The payoff is real: separate items give you finer security control — only data engineers touch Bronze while analysts see only Gold — and clearer isolation, so an accidental change in Gold can’t affect Bronze. Takeaway Don’t frame this as “Warehouse vs. Lakehouse.” Frame it as “How much Spark do I need?” If your workloads are structured and SQL-first, choose an all-in-one DW. If they’re unstructured or engineering-heavy, choose a hybrid approach, with Gold always served from the Warehouse. As your implementation grows, consider separating Bronze, Silver, and Gold into distinct Fabric items or workspaces to simplify governance and security. Ready to go deeper? Explore the Microsoft Fabric documentation for Data Warehouse and OneLake, then stay tuned for Part 2 of this series, where we'll walk through howBronze, Silver, and Gold layers are implemented in practice. This post is part of our Medallion Architecture on Fabric Data Warehouse series: Choosing your medallion pattern in Fabric Data Warehouse Building the Bronze → Silver → Gold layers Fabric DW best practices for medallion architectures Securing and governing your layers Performance tuning your medallion pipeline In the next post, we'll explore what Bronze, Silver, and Gold layers actually look like in Fabric Data Warehouse and how data moves between them.
Event-Driven Copy Job Execution with Fabric Activator (Generally Available)
Introduction Modern data platforms are increasingly moving toward event-driven architectures where data flows react to events or changes instead of running on rigid schedules. In Microsoft Fabric, two powerful capabilities enable this vision: Copy job: the simplest and most scalable way to move data across multiple clouds and tenants. Activator: a real-time engine that monitors data and reacts to events. A key enhancement brings these two capabilities closer together: you can now invoke Copy jobs directly from Activator—no pipeline required. This unlocks a simpler, faster, and more efficient way to build event-driven data movement in Fabric. Why this matters The limitations of scheduled data movement Traditionally, data movement in enterprise systems relies on fixed schedules: Run every five minutes Run every hour Run daily While simple to set up, this approach introduces several challenges: Inefficient execution (empty runs): Most scheduled jobs run even when no data has changed, leading to unnecessary computing usage and cost. Latency tradeoff: Running more frequently increases cost, while running less frequently increases data latency. Operational overhead: Managing schedules across hundreds or thousands of tables can quickly become complex and difficult to maintain. Activator + Copy Job: A natural fit What customers need is simple: Move data only when something happens or changes. For example, when a table in OneLake is updated or when a new file lands in storage. This is exactly the scenario where Data Activator shines. Activating data movement based on events is a natural combination of these two capabilities: Activator monitors data and detects events Copy job moves data efficiently across systems With this new capability, you can now connect them directly, enabling true event-driven data movement without introducing additional orchestration layers. Quick tutorial: Set up Activator to invoke a Copy job Step 1: Create a Copy job Define your source and destination Choose full or incremental copy Configure mappings and settings Step 2: Create an Activator rule Select a data source (e.g., OneLake, Eventstream, monitoring data) Define a condition: File created Table updated Metric threshold reached Step 3: Configure the action Choose “Invoke Copy job” Select the target Copy job Step 4: Start the rule Once enabled: Activator continuously monitors the defined condition When the condition is met → the Copy job runs automatically Data is moved only when needed, with no unnecessary runs. Learn More What is Copy job How to create a Copy job How to monitor a Copy job Submit feedback on Fabric Ideas and join the conversation in the Fabric Community. Questions or feedback? Leave your thoughts in the comment section.Simplify your data movement with Copy job: CDC with SQL estate (Generally Available)
Author: Ye Xu, Principal Program Manager - Copy job is the go-to solution in Microsoft Fabric Data Factory for simplified data movement across multiple clouds and tenants. With native support for bulk copy, incremental copy, and change data capture (CDC) replication, it can handle a wide range of movement scenarios through an intuitive, easy-to-use experience.Simplify data movement with Copy job: more control, more flexibility
Author: Ye Xu, Principal Program Manager - Copy job is the go-to solution in Microsoft Fabric Data Factory for simplified data movement across multiple clouds and tenants. With native support for bulk copy, incremental copy, and change data capture (CDC) replication, it can handle a wide range of movement scenarios through an intuitive, easy-to-use experience.
Outbound access protection for Data Factory (Generally Available)
Co-author: Abhishek Narain Workspace outbound access protection (OAP) is widely accessible for Data Factory workloads—including Pipelines, Copy Job, and Dataflows—as well as for Mirrored Databases such as Mirrored SQL Database and Mirrored Snowflake. Key benefits Enhanced outbound security: By leveraging OAP rules, organizations can ensure that the Data Factory items from the protected workspace can only connect to trusted endpoints allowed by workspace admins. All the other outbound connections to public internet and other destinations are blocked from the workspace. Granular control: Control outbound access per workspace. This allows you to apply differentiated controls across business units, environments (dev/test/prod), data domains, or project. Data exfiltration prevention: Workspace OAP when combined with Inbound protection can help the customer prevent the data from exfiltrated outside the workspace boundary. Better compliance: Meet stringent compliance and regulatory requirements by ensuring your sensitive data never leaves the workspace boundary if it’s not allowed by Workspace Admins. Additional functionalities Exploratory APIs in pipelines and Copy Job: APIs used for Browse, Preview, and Test Connection operations will support outbound access protection. Copy job in OAP will support Datawarehouse as destination. Workspace level granularity for Notebook and Spark Job Definitions connection types. Machine Learning Models and Experiments are supported with outbound access protection. Data Agent and Eventstreams are now supported with outbound access protection in preview. To learn more about Workspace OAP for Data Factory, set-up scope and limitations, refer to the Workspace outbound access protection overview documentation and Workspace outbound access protection for Data Factory documentation. What’s next? We are actively working to expand OAP support for additional experiences and plan to add support for Power BI Semantic Models and Reports soon. Your feedback is essential! Let us know how we can make Fabric even more secure and flexible for your workloads by sharing your feedback at Fabric Ideas – Microsoft Fabric Community.
