Best approach(es) to refresh Dev Warehouse from Prod Warehouse (Read-Write Copy) in Microsoft Fabric
We are currently evaluating the best practices for refreshing a Development Data Warehouse using data from a Production Data Warehouse within the same Fabric tenant. We have evaluated a few approaches based on the use case: Read-Only Access (Zero-Copy): If Dev only needs read access to Prod data, using OneLake Shortcuts is the clear choice. It provides instantaneous access with zero data duplication, no storage overhead, and minimal setup effort. Read-Write Access (Full Schema & Data Copy): If Dev requires a Read-Write copy (allowing developers to run DML operations like UPDATE, INSERT, or DELETE without impacting Production), our current options seem to be: Fabric Data Pipelines (Copy Activity): Scheduled or trigger-based pipelines to pull data from Prod to Dev. Cross-Database Querying / CTAS: Running T-SQL (CREATE TABLE AS SELECT ...) across workspaces. The Challenge: Both Read-Write options (Pipelines and CTAS) can become cumbersome to set up and maintain, especially as datasets grow. Additionally, duplicating large volumes of data incurs extra storage and CU (Capacity Unit) processing costs. Was wondering if there are better, more efficient options available in Microsoft Fabric to maintain or periodically refresh a Read-Write copy of a Warehouse in a Dev workspace? Is Table Cloning / Zero-Copy Clone supported or planned for cross-workspace scenarios? How are others managing periodic Dev environment refreshes from Prod without creating heavy data copy pipelines?
Fabric pipeline Deployment dev to prod
Hi there, I have been developing fabric data pipelines, warehouse, lakehouse in a fabric dev workspace. I noticed when deployment is done from WS_Dev to WS_Prod, pipelines are still pointing to Dev workspace. I can parameterize it. Where in Prod workspace do I need to configure that all objects should point to Prod workspace so code deployments from dev dont overwrite the variable in PROD. master pipeline with look up After lookup -- invoke pipeline child pipeline with copy activity.
