Inconsistent Behavior When Using Lakehouse Tables in Warehouse-Scoped Queries in Copy Activity
Hi Fabric Team,
I’m writing to report a bug we've encountered while using Microsoft Fabric. we rely on Fabric for our ETL and ELT processes, leveraging both Warehouse and Lakehouse features.
As you know, when setting up a copy activity, we are typically allowed to use a query only on Warehouse, not on Lakehouse tables. However, we discovered that Fabric currently allows users to reference Lakehouse tables in a Warehouse-scoped query, which results in subtle and hard-to-detect issues.
This behavior caused a critical problem in one of our production pipelines: records from yesterday’s load were not copied from Fabric to Oracle. After thorough investigation, we found that the issue stemmed from a query referencing a Lakehouse table, even though the copy activity was configured to use a Warehouse. Fabric did not validate this inconsistency, and as a result, the copy activity silently failed to move the expected data.
This bug could potentially have significant consequences in production environments. I recommend implementing a validation step to ensure that all objects used in a query are consistent with the selected source type (Warehouse vs. Lakehouse), to avoid such hidden issues in the future.
Kind regards,
Yazdan
Data Engineer
Recent ideas
Make Workspace-Level Surge Protection Limit Visible to Workspace Users
Please make the applicable workspace-level surge protection limit visible in the workspace settings. When surge protection is applied by Capacity Admins, users currently don't have visibility int...frithjof_v20 seconds agoCommunity ChampionNew0Views0likes0CommentsAllow Custom Recipient Settings for Gateway Expiry Notifications
Now, during the final 90 days before the gateway version end-of-support date, operations continue, but semantic model contacts users can receive warnings in email The warning message is: "The on-prem...YuanMao49 minutes agoNew MemberNew2Views0likes0CommentsHow to Back Up and Restore a Fabric Lakehouse for Debugging
Is there a way to clone a Lakehouse, including its data, in Microsoft Fabric? When we encounter a data issue in Production, our usual approach is to take a backup of the database, restore it to a se...gsrinivasan5 hours agoNew MemberNew39Views0likes1CommentTenant Administration for Fabric Cloud Connections - Enterprise Governance Gap
Microsoft Fabric currently provides robust security around cloud connections by treating them as user-owned securable resources. While this model works well for personal connections, it creates signi...hrenollet7 hours agoRegular VisitorNew77Views7likes1Comment