Inconsistent Behavior When Using Lakehouse Tables in Warehouse-Scoped Queries in Copy Activity
Hi Fabric Team,
I’m writing to report a bug we've encountered while using Microsoft Fabric. we rely on Fabric for our ETL and ELT processes, leveraging both Warehouse and Lakehouse features.
As you know, when setting up a copy activity, we are typically allowed to use a query only on Warehouse, not on Lakehouse tables. However, we discovered that Fabric currently allows users to reference Lakehouse tables in a Warehouse-scoped query, which results in subtle and hard-to-detect issues.
This behavior caused a critical problem in one of our production pipelines: records from yesterday’s load were not copied from Fabric to Oracle. After thorough investigation, we found that the issue stemmed from a query referencing a Lakehouse table, even though the copy activity was configured to use a Warehouse. Fabric did not validate this inconsistency, and as a result, the copy activity silently failed to move the expected data.
This bug could potentially have significant consequences in production environments. I recommend implementing a validation step to ensure that all objects used in a query are consistent with the selected source type (Warehouse vs. Lakehouse), to avoid such hidden issues in the future.
Kind regards,
Yazdan
Data Engineer
Recent ideas
Allow Workspace Admins to View Power BI Semantic Model Configuration Without Requiring Take Over
a { text-decoration: none; color: #464feb; } tr th, tr td { border: 1px solid #e6e6e6; } tr th { background-color: #f5f5f5; } Description As a Power BI Workspace Administrator, I understand why own...LeonardGreene7 minutes agoNew MemberNew1View0likes0CommentsOpen Fabric notebooks in View‑only by default
Problem: Developers and reviewers with edit permissions frequently open notebooks for reading. Notebooks currently open in edit mode with autosave, which causes accidental changes and noisy version h...AMD079129 minutes agoAdvocate IVNew331Views11likes1CommentService principal authentication support for data sources in Paginated(RDL) reports
We are only seeing two options(Basic, OAuth2) for data source authentication in a paginated report, Recently we see that the Service principal authentication enabled for Power BI(PBIX) reports, simi...Thatikonda_Vams36 minutes agoNew MemberNew133Views6likes2CommentsMetadata-Driven Copy Job Control at Table and Column Level
We currently achieve metadata-driven table and column selection in Copy Data activities by using dynamic queries. This allows us to dynamically determine which tables and columns should be processed ...Motasim095 hours agoNew MemberNew6Views0likes0CommentsDynamic ForEach Parallel Batch Count Based on Runtime Capacity
We would like Microsoft Fabric Data Factory to support dynamic configuration of the ForEach activity's parallel batch count at runtime. Currently, the ForEach batchCount is effectively configured as...Motasim095 hours agoNew MemberNew4Views0likes0Comments