Don't miss your chance to take the Fabric Data Engineer (DP-600) exam for FREE! Find out how by watching the DP-600 session on-demand now through April 28th.
Learn moreJoin the FabCon + SQLCon recap series. Up next: Power BI, Real-Time Intelligence, IQ and AI, and Data Factory take center stage. All sessions are available on-demand after the live show. Register now
I'm trying to create a Dataflow Gen2 that uses a variable library to remap the destination lakehouse between stages of a deployment pipeline. I followed these two guides:
https://learn.microsoft.com/en-us/fabric/cicd/variable-library/tutorial-variable-library
https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-gen2-variable-library-integration
I have created a variable library with entries for WorkspaceId and a LakehouseId.
I have two queries for WorkspaceId and LakehouseId in my Dataflow. These appear to be working. Just as a test, I added another query that writes these to a (hardcoded) lakehouse destination, and the data that appears when I run this is correct (the correct Workspace Id and Lakehouse Id).
However, when I try to use WorkspaceId and LakehouseId in the destination query of one of my actual data queries, I get an error "The key didn't match any rows in the table".
Here is the m script for the destination query that fails:
let
Pattern = Lakehouse.Contents([CreateNavigationProperties = false, EnableFolding = false, HierarchicalNavigation = true]),
Navigation_1 = Pattern{[workspaceId = WorkspaceId]}[Data],
Navigation_2 = Navigation_1{[lakehouseId = LakehouseId]}[Data],
Navigation_3 = Navigation_2{[Id = "Strong", ItemKind = "Schema"]}[Data],
TableNavigation = Navigation_3{[Id = "Destination", ItemKind = "Table"]}[Data]
in
TableNavigation<p>
(To clarify, that ID="Destination" is because the destination table name is also incidentally called "Destination" for unrelated reasons)
This was generated by using the destination lakehouse widget on the query, but then I replaced the hardcoded workspace Id and lakehouse Id with my variables. Other potentially relevant information - the destination lakehouse is schema-enabled. These queries don't use incremental refresh, although I have another Dataflow that I need to do this in that does use an incremental refresh.
Experience the highlights from FabCon & SQLCon, available live and on-demand starting April 14th.
If you have recently started exploring Fabric, we'd love to hear how it's going. Your feedback can help with product improvements.
| User | Count |
|---|---|
| 9 | |
| 6 | |
| 6 | |
| 5 | |
| 4 |
| User | Count |
|---|---|
| 30 | |
| 15 | |
| 11 | |
| 10 | |
| 6 |