surpriseturtle's avatar
surpriseturtle
Frequent Visitor
7 months ago
Status:
Abandoned

Fix sticky values in Dataflow Data Destination UI

When I modify the data desination table name in a Git repository and pull the changes into Fabric, running the dataflow from Fabric results in the old table name being used. It's only possible to change the desination table name through the Fabric UI, not the Git code. 

5 Comments

  • frithjof_v, I'll give it a try next time. Interestingly, I've never had an issue updating the part of the data destination code that uses a workspace ID and lakehouse ID. I update these lines to refer to library variables using Variable.ValueOrDefault, and have confirmed they are working correctly in different workspaces without having to publish the dataflow after the sync.
  • miguel's avatar
    miguel
    Icon for Community Admin rankCommunity Admin
    Status changed:
    New
    to
    Need Clarification

    Would you mind sharing how you're changing the table name through Git? What specific part of the mashup.pq document are you modifying?

     

    Sharing any repro steps will help us get to the bottom of it.

  • For example, like this: shared DimTABCRoster_DataDestination = let Pattern = Lakehouse.Contents([CreateNavigationProperties = false, EnableFolding = false]), Navigation_1 = Pattern{[workspaceId = workspace_id]}[Data], Navigation_2 = Navigation_1{[lakehouseId = renamed_lakehouse_id]}[Data], TableNavigation = Navigation_2{[Id = "DimTABCRoster", ItemKind = "Table"]}?[Data]? in TableNavigation; I was editing the Table Navigation step in the Git repository, where it filters to the table as "DimTABCRoster". Now that I take a closer look at it, it seems this step is referencing an existing table rather than creating a table. I would personally like for this step to create a new table if none exists with that name. It is more similar to the UI experience where a new table is created the first time you run the dataflow with a newly named destination.
  • miguel's avatar
    miguel
    Icon for Community Admin rankCommunity Admin
    Status changed:
    Need Clarification
    to
    Abandoned

    It is possible to change the table name in Git. However, we woudl recommend making the changes through the new Advanced Editor for the data destination query so you don't have to manipulate the whoel document manually or require Git:

    Advanced edit for data destination queries in Dataflow Gen2 - Microsoft Fabric | Microsoft Learn

     

    In the TableNavigation step we actually use the question marks (?) as a way to throw a null if the table isn't found. Depending on how the rest of your M Script is set, it may actually create a new table or not.

     

    The article below goes a bit more in-depth about the Settings that the data destination query could have and how such settings are set as an attribute record to the query that originates the data:

    Dataflow Gen2 data destinations and managed settings - Microsoft Fabric | Microsoft Learn

     

    For more questions about the behavior of data destinations, we recommend posting the questions in the Data Factory forum.

Recent ideas