frithjof_v's avatar
frithjof_v
Icon for Community Champion rankCommunity Champion
11 months ago
Status:
Completed

Automatically validate Dataflow Gen2 after deployment

Please fix this, so the Dataflow will get validated automatically after syncing changes from Git or using deployment pipelines:

 

"When you sync changes from GIT into the workspace or use deployment pipelines, you need to open the new or updated dataflow and save changes manually with the editor. This triggers a publish action in the background to allow the changes to be used during refresh of your dataflow. You can also use the on-demand Dataflow publish job API call to automate the publish operation."

 

https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-gen2-cicd-and-git-integration#limitations-and-known-issues

 

Because this currently doesn't happen automatically, it causes confusion, unnecessary manual work, and often we forget this and therefore get unexpected results when running the dataflow.

data factory
Data Factory | Dataflow

5 Comments

  • surpriseturtle's avatar
    surpriseturtle
    Frequent Visitor
    9 months ago
    Yes, this is sorely needed. The errors do not point to the need to publish either so a lot of users must have gone through this experience and struggled to figure out the root cause.
  • miguel's avatar
    miguel
    Icon for Community Admin rankCommunity Admin
    8 months ago
    Status changed:
    Planned
    to
    Completed

    After deploying a Dataflow, you can trigger a run / refresh of the Dataflow and it'll work without any interaction within the Dataflow editor.

  • AdeV-Oly's avatar
    AdeV-Oly
    Icon for Advocate I rankAdvocate I
    5 months ago

    With the recent change, triggering a Dataflow Gen2 refresh directly from the workspace after deployment is now possible, without first opening and saving the dataflow in the editor. This is a clear step forward compared to the previous experience.

    However, there are still some gaps that make the process feel only partially complete:

     

    First, the validation/publish step is not fully integrated into the deployment process itself. As a result, deployment alone does not leave the dataflow in a fully validated and consistent state.

     

    Second, this becomes more visible through an inconsistency in the UI. After deployment, if a refresh is manually triggered from the workspace:

    • … > Check validation correctly shows:

      • Validation Succeeded
      • An updated validation timestamp

    • But in contrast, the workspace list (Status column) continues to display an outdated validation state and timestamp.

    This creates a mismatch between the detailed validation view and the workspace overview, making it difficult to trust the Status column as a reliable indicator of the current validation state.

     

    miguel 
    Edited style

Recent ideas