DonHoog's avatar
DonHoog
Icon for Advocate I rankAdvocate I
6 months ago
Status:
Need Clarification

Deployment rules can be used on Dataflows Gen2 (CI/CD)

Hi,

 

We currently have a lot of dataflows gen1.

We use 3 environments in our deployment pipelines (DEV - TST - PRD).

Our company is investigating to upgrade our dataflows from gen1 to gen2.

 

However, we noticed we cannot use the deployment rules on dataflows gen2.

Can you please fix this, or at least tell us how we can use dataflows gen2 such that the parameters are automatically adjusted on the TST and PRD environment like we have now for dataflows of gen1.

We dont want to copy dataflows gen 2 to TST and PRD and manually adjust the parameters there, because it feels very stupid and we have a greater risk that somebody forgets and dataflows update with the wrong info from the wrong environment (or don't update at all) overnight.

2 Comments

  • What about DataFlow Gen2 which was migrated directly from Azure Data Factory to Fabric workspace ? I can not use Variable Library parameters as it was not visible to select. I think this Data Flow is a MDF transform. Do u know any how to solve this. It is grateful if you can help me for this problem.

     

  • miguel's avatar
    miguel
    Icon for Community Admin rankCommunity Admin
    Status changed:
    New
    to
    Need Clarification

    Hi!

    Our recommendation is to adopt either one of the approaches from the article below:

    CI/CD and ALM solution architectures for Dataflow Gen2 - Microsoft Fabric | Microsoft Learn

     

    The closest would be to use Fabric variable libraries which Dataflow Gen2 fully supports. 

     

    The benefits of this approach over the delployment rules is that from a Git perspective you are not affecting the actual mashup document. This means that the deployment is really just copying the same Dataflow from one environemnt to the next and the variables are the one passing the correct values for execution, but they simply remain as variables.

     

    The bad effect of using deployment rules was that each deployment would trigger 2 changes:

    1. Updating the content of the mashup document of the Dataflow with the one from the earlier environment

    2. Doing a sort of "find and replace" over the outcome of #1 

     

    This behavior would actually go against some principles of how ALM should be handled that the exact code that you tested in your earlier environment is just being pushed to a different environment because #2 fundamentally changes that code.

     

    Hope this helps!

Recent ideas