Forum Discussion

alfBI's avatar
alfBI
Responsive Resident
3 years ago

Best Strategy with pipelines including datasets & dataflows

Dear colleagues,

 

In an scenario as the one depicted below what would be the best strategy to ensure the references flagged in yellow (Workspaces/Dataflows references on datasets) & (gateways on dataflows) are properly managed during the deployment using PBI pipelines (Datasets and workflows are on different Workspaces)

 

To manage the Datasets references to workflows we are using the approach described here, using workspaces names and parameter rules on the pipeline. Approach looks fine but it loosks like using it lineage provided by PBI does not longer works (dataflow linked to dataset does not appear). Do you know any other method (considering that Workflows and dataflows are on different workspaces as default) ?

 

With regards dataflow dependenciesw with the gateway. Any idea about to manage the change on the pipeline? There is paramete and source rules but I would need a sort of "gateway rule".

 

Thx for your ideas

 

Dataflow

2 Replies

  • v-yueyunzh-msft's avatar
    v-yueyunzh-msft
    Community Support
    3 years ago

    Hi , alfBI 

    According to your descrioption, you update the M code to connect the dataflows by using the WorkspacesName and DataflowName. And you want to use the piepline from test to Pro. For the "Approach looks fine but it loosks like using it lineage provided by PBI does not longer works (dataflow linked to dataset does not appear)". Sorry for i may not surely understand what it means in your situation.

    For your last question, regarding dataflow dependencies with the gateway , you can use parameter and data source rules, but there is no "gateway rule" available now .

    For more information, you can refer to this document:
    Create deployment rules Power BI Application lifecycle management (ALM) - Power BI | Microsoft Learn

     

    Thank you for your time and sharing, and thank you for your support and understanding of PowerBI! 

     

    Best Regards,

    Aniya Zhang

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly

    • alfBI's avatar
      alfBI
      Responsive Resident
      3 years ago

      Hi Aniya,

       

      Thx for your answer.

       

      With regards the lineage issue I was referring to what you see on below test cases

       

      Test 2- Plain M query 

      Test 3 -M Query using Parameters to refers Workspace & Workflow ID

      Test 4 – M Query  referring to names .Using Parameters to refers Workspace & Workflow Names

       

       

       

      When displaying content lineage this is what you see on each case

       

      as you can see, dataflow does appear on case 2 & 3 (using IDs with or witohout parameters) but using Names the dependency with the dataflow is missed. 

       

      We are interesting on following pattern defined on test 4 as it allows to move/recreate dataflows/workspaces without having to be worried about ID dependencies across the system, but it's a pity not being able to see this dependency any more.

       

      Missing gateway rule is also painful for us because servers has same name on TST & PRD environments so we can´t parameters to switch the connection. Only chance we have now is to edirt manually dataflow connection and change the gateway.

       

      Regards,

      Regards,