Forum Discussion

Raki39's avatar
Raki39
Helper I
2 years ago
Solved

composite model and deployment Pipeline meta data refresh issue

We encountered an unusual issue with the power deployment pipeline. Here's the situation:
 
We've developed a composite data model in Power BI, which is connected to an AAS (Azure Analysis Services) data model via a live connection. we converted it to a direct query local model.
 
After deploying this model to the Power BI service and using it to create reports, we found a drawback. Whenever a new table or column is added to the AAS data model, we must download the Power BI data model file from the service, refresh it in the desktop app to reflect those changes, and then republish it (client is ok for this porcess) .
 
We're utilizing the deployment pipeline feature to move the reports to subsequent workspaces (such as QA and prod).
 
The specific issue can be illustrated with an example: In the DEV environment, we have four tables (A, B, C, D) in the AAS model, and the Power BI local data model correctly displays these tables. However, when we moved this Power BI data model to the QA environment using the deployment pipeline (with respective rules configured), only tables A, B, and C were available in the QA AAS data model. Surprisingly, we could still see table D along with A, B, and C in the QA-deployed Power BI data model.
 I hope this explanation provides a clearer understanding of the situation.

requesting the experts help with this situation. LeeBenjamin, Jeroen ter Heerdt is there any solution for this?
Need Help
  • LeeBenjamin's avatar
    LeeBenjamin
    2 years ago

    Raki39 This works by design - The Model's schema in QA is similar to that in Dev, and it still uses the PRODUCT table (even though the data source doesn't have it). Deploy does not erase existing data, so either the new data source have all the tables the model is using, or add a parameter (again controlled by a rule) that somehow in the model prevent using that table.

6 Replies

  • Raki39's avatar
    Raki39
    Helper I
    2 years ago

    LeeBenjamin 
    Sorry for the late reply. 
    We have received the same response from the microsoft support team. 

    It’s a by design behaviour of deployment pipeline for Composite model and we don’t have any workaround other than that to have same data  in all environments.

    Thanks,
    Raki

     

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi Raki39 ,

     

    You can create gen2 in data factory and link it to AAS, then create pipeline in data factory, and pipeline will select the dataflow you just created.

     

    If the pipeline is connected to dataflow, it will reflect the data changes in dataflow.

     

    Dataflow is a thing similar to PowerQuery, it basically has what PowerQuery has.

     

    For more information about Data Factory and Dataflow, please refer to :

    What is Data Factory - Microsoft Fabric | Microsoft Learn

    Creating a dataflow - Power BI | Microsoft Learn

     

    Please refer to the documentation below for specific steps on how to do the above:

    Copy Data from Azure Analysis Service (AAS) through Microsoft Fabric (c-sharpcorner.com)

     

    If you have any other questions please feel free to contact me.

     

    Best Regards,
    Yang
    Community Support Team

     

    If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
    If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

    • Raki39's avatar
      Raki39
      Helper I
      2 years ago

      Anonymous  Thank you some much for the response.
      I apologize for any confusion in my previous explaination. To ensure clarity, I've provided addational detail below with an attachment.

      "In our setup, we have Product and Sales tables in the AAS dev data source, while only the Sales table exists in the AAS QA and AAS prod data sources. Our Power BI semantic data model is connected to the AAS dev source in the Dev workspace. After deploying the semantic model to the QA and prod workspaces using a deployment pipeline, we noticed that the Product table appears in the deployed QA and prod semantic data models, even though it's not present in the respective AAS data sources. Despite refreshing the metadata, the Product table continues to appear. We're unable to identify the root cause of this issue.
      Could you please review it and offeer any possible solution or insight
      Regards,
      Raki

      • LeeBenjamin's avatar
        LeeBenjamin
        Microsoft Employee
        2 years ago

        Hi Raki39 
        If I understand right the scenario, then this is by design.

        Deployment pipelines pairs two items across neighbour stages so then on future deployments, it knows which item to override on the target stage. The pairing process is done based on the combination of item type+name while in case of conflict, the item's full path (workspace folders) will be considered as well.
        With that, it seems like your semantic-model items on Dev and QA are paired, and same is for those on QA and Prod, since they have the same name and item type. Once paired, the item on the target stage will be overwritten, regardless if it has the same tables or not as its source. Fully compare of 2 paired items is only done as part of Deployment pipelines' compare process which in the end presents indidcation of the target item vs. its source item.
        Best regards,

        Lee