Forum Discussion
Difference between Semantic Model refresh via Pipeline activity vs manual refresh in Fabric UI?
Hello,
I’m working on automating workspace cloning in Microsoft Fabric to support a multitenant platform.
I currently have scripts that:
Clone and create Fabric items (notebooks, pipelines, copy jobs, lakehouses, warehouses, semantic models, etc.)
Update connections, parameters, and references so everything points to the data sources in the cloned workspace
Most of this works as expected, but I’m running into an issue with refreshing a semantic model from a pipeline.
Scenario
I have a master_pipeline that triggers several child pipelines
The pipeline ends with a Semantic model refresh activity
After cloning the workspace and updating connections, this refresh activity fails
The error indicates that the semantic model cannot find a column. This is expected, since the cloned workspace points to a different data source where some columns differ.
However:
If I open the semantic model in the cloned workspace
Click Open semantic model (model view with tables and relationships)
Then click Refresh from the UI
The refresh succeeds and correctly loads data from the new warehouse.
Question
What is the functional difference between:
Refreshing a semantic model via a pipeline refresh activity
Refreshing it manually from the semantic model UI
More specifically:
Is there a way to replicate the manual semantic model refresh behavior from within a pipeline?
I’m trying to understand why the manual refresh succeeds while the pipeline refresh fails, even though both target the same semantic model and data source.
Any insight or pointers would be appreciated. Thanks!
Both the pipeline “Semantic model refresh” activity and the Fabric UI “Refresh” send a refresh request to the same semantic model engine. In normal cases, a refresh will not “fix” a model whose queries, relationships, or calculated objects reference columns that are missing in the target source. A refresh loads data; it does not redesign the model.
The behavior may differ in practice depending on specific circumstances.
-
Identity and credentials used
The pipeline refresh occurs through the Data Factory connection that you select in the activity settings. The UI refresh runs as an on-demand refresh inside the Power BI service context for the model. Different identities often mean different gateways, different permissions, different default databases, or different effective connections, even if you think both point to the same warehouse. -
Refresh scope and internal bindings
The pipeline activity supports selecting specific tables and partitions, and you can parameterize those selections. If you configured table or partition scope before cloning, those bindings can become stale after cloning or repointing. The activity defaults to a full refresh, but scope overrides can change what gets processed. -
Refresh mode differences
Power BI refreshes can be full or partial depending on incremental refresh policies. Advanced paths exist via XMLA to refresh partitions.
If your model uses incremental refresh, a “partial” style execution can surface different failures than a “full process everything” run, especially when schema drift exists.
Here is why your UI refresh might succeed even when the pipeline fails.
What I see from above is that opening the semantic model in the model view makes the metadata re-evaluate or reconnect before you click Refresh.The model uses the new source mapping, and then it works to refresh.
The pipeline activity skips any interactive “open model” step. It fires refresh immediately, using whatever metadata and bindings exist at that moment. If those still reference an older schema, you get “column not found.”I am sharing some options to replicate the UI outcome from a pipeline.
Option A. Remove schema drift as a design constraint
If tenant schemas differ, bake tolerance into Power Query so missing columns do not fail to refresh.
Example patterns:-
Use the table. SelectColumns with MissingField. Use null for optional columns
-
Add columns conditionally with a try; otherwise, null.
-
Standardize views in the warehouse per tenant, so the semantic model always sees a stable contract
This is the only approach that stays reliable at scale.
Option B. Use REST API “enhanced refresh” or XMLA for more control
If you need to orchestrate a refresh more predictably than the built-in activity, call the Power BI REST API enhanced refresh from a pipeline web activity. Enhanced refresh gives more control for large models and advanced scenarios.
If you are on capacity and XMLA is enabled, XMLA processing also gives you deterministic control over what gets processed.Option C. Add an explicit “metadata sync” step
Fabric does not currently expose a clean, documented “open model, then refresh” equivalent as a first-class pipeline activity. So if your success depends on that interactive metadata update, treat it as a missing automation primitive and avoid relying on it.
In practice, teams replace this with Option A or with an API-driven sequence where they rebind or update model definitions as part of the deployment, then refresh.
Thank you!
Proud to be a Super User!
📩 Need more help?
✔️ Don’t forget to Accept as Solution if this guidance worked for you.
💛 Your Like motivates me to keep helping-
Hi jbauti10,
Thank you for the response and testing the earlier suggestions.
Based on the behaviour described, this does not look like a refresh mode issue. Looks like the failure is caused by the schema differences between the original source and the cloned tenant data sources.- When you open the semantic model in the UI, the fabric revalidates metadata before the refresh. The pipeline activity does not perform that reconciliation step, it directly processes using the existing model bindings.
- For multitenant automation, the supported approach is to ensure all tenant sources expose an identical schema, for example by standardizing tables or using a consistent sql view layer. Once the schema contract is stable, both UI and pipeline refresh will behave the same.
If schemas differ across tenants, automated refresh will continue to fail intermittently.
Thanks and regards,
Anjan Kumar Chippa
Hi jbauti10,
Yes, updating the model via XMLA means deploying the changes to the table and column definitions using TMSL through supported tools.
- You should not manually generate lineageTag values. These are system managed and are handled automatically when the model is deployed through supported tools. Manually creating GUIDs is not the recommended approach.
- If you are modifying tables or columns, the recommended way is to deploy the updated complete and consistent object definition rather than patching individual properties. The required metadata will be included automatically as part of the proper deployment.
So in short the intended pattern is tool driven model deployment, not as manual partial metadata updates.
Thanks and regards,
Anjan Kumar Chippa
12 Replies
- MJParikh
Super User
Both the pipeline “Semantic model refresh” activity and the Fabric UI “Refresh” send a refresh request to the same semantic model engine. In normal cases, a refresh will not “fix” a model whose queries, relationships, or calculated objects reference columns that are missing in the target source. A refresh loads data; it does not redesign the model.
The behavior may differ in practice depending on specific circumstances.
-
Identity and credentials used
The pipeline refresh occurs through the Data Factory connection that you select in the activity settings. The UI refresh runs as an on-demand refresh inside the Power BI service context for the model. Different identities often mean different gateways, different permissions, different default databases, or different effective connections, even if you think both point to the same warehouse. -
Refresh scope and internal bindings
The pipeline activity supports selecting specific tables and partitions, and you can parameterize those selections. If you configured table or partition scope before cloning, those bindings can become stale after cloning or repointing. The activity defaults to a full refresh, but scope overrides can change what gets processed. -
Refresh mode differences
Power BI refreshes can be full or partial depending on incremental refresh policies. Advanced paths exist via XMLA to refresh partitions.
If your model uses incremental refresh, a “partial” style execution can surface different failures than a “full process everything” run, especially when schema drift exists.
Here is why your UI refresh might succeed even when the pipeline fails.
What I see from above is that opening the semantic model in the model view makes the metadata re-evaluate or reconnect before you click Refresh.The model uses the new source mapping, and then it works to refresh.
The pipeline activity skips any interactive “open model” step. It fires refresh immediately, using whatever metadata and bindings exist at that moment. If those still reference an older schema, you get “column not found.”I am sharing some options to replicate the UI outcome from a pipeline.
Option A. Remove schema drift as a design constraint
If tenant schemas differ, bake tolerance into Power Query so missing columns do not fail to refresh.
Example patterns:-
Use the table. SelectColumns with MissingField. Use null for optional columns
-
Add columns conditionally with a try; otherwise, null.
-
Standardize views in the warehouse per tenant, so the semantic model always sees a stable contract
This is the only approach that stays reliable at scale.
Option B. Use REST API “enhanced refresh” or XMLA for more control
If you need to orchestrate a refresh more predictably than the built-in activity, call the Power BI REST API enhanced refresh from a pipeline web activity. Enhanced refresh gives more control for large models and advanced scenarios.
If you are on capacity and XMLA is enabled, XMLA processing also gives you deterministic control over what gets processed.Option C. Add an explicit “metadata sync” step
Fabric does not currently expose a clean, documented “open model, then refresh” equivalent as a first-class pipeline activity. So if your success depends on that interactive metadata update, treat it as a missing automation primitive and avoid relying on it.
In practice, teams replace this with Option A or with an API-driven sequence where they rebind or update model definitions as part of the deployment, then refresh.
Thank you!
Proud to be a Super User!
📩 Need more help?
✔️ Don’t forget to Accept as Solution if this guidance worked for you.
💛 Your Like motivates me to keep helping -
- Thomaslleblanc
Super User
By editting the PowerQuery of the semantic model, that handles the refresh of the table changes. This does not happen in a refresh called from API, it just refreshes the M code and data.
This is by design.
If you were to just open the Power BI pbix file and click the Refresh in desktop (do not go to Transform), you would get the same resulting error.- jbauti10
Advocate I
Hello Thomaslleblanc,
Thank you for your response.
What do you mean by "editting the PowerQuery of the semantic model" Do you mean like adding a "refresh step" to the PowerQuery table logic?
Thanks for your help.
Best,Juan
- v-achippa
Community Support
Hi jbauti10,
Thank you for reaching out to Microsoft Fabric Community.
Thank you MJParikh and Thomaslleblanc for the prompt response.
As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided by the user's for the issue worked? or let us know if you need any further assistance.
Thanks and regards,
Anjan Kumar Chippa
- jbauti10
Advocate I
- v-achippa
Community Support
Hi jbauti10,
Thank you for the response and testing the earlier suggestions.
Based on the behaviour described, this does not look like a refresh mode issue. Looks like the failure is caused by the schema differences between the original source and the cloned tenant data sources.- When you open the semantic model in the UI, the fabric revalidates metadata before the refresh. The pipeline activity does not perform that reconciliation step, it directly processes using the existing model bindings.
- For multitenant automation, the supported approach is to ensure all tenant sources expose an identical schema, for example by standardizing tables or using a consistent sql view layer. Once the schema contract is stable, both UI and pipeline refresh will behave the same.
If schemas differ across tenants, automated refresh will continue to fail intermittently.
Thanks and regards,
Anjan Kumar Chippa