Forum Discussion
Difference between Semantic Model refresh via Pipeline activity vs manual refresh in Fabric UI?
- 7 months ago
Both the pipeline “Semantic model refresh” activity and the Fabric UI “Refresh” send a refresh request to the same semantic model engine. In normal cases, a refresh will not “fix” a model whose queries, relationships, or calculated objects reference columns that are missing in the target source. A refresh loads data; it does not redesign the model.
The behavior may differ in practice depending on specific circumstances.
-
Identity and credentials used
The pipeline refresh occurs through the Data Factory connection that you select in the activity settings. The UI refresh runs as an on-demand refresh inside the Power BI service context for the model. Different identities often mean different gateways, different permissions, different default databases, or different effective connections, even if you think both point to the same warehouse. -
Refresh scope and internal bindings
The pipeline activity supports selecting specific tables and partitions, and you can parameterize those selections. If you configured table or partition scope before cloning, those bindings can become stale after cloning or repointing. The activity defaults to a full refresh, but scope overrides can change what gets processed. -
Refresh mode differences
Power BI refreshes can be full or partial depending on incremental refresh policies. Advanced paths exist via XMLA to refresh partitions.
If your model uses incremental refresh, a “partial” style execution can surface different failures than a “full process everything” run, especially when schema drift exists.
Here is why your UI refresh might succeed even when the pipeline fails.
What I see from above is that opening the semantic model in the model view makes the metadata re-evaluate or reconnect before you click Refresh.The model uses the new source mapping, and then it works to refresh.
The pipeline activity skips any interactive “open model” step. It fires refresh immediately, using whatever metadata and bindings exist at that moment. If those still reference an older schema, you get “column not found.”I am sharing some options to replicate the UI outcome from a pipeline.
Option A. Remove schema drift as a design constraint
If tenant schemas differ, bake tolerance into Power Query so missing columns do not fail to refresh.
Example patterns:-
Use the table. SelectColumns with MissingField. Use null for optional columns
-
Add columns conditionally with a try; otherwise, null.
-
Standardize views in the warehouse per tenant, so the semantic model always sees a stable contract
This is the only approach that stays reliable at scale.
Option B. Use REST API “enhanced refresh” or XMLA for more control
If you need to orchestrate a refresh more predictably than the built-in activity, call the Power BI REST API enhanced refresh from a pipeline web activity. Enhanced refresh gives more control for large models and advanced scenarios.
If you are on capacity and XMLA is enabled, XMLA processing also gives you deterministic control over what gets processed.Option C. Add an explicit “metadata sync” step
Fabric does not currently expose a clean, documented “open model, then refresh” equivalent as a first-class pipeline activity. So if your success depends on that interactive metadata update, treat it as a missing automation primitive and avoid relying on it.
In practice, teams replace this with Option A or with an API-driven sequence where they rebind or update model definitions as part of the deployment, then refresh.
Thank you!
Proud to be a Super User!
📩 Need more help?
✔️ Don’t forget to Accept as Solution if this guidance worked for you.
💛 Your Like motivates me to keep helping -
- 6 months ago
Hi jbauti10,
Thank you for the response and testing the earlier suggestions.
Based on the behaviour described, this does not look like a refresh mode issue. Looks like the failure is caused by the schema differences between the original source and the cloned tenant data sources.- When you open the semantic model in the UI, the fabric revalidates metadata before the refresh. The pipeline activity does not perform that reconciliation step, it directly processes using the existing model bindings.
- For multitenant automation, the supported approach is to ensure all tenant sources expose an identical schema, for example by standardizing tables or using a consistent sql view layer. Once the schema contract is stable, both UI and pipeline refresh will behave the same.
If schemas differ across tenants, automated refresh will continue to fail intermittently.
Thanks and regards,
Anjan Kumar Chippa
- 6 months ago
Hi jbauti10,
Yes, updating the model via XMLA means deploying the changes to the table and column definitions using TMSL through supported tools.
- You should not manually generate lineageTag values. These are system managed and are handled automatically when the model is deployed through supported tools. Manually creating GUIDs is not the recommended approach.
- If you are modifying tables or columns, the recommended way is to deploy the updated complete and consistent object definition rather than patching individual properties. The required metadata will be included automatically as part of the proper deployment.
So in short the intended pattern is tool driven model deployment, not as manual partial metadata updates.
Thanks and regards,
Anjan Kumar Chippa
Hi jbauti10,
Thank you for the response and testing the earlier suggestions.
Based on the behaviour described, this does not look like a refresh mode issue. Looks like the failure is caused by the schema differences between the original source and the cloned tenant data sources.
- When you open the semantic model in the UI, the fabric revalidates metadata before the refresh. The pipeline activity does not perform that reconciliation step, it directly processes using the existing model bindings.
- For multitenant automation, the supported approach is to ensure all tenant sources expose an identical schema, for example by standardizing tables or using a consistent sql view layer. Once the schema contract is stable, both UI and pipeline refresh will behave the same.
If schemas differ across tenants, automated refresh will continue to fail intermittently.
Thanks and regards,
Anjan Kumar Chippa
- jbauti106 months ago
Advocate I
Hello v-achippa,
Thanks for the clarification. That explains the difference between the UI behavior and the pipeline activity.
However, this creates a serious limitation for automation scenarios. Since the UI performs metadata revalidation implicitly, but there is no API or pipeline equivalent, multitenant provisioning at scale becomes extremely difficult.
In ISV scenarios where workspaces and semantic models are cloned and rebound programmatically, we need a way to:
Explicitly trigger metadata revalidation
Push schema changes to a semantic model
Ensure model bindings are reconciled before refresh
Without a programmatic way to perform this reconciliation step, any schema drift across tenants breaks automation and forces manual intervention through the UI, which defeats the purpose of scalable provisioning.
Is there a roadmap item to expose metadata revalidation as an API endpoint?
This capability is essential for serious multitenant automation on Fabric.
Thanks.
- v-achippa6 months ago
Community Support
Hi jbauti10,
Yes you are correct that currently there is no API or pipeline activity that explicitly triggers the same metadata validation that happens when opening the semantic model in the UI.
Refresh only processes data, it does not update or reconcile the model definition.
So in automation scenarios, if the model is rebound to a different source, the supported approach is to redeploy or update the model definition through XMLA after rebinding and before refresh. Redeploying the model make sure that the metadata is synchronized in a controlled way.
If this capability is important for your scenario, I request you to please submit a detailed feedback through the Fabric Ideas portal so the product team can evaluate it.
Fabric Ideas - Microsoft Fabric Community
Thanks and regards,
Anjan Kumar Chippa
- jbauti106 months ago
Advocate I
Hello v-achippa ,
Thank you for the clarification.
When you mention updating the model definition through XMLA, are you specifically referring to deploying updated TMSL that modifies the table and column definitions of the semantic model?
If so, I have a couple of questions:
For newly introduced columns, how should the lineageTag be generated? Is it expected that we generate a GUID ourselves, or is there a recommended approach to ensure consistency?
When updating the model this way, should all related column properties also be explicitly defined in TMSL such as dataType, summarizeBy, sourceColumn, etc., to ensure full metadata synchronization?
I want to make sure we are following the intended pattern before implementing this at scale.
Thanks again for your help.
Juan