Forum Discussion
Row duplication after refresh in power bi service without using incremental refresh
Rows are being duplicated after refresh in the power bi service without using incremental refresh. The data source for the table is SQL. I have several different workspaces using this table and the row duplication does not happen across all workspaces, it seems to be random. I can refresh the model in power bi desktop and the row duplication does not exist. The table is not joined or merge with other tables in power query. What could be causing this to happen?
Hi h_waters ,
You can troubleshoot the issue by following some checks like-
1. If your SQL source uses views, temporary tables, or non-deterministic queries (e.g., with NEWID() or GETDATE() without filtering) — duplicates can appear randomly.Apply deterministic ORDER BY or enforce row uniqueness using DISTINCT or ROW_NUMBER() in SQL.
2. Power BI Desktop connects directly to SQL, while Power BI Service uses a gateway.
There might be:
Multiple gateways configured. Query retries or timeouts during refresh that result in partial results being returned multiple times.
Audit the gateway(s) used by each workspace.
Ensure all gateways are up-to-date and correctly pointed to production DBs.
Check SQL logs to see if queries are executed more than once per refresh.3.If you're using deployment pipelines or CI/CD, it's possible that old or duplicate versions of the same table were deployed and not cleared properly.
This may result in multiple copies of the same query/table existing under different internal IDs.
Rebuild the affected table completely:
Delete the table.
Save and close Power BI Desktop.
Reopen and recreate the table from scratch.
Ensure the model doesn’t have duplicate queries with similar names.
Also ensure that the model does not have many-to many relationships.Hope this helps!
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly..
Thank You.
10 Replies
v-sdhruvCommunity Support
Hi h_waters ,
You can troubleshoot the issue by following some checks like-
1. If your SQL source uses views, temporary tables, or non-deterministic queries (e.g., with NEWID() or GETDATE() without filtering) — duplicates can appear randomly.Apply deterministic ORDER BY or enforce row uniqueness using DISTINCT or ROW_NUMBER() in SQL.
2. Power BI Desktop connects directly to SQL, while Power BI Service uses a gateway.
There might be:
Multiple gateways configured. Query retries or timeouts during refresh that result in partial results being returned multiple times.
Audit the gateway(s) used by each workspace.
Ensure all gateways are up-to-date and correctly pointed to production DBs.
Check SQL logs to see if queries are executed more than once per refresh.3.If you're using deployment pipelines or CI/CD, it's possible that old or duplicate versions of the same table were deployed and not cleared properly.
This may result in multiple copies of the same query/table existing under different internal IDs.
Rebuild the affected table completely:
Delete the table.
Save and close Power BI Desktop.
Reopen and recreate the table from scratch.
Ensure the model doesn’t have duplicate queries with similar names.
Also ensure that the model does not have many-to many relationships.Hope this helps!
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly..
Thank You.I updated the gateway and this seems to have corrected the problem. I'm also working with the my team to create create unique row id for all tables. Thanks so much for this input!
- BhavinVyas3003
Super User
Please see if this thread useful to you,
I am not using incremental refresh. There must be something else causing the issue.
- BhavinVyas3003
ok since you mentioned in your question incremental thought of sharing.
- Madalina2801
Advocate II
Hi,
I'm facing the same issue with duplicate rows after refresh but only for one table in my model. I did a small change to a DAX formula in Desktop and it was ok. Then the issue appeared in Service after the refresh. I'm not using incremental refresh or gateway connection (the source is Synapse).
Do you have any idea what could cause this issue? What would be the solution?
Thank you!
- v-sdhruv
Hi Madalina2801 ,
A small DAX change in Desktop was fine, but duplication appeared after refresh in Service. This could be due to:
- Cached model states in Service not aligning with the updated DAX logic.
- Service-side refresh triggering a full reload without clearing previous cache.
After making DAX changes, republish the model and manually trigger a refresh. Also, consider clearing the dataset cache in Power BI Service if possible.
Hope this helps!- Madalina2801
Hi v-sdhruv,
Thanks for your reply!
This behavior is very strange. I triggered the on demand refresh and the duplicate rows dissapeared without changing/publishing/deploying the report. How can I clear the dataset cache in Power BI Service?
Is this a known issue? What could be the root cause for this random behavior and how can we prevent this from happening again in the future?
Thank you!
