Forum Discussion
Incremental Refresh Error
Hi Power BI community,
I want to apply incremental refreshes but I am having some problems. Firstly, I am getting this error message saying I have duplicate values and I have confirmed I do not have duplicate values in the coloumn it is saying I am. Can anyone explain why this might be the case. When I refresh from the desktop and publish it to Power BI online I have no problems but when my scheduale refresh triggers I get this error.
If anyone has ran into this problem or knows why its saying I have duplicate values in a column that is all disnict your help would be greatly apprecaited!
Thanks,
Adam
Hi Awollet33 ,
Try the solution from official document.
Cause: Post dates have changed
With a refresh operation, only data that has changed at the data source is refreshed in the dataset. As the data is divided by a date, it’s recommended post (transaction) dates are not changed.
If a date is changed accidentally, then two issues can occur: Users notice some totals changed in the historical data (that is not supposed to happen), or during a refresh an error is returned indicating a unique value is not in fact unique. For the latter, this can happen when the table with incremental refresh configured is used in a 1:N relationship with another table as the 1 side and should have unique values. When the data is changed (for a specific ID), that ID then appears in another partition and the engine will detect the value is not unique.
Solution: Refresh specific partitions
Where there is a business need to change some past data from the dates, a possible solution is to use SSMS to refresh all partitions from the point where the change is located up to the current refresh partition, thus keeping the 1 side of the relationship unique.
Best Regards,
Jay
5 Replies
Awollet33 do you have null/blank values in that column? The null value is not allowed, just double-check.
✨ Follow us on LinkedIn
Check my latest blog post The Power of Using Calculation Groups with Inactive Relationships (Part 1) (perytus.com) I would ❤ Kudos if my solution helped. 👉 If you can spend time posting the question, you can also make efforts to give Kudos to whoever helped to solve your problem. It is a token of appreciation!
⚡ Visit us at https://perytus.com, your one-stop-shop for Power BI-related projects/training/consultancy.⚡
Awollet33 make sure you are checking the full dataset from your back end source, if you have partial data in pq based on range start and end, you might not find it but once the report is published, the first time it is going to do a full load and there might be blank row which otherwise you are not seeing in PQ. You have to confirm it at the source.
Hi Awollet33 ,
Try the solution from official document.
Cause: Post dates have changed
With a refresh operation, only data that has changed at the data source is refreshed in the dataset. As the data is divided by a date, it’s recommended post (transaction) dates are not changed.
If a date is changed accidentally, then two issues can occur: Users notice some totals changed in the historical data (that is not supposed to happen), or during a refresh an error is returned indicating a unique value is not in fact unique. For the latter, this can happen when the table with incremental refresh configured is used in a 1:N relationship with another table as the 1 side and should have unique values. When the data is changed (for a specific ID), that ID then appears in another partition and the engine will detect the value is not unique.
Solution: Refresh specific partitions
Where there is a business need to change some past data from the dates, a possible solution is to use SSMS to refresh all partitions from the point where the change is located up to the current refresh partition, thus keeping the 1 side of the relationship unique.
Best Regards,
Jay
