Forum Discussion

ramashish74's avatar
ramashish74
Frequent Visitor
4 years ago

Power BI Incremental refresh duplicate issues

Hi,

 

Greetings !!

 

I am facing duplicate row data issue in Power BI incremental dataset.

Need your assistance on the same.

 

Regards

Ramashish Gupta

 

Need Help

3 Replies

  • ibarrau's avatar
    ibarrau
    Super User
    4 years ago

    Hi. Do you have the "Detect data changes activated"? be sure that if you use that option, it is usually recommended to use a modified date or some way to identify the changed row from the new row with the change. This is incremental refresh, so it's inserting, it will not replace a row in the old history. If you data changes, think about a way of tracking the most recent row (if you want to keep the data changes). If you don't won't that option you probably need to refresh until last period that might be modified.

    I hope that helps,

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      I have a similar scenario 

      I have only last updated date in my data 

      so o enable query parameters on that field 

      so it runs fine for some refreshes

      all of a sudden, it starts duplicating the values showing multiple rows and totals gets inflated 

      so as per documentation, we cannot use same column for both detect data changes and partitions 

      how can I resolve this ??

       

  • v-janeyg-msft's avatar
    v-janeyg-msft
    Community Support
    4 years ago

    Hi, ramashish74 

     

    You can see the official doc about troubleshoot incremental refresh, the error may cause by the Post dates have changed.

     

    Detail: With a refresh operation, only data that has changed at the data source is refreshed in the dataset. As the data is divided by a date, it’s recommended post (transaction) dates are not changed.

    If a date is changed accidentally, then two issues can occur: Users notice some totals changed in the historical data (that is not supposed to happen), or during a refresh an error is returned indicating a unique value is not in fact unique. For the latter, this can happen when the table with incremental refresh configured is used in a 1:N relationship with another table as the 1 side and should have unique values. When the data is changed (for a specific ID), that ID then appears in another partition and the engine will detect the value is not unique.

     

    Did I answer your question? Please mark my reply as solution. Thank you very much.
    If not, please feel free to ask me.

    Best Regards,
    Community Support Team _ Janey