Forum Discussion

robertDTS's avatar
robertDTS
New Member
7 years ago

Data Refresh Fails, Error message claims duplicate ID value in table

I have a PBIX file where when I go to refresh my data from a SQL server the refresh eventually fails with an error message that says one of my tables contains duplicate IDs. I went to my actual SQL D...
Need Help
Anonymous's avatar
Anonymous
Not applicable
7 years ago

We have about 20 Power BI app workspaces scheduled to refresh on a regular bases... we get the false duplicate error message about 1 in every 100 refreshes. For those with the false duplicate error reported, we perform a manual on-demand refresh from the service and this solves the false reported duplicate 100% of the times. 

 

It has been confirmed as a "false" duplicate error, but must be some other error.

 

Maybe its something to do with not clearing the table completely before reloading.

 

We have also seen the same behavior with PBI desktop, and again just doing another immediate refresh the reported duplicate is gone.    Weird

  • Anonymous's avatar
    Anonymous
    Not applicable
    7 years ago

    This "false duplicate" error is still occurring for us. However it is increasing in frequency, we are now seeing this in more and more PBI apps, and the false error is happening about 20% of the time during scheduled refreshes. We are now seeing it reported as a problem experienced by an increased number of content producers too.  

     

    We can't reproduce the false error on-demand which limits our ability to troubleshoot.  We are sharing this information here in hopes of keeping MS and the community aware of the problem.

     

    We currently ignore the false error messages as it disappears with the next refresh cycle.   However it appears the symptom is much more broad, varied, and not localized to our tenant or premium capacity.

  • bainmail's avatar
    bainmail
    Regular Visitor
    7 years ago

    Can you give more information on "we perform a manual on-demand refresh from the service and this solves the false reported duplicate 100% of the times." 

     

    I am having the same issue and need to try this, but I am not sure what to do. Thanks.

    • Anonymous's avatar
      Anonymous
      Not applicable
      7 years ago

      I should NOT have said it solves the problem... it just proves that there is not really a duplicate ID 100% of the time.

       

      The problem is still happening for us and how at a much greater frequency.

      • mahidharsridhar's avatar
        mahidharsridhar
        New Member
        6 years ago

        Can confirm this is still an issue, started facing this recently. Also confirmed the source data has no duplicates at all, when this error pops up, in 99% of cases just re-doing the refresh makes it go away. Definitely a bug where the string matching routine within PBI thinks something is a dup, when in reality it isn't

         

        For a workaround, please see below

        https://exceleratorbi.com.au/preventing-managing-refresh-failures-in-power-bi/