Forum Discussion

DHB's avatar
DHB
Helper V
9 years ago

Duplicate data

I’ve loaded a .txt file into PBI and am getting duplicates in what should be a column of unique values. These don’t appear in the source and when I load the same data into Tableau this doesn’t happen...
Need Help
BhaveshPatel's avatar
BhaveshPatel
Super User
9 years ago

These seems strange behaviour. I have never encountered such a problem. If you can share your file, I would test it in both the (Tableau and PowerBI) environments you have mentioned and see what is wrong with the ETL process.

 

Meanwhile, Go to the Query Editor, and Select Transform and Go to Statistics option.

Please select the distinct count values first and you will see how many no. of distinct rows you have in your text file.

 

 

 

DHB's avatar
DHB
Helper V
9 years ago

Hi Bhavesh,

 

Thank you for the advice.

 

When I put the data into the Query Editor and changed the id column from whole number to text it resolved the issue.  The same thing worked for other .txt files that had the same problem.  This is good news but it's going to be awkward doing that every time.  Can I automate this somehow?

 

I'll have to get permission to share a file so will let you know asap on that.

  • BhaveshPatel's avatar
    BhaveshPatel
    Super User
    9 years ago

    Hi DHB,

     

    Unfortunately not, PowerBI selects the best data type suitable for your column automatically  but sometimes it may not be the correct data type and you have to manually override that in the query stage. 

     

    Hope this has clarified your understanding.