Duplication of values/rows when parsing a JSON

[Hank_]

Hello everyboy, I really hope you can help with this issue that's giving me headache!

I'm importing from MySQL several tables with 3 columns:

 - id

 - json

 - timestamp

I transform the JSON and then expand it to get all the columns. There I have dozens of them.

At this point the query somehow creates duplicates! The id, and all the values of the rows are duplicated. (the id is out of the json).

At every data refresh the number of duplicated changes, between 1%-3%. It's weird! 

EDIT (I forgot to write this): THE DUPLICATED ROWS OVERWRITE OTHER ROWS

Did anybody had this issue? May this be a bug? When working with less records (50-60000) I don't have this problem.

I cannot upload the pbix because of Company's data... I cannot change all the jsons I receive because I get them from a db, and there is 200000 records.

[Hank_]

Sorry for not replying, but I've been very busy and when possible we tried to update PBI, .net Framework and other stuff with the IT. I've exported manually the json files from MySQL and imported them as xlsx files and everything was fine. 

Finally we understood what was wrong: it was the connector to MySQL, which was running on my local PC thanks to XAMPP 7.4.8.

I got installed from my IT MariaDB connector 3.1.13 and now I can import without those duplication of records that overwrites other records! I would be happy to report this to Microsoft because we are sure this is a kind of bug.

[jennratten]

Great news.

[jennratten]

Can you check your json to see if related objects are included?  Related objects are included in the example below.  The table in the selected cell contains all values from the current row, which can be seen in the preview section.  If all columns, including the related objects, were expanded, the values would be duplicated.  Note, the example is a snip of SQL table - I didn't have a JSON example handy, but the theory is the same.  To fix this, you would need to either omit related objects from the json, or only expand non-related objects.

This page has information about json and related objects.

https://jsonapi.org/format/