Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
Hank_
Helper I
Helper I

Duplication of values/rows when parsing a JSON

Hello everyboy, I really hope you can help with this issue that's giving me headache!

 

I'm importing from MySQL several tables with 3 columns:

 - id

 - json

 - timestamp

 

I transform the JSON and then expand it to get all the columns. There I have dozens of them.

 

At this point the query somehow creates duplicates! The id, and all the values of the rows are duplicated. (the id is out of the json).

At every data refresh the number of duplicated changes, between 1%-3%. It's weird! 

EDIT (I forgot to write this): THE DUPLICATED ROWS OVERWRITE OTHER ROWS

 

Did anybody had this issue? May this be a bug? When working with less records (50-60000) I don't have this problem.

I cannot upload the pbix because of Company's data... I cannot change all the jsons I receive because I get them from a db, and there is 200000 records.

3 REPLIES 3
Hank_
Helper I
Helper I

Sorry for not replying, but I've been very busy and when possible we tried to update PBI, .net Framework and other stuff with the IT. I've exported manually the json files from MySQL and imported them as xlsx files and everything was fine. 

 

Finally we understood what was wrong: it was the connector to MySQL, which was running on my local PC thanks to XAMPP 7.4.8.

 

I got installed from my IT MariaDB connector 3.1.13 and now I can import without those duplication of records that overwrites other records! I would be happy to report this to Microsoft because we are sure this is a kind of bug.

Great news.

jennratten
Super User
Super User

Can you check your json to see if related objects are included?  Related objects are included in the example below.  The table in the selected cell contains all values from the current row, which can be seen in the preview section.  If all columns, including the related objects, were expanded, the values would be duplicated.  Note, the example is a snip of SQL table - I didn't have a JSON example handy, but the theory is the same.  To fix this, you would need to either omit related objects from the json, or only expand non-related objects.

 

jennratten_0-1630158698373.png

 

 

This page has information about json and related objects.

https://jsonapi.org/format/ 

Helpful resources

Announcements
July 2024 Power BI Update

Power BI Monthly Update - July 2024

Check out the July 2024 Power BI update to learn about new features.

July Newsletter

Fabric Community Update - July 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors