Forum Discussion
Prefer CSV or JSON for my dataflows
I can't find it at the moment but I did see a blog comparing load times and think csv came out top. Under the hood when you run a data flow it stores the data as csv in azure data lake storage so it's very optimised for that sort of thing.
My hunch would be that you're unlikely to notice a huge difference. (Worth testing!)
One thing to consider that will make a difference if you're reading lots of csv (or json) in is two separate the ingest from transformations in dataflows (assuming you have premium or premium per user).
Create a first data flow that simply pulls in the raw data.
Then create a second which uses the ingested data in the first to do transformations on.
Hi bcdobbs ,
Do you mean to say that, according to that blog, csv seemed to be taking the least time? It would be great if you could direct me to that blog or any other source discussing this issue.
Also, yes I have made separate dataflows to perform the extraction and transformation operations on. This actually does help a lot.
Best Regards,
Diptanshu Lal
Sorry. Blog was csv vs parquet.
https://www.datalineo.com/post/parquet-and-csv-querying-processing-in-power-bi
Both csv and json are plain text formats so there won't be much in it. However json carries all the field names for every record and so for the same volume of data the files will be larger. Therefore I would expect csv to load marginally faster than json (a lot -
also depends on how flat the json files are).
Honestly I don't think you'll find a huge performance impact either way.