Forum Discussion
Helper ICall web API, receive ZIP, decompress multiple JSON files, Load Them
Hello everyone,
Still a noob... I have a simple question. I searched around the forums and found this discussion as the most relevant but I can't make it work and maybe my case is different so I thought to ask.
I have an API connection that I can access via a simple web call:
This call returns a ZIP file containing many JSON files.
Making the call is simple but I get stuck on when I receive the ZIP file... how can I:
- Decompress the many JSON files
- Get M/PowerQuery to recognise them as JSON files (which we all know it can)
- Load them into 1+ tables
Preferably 1 table would be great but I can work with millions of them... My issue is how to go from Step 1 to Step 3. I have tried variations of the forum post I linked above and some other google results like:
- https://blog.crossjoin.co.uk/2015/12/08/working-with-compression-in-power-query-and-power-bi-desktop/
- https://social.technet.microsoft.com/Forums/en-US/1c017585-9722-41a6-a23d-51d4c3d64adb/can-i-query-csv-file-in-zip-file-by-excel-power-query-?forum=powerquery
As always, I thank & salute you all!
Rafa
dyingod,
Yes, File.Contents function requires path parameter.
In addition, I haven't found any built-in decompress function that can be used to decompress .zip file. Is there any possibility that your web service return .gzip file? If so, you can use Binary.Decompress to automatically proceed this transformation.
Regards,
Lydia
7 Replies
dyingod,
Create the unzip function as described in this blog: http://sql10.blogspot.sg/2016/06/reading-zip-files-in-powerquery-m.html . Then invoke the function and create a custom column in the returned table. There is an example for your reference.
Regards,
Lydia
dyingodHey Lydia!
Thanks so much for this. I had also looked into that link but couldn't manage to do anything with it.
I get stuck here:
let Source = File.Contents("[Full path to your zip file]"), Files = UnzipContents(Source) in FilesAt the [Full path to your zip file], which make me think of a place in my drive but obviously it's not as it's within Power BI so what do I put in there? I tried calling the query with the ZIP by name but that didn't work. I really am a newbie. :(
I think the Custom Column trick sounds amazing and I never thought that possible... but I need to get over this other obstacle first. Can I call on your awesome help again? :)
Thank you!
Rafadyingod,
Is there any possiblity that you download the zip file to your computer from web?
Regards,
Lydia
- apo1979prio
Hello!!!
Only 7 Steps for GET JSON:
let
Source = SharePoint.Files("https://Mysharepoint", [ApiVersion = 15]),
Step01 = Table.SelectRows(Source, each Text.StartsWith([Folder Path], "https://Mysharepoint_SearchFolder/")),
Step02 = Table.SelectColumns(Step01, {"Content", "Name", "Extension", "Folder Path"}),
Step03 = Table.SelectRows(Step02, each [Extension] = ".json"),
Step04 = Table.SelectColumns(Step03, {"Content"}),
Step05 = Table.AddColumn(Step04, "Json", each Json.Document([Content], 65001)),
Step06 = Table.SelectColumns(Step05, {"Json"}),
Step07 = Table.ExpandListColumn(Step06, "Json")
in
Step07
