Getting data from web api in parallel

[jkbrookes]

Hello, first time asking a question here so thank you in advance for any help.

I've got a custom m-script that is authenticating to our monitoring platform's api using oauth.  It uses their search api to pull a list of every device we are monitoring.  Following some hints I got online I am able to pull the multiple pages of items one at a time.

The platform then has a "get device" api where I can get further details about each individual device.  I want to do this for each device that is returned in the above search but ideally that will be done in parallel as that search can return 8000-10000 devices (and thus 8000-10000 calls to the get device api).

Assuming I could do this in parallel there would be nothing stopping me from pulling the search pages in parallel as well.

How can I run these api calls in parallel with each other?  I believe I could set up separate queries if I knew the number of devices and pages that would be returned, but those things will be very dynamic.

[Eric_Zhang]

@jkbrookes

I think you can save the device api url in one column and add an extra column with Web.Contents.

Check this similar thread.

[jkbrookes]

So I added the extra column calling the function which calls Web.Contents and I pass the url to that function:

#"Added Custom" = Table.AddColumn(DeviceTable, "Records", each GetAPI(#"Base URL" & [uniqueId] & "/devices/" & [id])),

DeviceTable is generated from some other api calls and can contain anywhere from 8000-10000 rows in it.  It still looks like the GetAPI function is being called one at a time, row by row, waiting for each row's GetAPI function call to finish before moving on to the next row.

What I'd like to happen is have all of those rows run their respective GetAPI function at the same time (or at least call them in larger chunks than one at a time).  Is this possible?  Am I calling something wrong?