Starting December 3, join live sessions with database experts and the Microsoft product team to learn just how easy it is to get started
Learn moreGet certified in Microsoft Fabric—for free! For a limited time, get a free DP-600 exam voucher to use by the end of 2024. Register now
We are reading data from a REST API. The process is fairly common: do one call to get a list of IDs, then do one or more calls for each ID to get the details. For a small data set, everything works fine. With a large number of IDs, the number of REST calls arriving at the server simulateously is extremely high. This causes the server to return 502/503 errors and the data loading fails.
With other languages, a typical solution is to either make the calls sequentially instead of in parallel, or to rate-limit the calls in some way. The reduces the onslaught of requests to the server and allows the server to answer the requests instead of rejecting them. Is there a way to accomplish this with Power Query?
Regards,
G
Solved! Go to Solution.
You can use Function.InvokeAfter to introduce a delay between requests. This code has a 2 second delay between web requests
let
id_list = {1 .. 5},
GetWeb = (id) => Json.Document(Web.Contents("https://jsonplaceholder.typicode.com/todos/" & Text.From(id))),
Output = List.Transform(id_list, each Function.InvokeAfter( ()=>GetWeb(_), #duration(0,0,0,2)))
in
Output
regards
Phil
If I answered your question please mark my post as the solution.
If my answer helped solve your problem, give it a kudos by clicking on the Thumbs Up.
Proud to be a Super User!
You can use Function.InvokeAfter to introduce a delay between requests. This code has a 2 second delay between web requests
let
id_list = {1 .. 5},
GetWeb = (id) => Json.Document(Web.Contents("https://jsonplaceholder.typicode.com/todos/" & Text.From(id))),
Output = List.Transform(id_list, each Function.InvokeAfter( ()=>GetWeb(_), #duration(0,0,0,2)))
in
Output
regards
Phil
If I answered your question please mark my post as the solution.
If my answer helped solve your problem, give it a kudos by clicking on the Thumbs Up.
Proud to be a Super User!
Beautiful!
Starting December 3, join live sessions with database experts and the Fabric product team to learn just how easy it is to get started.
March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Early Bird pricing ends December 9th.
User | Count |
---|---|
94 | |
92 | |
82 | |
71 | |
49 |
User | Count |
---|---|
143 | |
121 | |
111 | |
59 | |
57 |