Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The ultimate Microsoft Fabric, Power BI, Azure AI & SQL learning event! Join us in Las Vegas from March 26-28, 2024. Use code MSCUST for a $100 discount. Register Now

Reply
gaccardo-slb
Advocate II
Advocate II

How to avoid HTTP 502/503 when accessing a REST API

We are reading data from a REST API.  The process is fairly common: do one call to get a list of IDs, then do one or more calls for each ID to get the details.  For a small data set, everything works fine.  With a large number of IDs, the number of REST calls arriving at the server simulateously is extremely high.  This causes the server to return 502/503 errors and the data loading fails.

 

With other languages, a typical solution is to either make the calls sequentially instead of in parallel, or to rate-limit the calls in some way.  The reduces the onslaught of requests to the server and allows the server to answer the requests instead of rejecting them.  Is there a way to accomplish this with Power Query?

 

Regards,

G

1 ACCEPTED SOLUTION
PhilipTreacy
Super User
Super User

Hi @gaccardo-slb 

You can use Function.InvokeAfter to introduce a delay between requests.  This code has a 2 second delay between web requests

 

let
    id_list = {1 .. 5},
    GetWeb = (id) => Json.Document(Web.Contents("https://jsonplaceholder.typicode.com/todos/" & Text.From(id))),
    Output = List.Transform(id_list, each Function.InvokeAfter( ()=>GetWeb(_), #duration(0,0,0,2)))    
in
    Output

 

regards

Phil


If I answered your question please mark my post as the solution.
If my answer helped solve your problem, give it a kudos by clicking on the Thumbs Up.



Did I answer your question? Then please mark my post as the solution.
If I helped you, click on the Thumbs Up to give Kudos.


Blog :: YouTube Channel :: Connect on Linkedin


Proud to be a Super User!


View solution in original post

2 REPLIES 2
PhilipTreacy
Super User
Super User

Hi @gaccardo-slb 

You can use Function.InvokeAfter to introduce a delay between requests.  This code has a 2 second delay between web requests

 

let
    id_list = {1 .. 5},
    GetWeb = (id) => Json.Document(Web.Contents("https://jsonplaceholder.typicode.com/todos/" & Text.From(id))),
    Output = List.Transform(id_list, each Function.InvokeAfter( ()=>GetWeb(_), #duration(0,0,0,2)))    
in
    Output

 

regards

Phil


If I answered your question please mark my post as the solution.
If my answer helped solve your problem, give it a kudos by clicking on the Thumbs Up.



Did I answer your question? Then please mark my post as the solution.
If I helped you, click on the Thumbs Up to give Kudos.


Blog :: YouTube Channel :: Connect on Linkedin


Proud to be a Super User!


Beautiful!

Helpful resources

Announcements
Fabric Community Conference

Microsoft Fabric Community Conference

Join us at our first-ever Microsoft Fabric Community Conference, March 26-28, 2024 in Las Vegas with 100+ sessions by community experts and Microsoft engineering.

February 2024 Update Carousel

Power BI Monthly Update - February 2024

Check out the February 2024 Power BI update to learn about new features.

Fabric Career Hub

Microsoft Fabric Career Hub

Explore career paths and learn resources in Fabric.

Fabric Partner Community

Microsoft Fabric Partner Community

Engage with the Fabric engineering team, hear of product updates, business opportunities, and resources in the Fabric Partner Community.