Dataflow failing with 500 error only first time then it works

[pieduke88]

I am trying to get data from an Autodesk API using a Dataflow Gen1

The request succeeds intermittenly: first time it fails after exactly 14s, then if I try right after it works.

If I then wait an hour and try again, it fails after 14s and so on.

What could be causing this? The error just says 500 internal server error which is not really documented on the Autodesk API side  - this is the one I am using https://aps.autodesk.com/en/docs/model-derivative/v2/reference/http/metadata/urn-metadata-guid-prope...

Thanks

[Anonymous]

Hi @pieduke88,

As we haven’t heard back from you, we would like to follow up to see if the solution provided by the super user resolved your issue. Please let us know if you need any further assistance.
If our super user response resolved your issue, please mark it as "Accept as solution" and click "Yes" if you found it helpful.

Regards,
Vinay Pabbu

[Anonymous]

Hi @pieduke88,

May I ask if you have gotten this issue resolved?

If it is solved, please mark the helpful reply or share your solution and accept it as solution, it will be helpful for other members of the community who have similar problems as yours to solve it faster.

Regards,
Vinay Pabbu

[Anonymous]

Hello @pieduke88,
Just wanted to drop a quick note, were you able to resolve the issue you raised?
If so, it would be really helpful for the community if you could mark the answer that helped you the most. If you're still looking for guidance, feel free to give us an update, we’re here for you!

Regards,

Vinay Pabbu

[rohit1991]

Hi @pieduke88 

This looks like a cold-start/idle issue on the Autodesk endpoint: after ~1 hr idle the first call dies at ~14s with 500, the next call works.

Fixes

• Add retries with backoff in Power Query (Function.InvokeAfter) and raise Timeout on Web.Contents (30–60s).

• Send a quick warm-up “ping” before the heavy call, or schedule a small keep-alive every 30–60 min.

• Ensure OAuth token refresh happens before the call; 500s can hide auth expiry.

• Watch rate limits; stagger requests if needed.

• If it keeps happening, log timestamp/request ID and open a ticket with Autodesk.

[speedramps]

This kind of intermittent 500 Internal Server Error after a period of inactivity is common problem.
It then works ok on the next requests, until there is another period of inactivity. Afterwhich the problem reoccurs.

I am soory I am not an expert on resolving but if you search on Google then you will find technical answer, like this one ...

https://learn.microsoft.com/en-us/answers/questions/2074648/intermittent-error-500-on-first-access-w...

I have heard of data engineers running a dummy API to "stir up" a server before runing a API to get data.