Scheduled dataset refresh fails randomly

Hi - could do with some help:

Our dataset connects to a API which then connects to our database.   On our scheduled refreshes it normally refreshes fine, however at random points some of the refreshes will just fail with one of 3 different processing error messages :

1. We reached the end of the buffer.
2. (504) Gateway Time-out
3. (502) Bad Gateway

There is not order to any of these errors they just happen randomly. We could go weeks without any problems then out of the blue one of the refreshes fails with one of these errors.

We have tried multiple different things to try and combat this :

1. Limited the amount of results returned by the API and then just called the query recursively to get all the results
2. Doubled the servers CPU & RAM 
3. Extended the Timeout for each query in Power BI

We are just wondering is anyone has had any sort of problem like this and any soultions to try would be greatly appreciated.