Hi Maz_Nah,

Thank you for reaching out to Microsoft Fabric Community.

 

1. The issue occurs because, during the execution the pipeline attempts to fetch all the data for the specified date range and the connection is closed by the server due to resource constraints or timeouts. The API limits the data request based on the range or volume, causing the connection to be interrupted, and the pipeline fails to handle it properly.
2. The error indicates an HTTP failure, when attempting to retrieve data from the API endpoint and it suggests that the connection between the pipeline and the API server was interrupted or terminated unexpectedly, this can be possibly due to:
   • Server-side restrictions on the API.
   • Network issues or timeout during the data retrieval.
   • Exceeding API rate limits or query complexity
3. Break down the date range into smaller intervals (for example 2–3 days) to reduce the load on the API and avoid timeout issues.
4. The Import Schema operation fails because the schema definition is not retrieved correctly from the REST API, so define a schema manually.
5. Check the API documentation for rate-limiting policies and ensure your pipeline does not exceed the allowed limits.
6. Configure your HTTP dataset or REST linked service to include a timeout and retry policy to handle intermittent connection issues:
   • Timeout: Increase the timeout period to accommodate large data retrievals.
   • Retry Policy: Set the retry count to handle temporary/transient failures.

 

If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it! 

 

Thanks and regards,

Anjan Kumar Chippa