Processing HTTP multipart responses in Power Query

[xli629b]

For our Power BI connector, we have very large responses that we need to stream part-by-part in an HTTP multi-part response; this would greatly improve performance as we would be able to send each chunk of data as soon as it becomes available.

The response, conforming to HTTP multipart/mixed spec, looks like this in its raw form:

---
content-type: application/json; charset=utf-8

{"hasNext":true,"data":{"books":[]}}
---
content-type: application/json; charset=utf-8

{"hasNext":true,"incremental":[{"path":["books",0],"items":[{"title":"Things Fall Apart","author":"Chinua Achebe"}]}]}

 The server streams multiple parts over time with separators and content-type info between each part.

If you use Web.Contents, it just returns the whole string displayed above instead of interpreting the multipart response correctly. We could do text processing on that string, but that defeats the point. Is there a way to handle multipart responses in Power Query so that each part can be processed in a streaming manner?

[lbendlin]

What is the original response code?  Remember you have this:

Handling status codes with Web.Contents for Power Query connectors - Power Query | Microsoft Learn

The Web.Contents function has some built-in functionality for dealing with certain HTTP status codes. The default behavior can be overridden in your extension using the ManualStatusHandling field in the options record.