Forum Discussion
Anonymous
6 years agoNot applicable
Gateway "System.OutOfMemoryException: Array dimensions exceeded supported range" Error
Hello, I am working on getting a Dataflow to pull data from a Web API via an On-premises Gateway. The Web API is returning ~2M rows of data, ~1.5 GB in size in ~3 minutes. The Web API applies a sign...
- Anonymous6 years ago
It turns out this was a case of bad Power Query logic. I removed:
UnpackedStream = Lines.FromBinary( WebResult ) {0},So that my Power Query logic is:
DataRequestHeaders = ( _accessToken as text ) as record => [ #"Authorization" = "Bearer " & _accessToken, #"Content-Type" = "application/json; charset=utf-8" ], DataRequestParameters = [ RelativePath = "customers", Headers = DataRequestHeaders( AccessToken ), Timeout = #duration(0,1,0,0), ExcludedFromCacheKey={"Authorization"}, ManualStatusHandling = { 500 }, IsRetry = true ], WebResult = Web.Contents( "https://myUrl/", DataRequestParameters ), DataRequestResult = Json.Document(WebResult), RecordTable = Table.FromList(DataRequestResult, Splitter.SplitByNothing(), null, null, ExtraValues.Error), ResultTable = Table.ExpandRecordColumn( RecordTable, "Column1",{ "ID","Customer" /* 20 additional columns */ })And I am no longer encountering any issues. Thanks,
Anonymous
6 years agoNot applicable
It turns out this was a case of bad Power Query logic. I removed:
UnpackedStream = Lines.FromBinary( WebResult ) {0},So that my Power Query logic is:
DataRequestHeaders = ( _accessToken as text ) as record =>
[
#"Authorization" = "Bearer " & _accessToken,
#"Content-Type" = "application/json; charset=utf-8"
],
DataRequestParameters = [
RelativePath = "customers",
Headers = DataRequestHeaders( AccessToken ),
Timeout = #duration(0,1,0,0),
ExcludedFromCacheKey={"Authorization"},
ManualStatusHandling = { 500 },
IsRetry = true
],
WebResult = Web.Contents( "https://myUrl/", DataRequestParameters ),
DataRequestResult = Json.Document(WebResult),
RecordTable = Table.FromList(DataRequestResult, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
ResultTable = Table.ExpandRecordColumn( RecordTable, "Column1",{ "ID","Customer" /* 20 additional columns */ }) And I am no longer encountering any issues. Thanks,