Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
6 years ago
Solved

Gateway "System.OutOfMemoryException: Array dimensions exceeded supported range" Error

Hello, I am working on getting a Dataflow to pull data from a Web API via an On-premises Gateway. The Web API is returning ~2M rows of data, ~1.5 GB in size in ~3 minutes. The Web API applies a sign...
  • Anonymous's avatar
    Anonymous
    6 years ago

    It turns out this was a case of bad Power Query logic. I removed:

    UnpackedStream = Lines.FromBinary( WebResult ) {0},

     So that my Power Query logic is:

    DataRequestHeaders = ( _accessToken as text ) as record =>
    		[
    			#"Authorization" = "Bearer " & _accessToken,
    			#"Content-Type" = "application/json; charset=utf-8"
    		],
    DataRequestParameters = [
    			RelativePath = "customers",
    			Headers = DataRequestHeaders( AccessToken ),
    			Timeout = #duration(0,1,0,0),
    			ExcludedFromCacheKey={"Authorization"},
    			ManualStatusHandling = { 500 },
                IsRetry = true
    		],
    WebResult = Web.Contents( "https://myUrl/", DataRequestParameters ),
    DataRequestResult = Json.Document(WebResult),
    RecordTable = Table.FromList(DataRequestResult, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    ResultTable = Table.ExpandRecordColumn( RecordTable, "Column1",{ "ID","Customer" /* 20 additional columns */ }) 

     And I am no longer encountering any issues. Thanks,