Forum Discussion
Unable to Combine Data - Custom Function
- 4 years ago
Hi ITSNev ,
this can indeed be a pain.
Try combining these queries into one like so:let FnGetProjects_ = (offset as number) => let //start_date=eRSActualsStartOfMonth, //end_date=MaxProjectEndDate, records = Number.ToText(offset), header = [ #"Authorization"="Bearer XXXXXXXXXXXXXXXXX", #"Content-Type"= "application/json"], content = "", //"{""last_date:ex"": [null, ""2022-05-1""]""}", url = "https://app.YYYYYYYYYYYYYYYY.cloud/rest/v1/", response = Web.Contents(url, [RelativePath = "projects/search?", Query=[ offset = records, limit = "500" ], Content=Text.ToBinary(content),Headers=header]), out = Json.Document(response,1252) in out, Source = List.Generate ( () => [offset = 500, actuals = #"FnGetProjects"( 0 ) ] , //each [actuals][total_count] <> 0, each not List.IsEmpty([actuals][data]), each [offset = [offset] + 500, actuals = #"FnGetProjects_" ([offset]) ], each [actuals] ), #"Converted to Table" = Table.FromList(Source, Splitter.SplitByNothing(), null, null, ExtraValues.Error), #"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"data"}, {"data"}), #"Expanded data" = Table.ExpandListColumn(#"Expanded Column1", "data"), #"Expanded data1" = Table.ExpandRecordColumn(#"Expanded data", "data", {"XXXXXX"}) in #"Expanded data1"
Imke and ITSNev,
can I ask if the underlying problem that was causing the combination error in the PBI service had to do with calling out to a custom function in another query and if you are essentially migrating that function code into a single query above? I'm faced with a simpler scenario and have tried disaggregating portions of the same query which access data sources but cannot completely eliminate calling a helper function to format and treat a binary wb.contents call result. I had not considered moving the code for these functions into the main query instead.
Regards,
Henrik
- Element1153 years agoMemorable Member
I fully understand now what this is all about. In another thread, I got Microsoft to point me to one of their doc pages: this is main source of the problem: the Data Privacy Firewall inside the PQ engine.
https://learn.microsoft.com/en-us/power-query/dataprivacyfirewall
In a nutshell, all code outside of your query's top level let...in is not allowed to reference any code outside of the query itself. I'll publish a more exhaustive answer example in response to Imke first response later as I am still busy testing.... I got the Refresh in the Service to work but now no values are coming out! Fun, huh? 😁