Forum Discussion
Convert this code which is a Function into a self looping piece of code for batch
Hi AlanFoley
You can loop using List.Generate, something like this
let
Source = List.Generate(()=>
[Result = Web.Contents("https://www.microsoft.com/en-us/microsoft-365/blog/excel/"), nextPageKey = 2],
each [nextPageKey] <= 10,
each [ Result = Web.Contents("https://www.microsoft.com/en-us/microsoft-365/blog/excel/",
[ RelativePath = Text.From(nextPageKey)]
),
nextPageKey = [nextPageKey] + 1
],
each [Result])
in
Source
In your case it would look more like this
let
Source = List.Generate(()=>
[Result = "", appendedData = {}, nextPageKey = "", loop = 0],
each [nextPageKey] <> null,
each [ Result = Json.Document(Web.Contents("https://domain.com/api/v2",
[ RelativePath = if loop = 0 then "entities?entitySelector=type%28APPLICATION%29&from=now-8d&fields=%2BfirstSeenTms"
else "entities?nextPageKey=" & nextPageKey,
Headers=[Authorization="Api-Token xxxxx"]
]
)),
loop = [loop] + 1,
nextPageKey = try Result[nextPageKey] otherwise null,
currentData = Result[entities],
appendedData = if currentData is null and data is null then {}
else if data is null then List.Combine({{}, currentData})
else if data is null then List.Combine({data, {}})
else List.Combine({data, currentData})
],
each [Result])
in
Source
I have no way to test this for you as I don't know what website you are working with or have authorisation to it.
Also, there are things like 'data' which aren't defined so I don't know what this is supposed to hold.
In this section of code
you are testing 'data' twice to see if it is null but trying to do 2 different things so that isn't right 🙂
Hopefully this will get you started along the right path. See if you can modify my code in the attached file to work for you and if you can't please post back with more info about what exactly you are doing and supply your PBIX file if you can.
Regards
Phil
- AlanFoley4 years agoFrequent Visitor
Hi Phil
Thank you for your detailed response and for taking the time to assist me
I will need some time to review and understand your code as well as retrofit your suggestions in my PowerBI desktop file
I will provide some feedback as soon as I can
Thanks again
Alan
- AlanFoley4 years agoFrequent Visitor
Hi Phil
In hindsight it may be better to post the actual requirment instead of trying to fix my dodgy code
As per your request I will attached the .pbix - I will also attach the actual returned data in .json format for each of the API calls
This is the sequence of API calls (Domain name abd Api-Token modified)
1. API Call: https://domain.com/api/v2/entities?entitySelector=type%28APPLICATION%29&from=now-8d&fields=%2BfirstSeenTms&Api-Token=yyyy
On the first call there are a number of "Data Points" returned:
- totalCount - Number of records in response
- pageSize - Limit imposed per API call
- nextPageKey - Value to be passed to get the next payload
- entities - Actual data to be used2. API Call: https://domain.com/api/v2/entities?nextPageKey=yyyy%22&Api-Token=xxx
On the second call we pass the "nextPageKey" parameter only and its value - the "totalCount", "pageSize" and "entities" "Data Points" are returned (Because this is the last response with data the "nextPageKey" value is actually ommitted so we can't check if nextPageKey=NULL or something similarIf the returning response had many rows there would be a unique nextPageKey for the next response
Much Appreciated
Thanks
Alan
- AlanFoley4 years agoFrequent Visitor
I am not able to attach any files ..how do I attach files?
- PhilipTreacy4 years agoSuper User
hi AlanFoley
You can use OneDrive, Dropbox or Google Drive to upload your files then link to them.
Regards
Phil
- AlanFoley4 years agoFrequent Visitor
Hi Phil
Have spent quite some time understanding the solution you provided ..I modified my code as per your notes - however no output is returned - the list is empty
I have included the "Query No Output.PNG" file to the file share showing the return "Empty List" as well as a screenshot of the code I am using - It could be something simple I am missing
Appreciate if you could have another look
Thanks
Alan
- AlanFoley4 years agoFrequent Visitor
Just noticed the screenshot does not show the value of "in"
It is as below:
in
Source