Forum Discussion
Export Jira V3 api data to PowerBI dataflow
- 11 months ago
Hi Ahmed-Elfeel ,
Yeah I knew procedurally what needed to be done, but familiarity with M is still developing.
After a solid night's reading and a bit of trial and error here's the final (working) solution:
let // Fetch an individual page, nextPageToken is an optional paramater GetPage = (optional nextPageToken as text) => let // Create URL, use nextPageToken if present Url = "https://mycompanydomain.atlassian.net", relativePath = if nextPageToken = null or nextPageToken = "" then "/rest/api/3/search/jql?jql=project=PROJ&maxResults=100&fields=*all" else "/rest/api/3/search/jql?jql=project=PROJ&maxResults=100&fields=*all&nextPageToken=" & nextPageToken, Response = Json.Document(Web.Contents(Url, [RelativePath = relativePath])), Issues = try Response[issues] otherwise {}, NextToken = try Response[nextPageToken] otherwise null, Result = [Issues = Issues, NextToken = NextToken] in Result, // Recursive function to collate results from all pages GetAllPages = (nextPageToken as nullable text) as list => let CurrentPage = GetPage(nextPageToken), CurrentIssues = CurrentPage[Issues], NextToken = CurrentPage[NextToken], MoreIssues = if NextToken <> null then @GetAllPages(NextToken) else {}, Combined = List.Combine({CurrentIssues, MoreIssues}) in Combined, // Call the recursive function starting with nextPageToken = null AllIssues = GetAllPages(null),Appreciate your assistance though 🙂
Regards
Hi Ahmed-Elfeel ,
Yeah I knew procedurally what needed to be done, but familiarity with M is still developing.
After a solid night's reading and a bit of trial and error here's the final (working) solution:
let
// Fetch an individual page, nextPageToken is an optional paramater
GetPage = (optional nextPageToken as text) =>
let
// Create URL, use nextPageToken if present
Url = "https://mycompanydomain.atlassian.net",
relativePath = if nextPageToken = null or nextPageToken = ""
then "/rest/api/3/search/jql?jql=project=PROJ&maxResults=100&fields=*all"
else "/rest/api/3/search/jql?jql=project=PROJ&maxResults=100&fields=*all&nextPageToken=" & nextPageToken,
Response = Json.Document(Web.Contents(Url, [RelativePath = relativePath])),
Issues = try Response[issues] otherwise {},
NextToken = try Response[nextPageToken] otherwise null,
Result = [Issues = Issues, NextToken = NextToken]
in
Result,
// Recursive function to collate results from all pages
GetAllPages = (nextPageToken as nullable text) as list =>
let
CurrentPage = GetPage(nextPageToken),
CurrentIssues = CurrentPage[Issues],
NextToken = CurrentPage[NextToken],
MoreIssues = if NextToken <> null
then @GetAllPages(NextToken)
else {},
Combined = List.Combine({CurrentIssues, MoreIssues})
in
Combined,
// Call the recursive function starting with nextPageToken = null
AllIssues = GetAllPages(null),Appreciate your assistance though 🙂
Regards
Make sure this performs well enough. With recursive solutions (and List.Accumulate) you are lugging all the so far retrieved data around into each new iteration. That can lead to memory shortage. Better to use List.Generate and the browser cache.