Forum Discussion
powerbi load JIRA worklog via RESTAPI so slow
Hi Anonymous ,
In your query, this one
#"Add worklog url" = Table.AddColumn(#"Added Index", "worklog url", each #"my company JIRA"&"/rest/api/latest/issue/"&[Issue key]&"/worklog"),
#"Invoked Custom Function1" = Table.AddColumn(#"Add worklog url", "worklog", each Json.Document(Web.Contents([worklog url])))
has queried again by the API in each rows, which means if you have 1000 rows in a page, the 1000 rows with be re-queried after executing all the queries above,which will increase the calling numbers of API,so you can first remove the last query to check whether the loading has been sped up.If it is the issue,will remove the query be a best solution for you?
Kelly
hi, v-kelly-msft
I had the same feeling about performance bottleneck you mention above, which is API call per row, And it is testified by the fact data will be loaded fast if #"Invoked Custom Function1" line is removed.
What I am curious is that there is the same API call per row in https://community.powerbi.com/t5/Desktop/Get-data-from-Jira-Rest-API/td-p/516130, but seems fixed by changing from "each Json.Document(Web.Contents(#"AddIndex"{[Index]}[Custom]))" to "each Json.Document(Web.Contents([Custom]))". That give me hope that maybe the poor performence is caused by some rule mentioned at this link, which is "avoiding loading table in power query"?
If the poor performance is caused by API call per row 100 percentage, then I don't know how to get worklog from JIRA quickly. there are 3 potential approaches to get worklog data what I can find by now:
- get whole worklog since date via https://jiradc.int.net.mycompany.com/rest/api/latest/worklog/updated?since=sometimestamp
- get worklog one by one via "GET /rest/api/2/issue/{issueIdOrKey}/worklog/{id}"
- get worklog for multi issues via eg. https://jiradc.int.net.mycompany.com/rest/api/latest/search?jql=issue%20in%20(ISSUE1, ISSUE2)&fields=worklog
BUT all above are not satisfied for me since
for 1: I don't want to hunt my interested data among whole company, it is too huge.
for 2:there are 4000 issues by now, one issue one api cal takes too much time.(the current problem we mentioned here)
for 3: was thought excellent, but sadly, the max count of worklog return per issue is 20, but of course there are plenty of issues whose count of worklogs excced.
I am tortured by this for 3 days!
- v-kelly-msft6 years agoCommunity Support
Hi Anonymous ,
Go to query editor,at the bottom of column profile, choose"entire dataset",as shown below:
Then remove " #"Invoked Custom Function1" = Table.AddColumn(#"Add worklog url", "worklog", each Json.Document(Web.Contents([worklog url])))" and go to the column "#"Add worklog url",check the number of rows and distinct rows to see whether they are close to: (using groupby functionality)
Best Regards,
KellyDid I answer your question? Mark my post as a solution!- Anonymous6 years agoNot applicable
#"Invoked Custom Function1" and "#"Add worklog url" have the same number of rows
(distinct row), about 4000 rows. But to be honestly, I don't know the point to check rows count after we both realize 4000 rows lead 4000 API calls maybe cause time-cost. what I am curious is how poor performance gone at https://community.powerbi.com/t5/Desktop/Get-data-from-Jira-Rest-API/td-p/516130, am I missing something?
- v-kelly-msft6 years agoCommunity Support
Hi Anonymous ,
Then you'd better use another kind of API which can process multiple queries...
Best Regards,
KellyDid I answer your question? Mark my post as a solution!