Admin - Datasets GetDatasourcesAsAdmin request limit

[mrraut]

I have been using the 'Admin - Datasets GetDatasourcesAsAdmin' REST api (https://learn.microsoft.com/en-us/rest/api/power-bi/admin/datasets-get-datasources-as-admin) in our scripts for a while now and it has been working as expected, however, I have been getting error with Status code : 429. You have exceeded the amount of requests allowed in the current time frame and further requests will fail. Retry in xxxx seconds.

The error occurs after 30 odd requests.

Has anything changed recently on this api?

[samsonfrb · Accepted solution]

Hi all,
I got a reply from Microsoft about the issue:

We have confirmation about a new throttling limit on this specific API call to 50 calls per hour.

Product Group informs they are aware this new limit is affecting users and their actual implementations, for this reason they share this workaround in the meantime a solution is applied about the new throttling limit:

“While we work on refining the throttling limits for the GetDatasourcesAsAdmin API, we understand the importance of providing viable alternatives for our users. As a temporary workaround, we recommend leveraging our scanner APIs to access all data source information for your tenant. The scanner APIs offer a robust solution for retrieving datasets and related data source information.

For more information, please refer “-

https://learn.microsoft.com/en-us/rest/api/power-bi/admin/workspace-info-get-scan-result

I hope they align the final throttling limit with GetXXXUsersAsAdmin APIs (200 calls per hour) so that with have consistency with our scripts. Or that they implement an API that return all datasources of all semantic models of the tenant in a single call.

Using scanner API is more involved (asynchronous calls, complex Json result) requiring a longer implementation.

Best regards,

Frederick

[samsonfrb]

Hey Amine, I hope you are doing fine.

Get-PowerBIDatasource has the same problem, except that it takes a long time before throwing an exception:
+ CategoryInfo : WriteError: (Microsoft.Power...werBIDatasource:GetPowerBIDatasource) [Get-PowerBIDatasource], TaskCanceledException
+ FullyQualifiedErrorId : Une tâche a été annulée.,Microsoft.PowerBI.Commands.Data.GetPowerBIDatasource

Frederick

[aj1973]

Hello Frederick, mon ancien collègue non! Oui je vais bien et toi! je dois admettre que j'ai appris beaucoup de chose de toi 😉.

Alors: Is this an info you wanted to share with me regarding the API or an issue you want to understand?

[samsonfrb]

Hi Amine,

Doing great, thanks. It was not a question but an information.

Both 'Get-PowerBIDatasource' CmdLet and 'Admin - Datasets GetDatasourcesAsAdmin' REST API fail after 50 calls in the same hour. Before January 28 2024, both worked fine. 

Frederick

[karo]

Hi, I am experience similar issues since 31st of January 2024. Have you find any solution?

[mrraut]

I ended up adding a 60 min sleep time in the loop. Took me 2 days to go through all the datasets in our tenant.

if datasouces.status_code == 429:
                print(len(set(dataset_id)), "datasets out off",total_datasets,"scanned. Encountered 429 error. Sleeping for 3600 secs")
                sleep(3601)
                auth = ClientSecretCredential(authority = 'https://login.microsoftonline.com/',
                                                        tenant_id = tenant,
                                                        client_id = client,
                                                        client_secret = client_secret)
                access_token = auth.get_token(api)
                access_token = access_token.token
                headers = {'Authorization': f'Bearer {access_token}'}
                datasouces = requests.get('https://api.powerbi.com/v1.0/myorg/admin/datasets/'+datasets_info['id']+'/datasources', headers=headers)

[samsonfrb]

Good workaround,

Problem is that people keep adding/modifying/deleting semantic models.

We need a solution where we have a clear picture every 24h.

For reports/apps/dashboards/semantic models permissions, the limit is 200 calls/hour which is OK to crawl all permissions every 24h.
But this (around 50) limit per hour is not acceptable.
The Microsoft support team reached out to the product team. I gave this forum as additional information so that they know that it affects others organizations too.
I'll update this forum when I get news.

Frederick

[aj1973]

Please let us know when you get the reply from MSFT on the matter.

Thanks Fred

[BiLukJak]

Seems like there is implemented limitation on number of calls for that api. All my jobs were running perfectly fine until 28/01. After that it works only if reduce calls below 50 but when I try to rerun on different scope than it is getting stuck. This is the same for two different tenants in two different regions. Microsoft, fix it or update documentation.