Limit on get request (Admin - Datasets GetDatasourcesAsAdmin)

[mhani]

Hi,

I recently started to discover Power BI Admin APIs. And I wanted to bring in the GetDatasources, to get all the data sources for the datasets In my tenant. I do it using Python.

Although when I try to get all the data sources, it stops after around 1700 datasets. Even though I have inserted sleep into the script.

Does anyone know if I am hitting a limit for requests? Or does anyone have an idea to solve this? We have around 11000 datasets, so we are far from home with 1700.

Michael

[Kagiyama_yutaka]

I think that call just stops its own enum around ~1700… if u need the full set, run a Scanner API workspace‑scan and pull the scanResult — it’s the only route that never drops on large tenants.

[Bipin-Lala · Accepted solution]

With 11,000 datasets, you have definitely moved past the "standard" Admin APIs limit.

The  GetDatasourcesAsAdmin explicitly states a limit of 300 requests per hour.

The Solution: The Scanner API (Metadata Scanning)

For a tenant of your size, you should switch to the Scanner API. It is designed specifically to handle metadata for up to 10,000+ workspaces efficiently.

Even I use the Scanner APIs to scan our entire tenant, since we also have a large tenant with more than 17k reports and more than 10k datasets.

Microsoft calls this Metadata Scanning. Refer to the link - https://learn.microsoft.com/en-us/fabric/governance/metadata-scanning-overview for detailed steps, I have tried to give a brief oveview below - 

Unlike the standard "Loop and Fetch" method, the Scanner API is asynchronous and works in four stages:

1. Get Modified Workspaces: Call admin/workspaces/modified. This gives you a list of workspace IDs that have changed (or all of them if it's your first run).

2. Trigger the Scan: Call admin/workspaces/getInfo. You post a list of Workspace IDs (in batches of 100). This returns a ScanID.

3. Check Status: Poll admin/workspaces/scanStatus/{ScanID} until it says "Succeeded."

4. Get the Result: Call admin/workspaces/scanResult/{ScanID}. This returns a massive JSON containing everything: datasets, tables, and—crucially for you—the Datasource details.

I would also recommend referring to Rui Ramano's awesome Power BI Monitor Solution that makes use of Scanner APIs for Admin Monitoring - https://github.com/RuiRomano/pbimonitor

Let me know if this helps or if you need anything else!

Thanks & Regards,

Bipin