Forum Discussion

mhani's avatar
mhani
Frequent Visitor
3 years ago
Solved

Limit on get request (Admin - Datasets GetDatasourcesAsAdmin)

Hi,   I recently started to discover Power BI Admin APIs. And I wanted to bring in the GetDatasources, to get all the data sources for the datasets In my tenant. I do it using Python.   Although ...
  • Bipin-Lala's avatar
    5 months ago

    With 11,000 datasets, you have definitely moved past the "standard" Admin APIs limit.

     

    The  GetDatasourcesAsAdmin explicitly states a limit of 300 requests per hour.

     

    The Solution: The Scanner API (Metadata Scanning)

    For a tenant of your size, you should switch to the Scanner API. It is designed specifically to handle metadata for up to 10,000+ workspaces efficiently.

     

    Even I use the Scanner APIs to scan our entire tenant, since we also have a large tenant with more than 17k reports and more than 10k datasets.

     

    Microsoft calls this Metadata Scanning. Refer to the link - https://learn.microsoft.com/en-us/fabric/governance/metadata-scanning-overview for detailed steps, I have tried to give a brief oveview below - 

     

    Unlike the standard "Loop and Fetch" method, the Scanner API is asynchronous and works in four stages:

     

    1. Get Modified Workspaces: Call admin/workspaces/modified. This gives you a list of workspace IDs that have changed (or all of them if it's your first run).

    2. Trigger the Scan: Call admin/workspaces/getInfo. You post a list of Workspace IDs (in batches of 100). This returns a ScanID.

    3. Check Status: Poll admin/workspaces/scanStatus/{ScanID} until it says "Succeeded."

    4. Get the Result: Call admin/workspaces/scanResult/{ScanID}. This returns a massive JSON containing everything: datasets, tables, and—crucially for you—the Datasource details.

    I would also recommend referring to Rui Ramano's awesome Power BI Monitor Solution that makes use of Scanner APIs for Admin Monitoring - https://github.com/RuiRomano/pbimonitor

     

    Let me know if this helps or if you need anything else!

     

    Thanks & Regards,

    Bipin