Partial result returned from Power BI REST API Datasets - Get Dataset To Dataflows Links In Group

[kuraki]

Since last Thursday, this Power BI REST API suddenly stopped to work properly.

https://api.powerbi.com/v1.0/myorg/groups/{groupId}/datasets/upstreamDataflows

When calling this API using service principal in Power Query in a gen2 dataflow (non CI/CD), only part of the DatasetIDs in part our many workspaces returned only part of the upstream DataflowIDs (all are gen1 dataflows, not sure if it is coincident or not). It was working flawlessly all the time before, i.e. a full list of DatasetIDs in ALL workspaces, and returned ALL their corresponding upstream DataflowIDs from different workspaces (included both gen1 and gen2 non CI/CD dataflows).

However, when calling the above API using the "Try It" button here

Datasets - Get Dataset To Dataflows Links In Group - REST API (Power BI Power BI REST APIs) | Micros...

we still can receive the full expected result as described above.

FYI, I tried to call all other APIs as follows, using both service principal and “Try It” button, and has no issue at all.  So it seems unlikely a permission/token issue.  And we have not changed any permission in the Azure app, our workspaces, etc.

 https://api.powerbi.com/v1.0/myorg/gateways

https://api.powerbi.com/v1.0/myorg/gateways/{gatewayId}/datasources

https://api.powerbi.com/v1.0/myorg/groups

https://api.powerbi.com/v1.0/myorg/gateways

https://api.powerbi.com/v1.0/myorg/groups/{groupId}/datasets/{datasetId}/refreshes

https://api.powerbi.com/v1.0/myorg/groups/{groupId}/dataflows

https://api.powerbi.com/v1.0/myorg/groups/{groupId}/datasets/{datasetId}/datasources

and some more...

Please advise the possible cause and how to fix it.  Thanks.

[kuraki · Accepted solution]

@v-hashadapu @Jai-Rathinavel   FYI, the issue has gone by itself without changing anything and https://api.powerbi.com/v1.0/myorg/groups/{groupId}/datasets/upstreamDataflows is returning full list of datasets in all workspaces now.

Thanks both for your help.

[v-hashadapu]

Hi @kuraki , Thank you for the update. We're glad to hear the issue has been resolved.

If you have any more questions, please feel free to create a new post. We're always happy to help.

[kuraki]

@Jai-Rathinavel  @v-hashadapu Thanks for your reply.   We have a total of 120 datasets in 40 workspaces.

Try It!
All 120 datasets in 40 workspaces can be returned in different workspace/dataset related APIs I called.

Service Principal
The problem is only relaed to this API happened since last Thursday 

https://api.powerbi.com/v1.0/myorg/groups/{groupId}/datasets/upstreamDataflows

It returns only 2 datasets from 2 workspaces.  Before that, it can returns all 120 datasets in all 40 workspaces.

For other dataset related APIs like the following, they can return all 120 datasets in all 40 workspaces.

https://api.powerbi.com/v1.0/myorg/groups/{groupId}/datasets/{datasetId}/datasources

https://api.powerbi.com/v1.0/myorg/groups/{groupId}/datasets/{datasetId}/refreshes

FYI, I have been using service principal to call various APIs to auto grab info to build a power bi report to show the lineage view for all reports, datasets and dataflows, and it has been working for over a year, but the info about dataset to dataflow link suddenly broken  as https://api.powerbi.com/v1.0/myorg/groups/{groupId}/datasets/upstreamDataflows only returned partial result since last Thursday.   But the info returned by other APIs on workspaces, reports,  datasets, dataflows still match with our full lists.

Sorry for I'm relatively new to these APIs and doesn't know much technical about them...😅

[Jai-Rathinavel]

@kuraki 
1. Since the API is not working suddenly I would advice you to open a support ticket with the help of the platform admin, I have came across few sceanrios were the API which was working returning partial results
2. Can you check if the service principal has been added to all the workspaces ?

[kuraki]

@Jai-Rathinavel Thank for very much for your prompt reply.

1. Thanks.  Will raise a support ticket.

2. Yes, service principal is added to all workspaces.

[v-hashadapu]

Hi @kuraki , Thank you for reaching out to the Microsoft Community Forum.

@Jai-Rathinavel  is correct to ask for dataset counts, you also need to capture and compare the full requests and decoded tokens, check for pagination (nextLink), confirm the service principal is enabled in the Power BI admin and has the necessary workspace membership and token scopes and as a fallback, enumerate per-workspace/per-dataset upstream links to surface anything the aggregated call misses.

Datasets - Get Dataset To Dataflows Links In Group - REST API (Power BI Power BI REST APIs) | Micros...

Power BI REST APIs for embedded analytics and automation - Power BI REST API | Microsoft Learn

Datasets - Get Datasets - REST API (Power BI Power BI REST APIs) | Microsoft Learn

Dataflows - Get Upstream Dataflows In Group - REST API (Power BI Power BI REST APIs) | Microsoft Lea...

Thank you @Jai-Rathinavel  for your valuable response.

[Jai-Rathinavel]

@kuraki Can you please check the number of datasets you're seeing when using the Service Principal vs “Try It” method?