Service Principals running specific Notebooks from Pipelines Fail

Hi macrohardy,

This issue is likely due to the version of semantic-link (sempy.fabric) available in the default Fabric runtime.

From the error and test results, it doesn't seem related to workspace permissions or Service Principal configuration, since:

• the notebook runs interactively without issues
• the same Service Principal works for other notebooks and lakehouse tasks
• direct REST API calls are successful
• the failure only happens with fabric.list_workspaces()

The key detail is the internal call to:

https://api.powerbi.com/powerbi/globalservice/v201606/clusterdetails

Older semantic-link versions don't always handle Service Principal authentication correctly in Pipelines, especially with internal Power BI metadata APIs.

Using direct REST API calls, as described above, is currently the most reliable workaround because it avoids the authentication flow used by sempy.fabric.

A newer semantic-link version seems to fix this. Upgrading the package in a custom Fabric environment has been reported to resolve the problem.

Right now, you can either:

1. Use direct REST API calls as a workaround
2. Create a custom Fabric environment with an updated semantic-link package
3. Wait for Microsoft to update the default Fabric runtime package

This appears to be a runtime or library limitation, not a notebook configuration issue.

The following documents are provided for your reference:
Enable service principal authentication for admin APIs - Microsoft Fabric | Microsoft Learn
sempy.fabric package | Microsoft Learn
Service Principal support for Semantic Link - Microsoft Fabric | Microsoft Learn

Thank you.