fabric-cicd library (Python) - TokenError with DefaultAzureCredential despite explicit bearer token

[Shiva3]

Service: Microsoft Fabric (or Azure Data Factory if they redirect, but start with Fabric)

Problem Type: Development and Deployment / API and Connectivity

Problem Details:

We are experiencing a persistent TokenError when using the fabric-cicd Python library (version 0.1.19) in our Azure DevOps CI/CD pipeline to deploy Microsoft Fabric Data Pipelines. The error indicates DefaultAzureCredential is failing to acquire an AAD token, even though we are explicitly providing a bearer token obtained via MSAL.

Issue Description:

Our CI/CD pipeline consists of two primary Python scripts:

1. auth-api.py: Acquires a bearer token for an Entra ID service account (username/password flow, MFA disabled) using msal. This token acquisition is consistently successful, and the token is passed as a secret pipeline variable FABRIC_TOKEN to subsequent steps. This specific authentication method is used due to a known limitation in Fabric REST API for Data Pipelines which does not support authentication with Service Principal.

2. deploy.py: Uses the fabric-cicd library to deploy Data Pipelines and Notebooks to a Microsoft Fabric workspace.

The deploy.py script consistently fails at the point of FabricWorkspace initialization (or shortly thereafter during an internal operation) with the following traceback:

    self.endpoint = FabricEndpoint(
                    ^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.12.10\x64\Lib\site-packages\fabric_cicd\_common\_fabric_endpoint.py", line 40, in __init__
    self._refresh_token()
  File "C:\hostedtoolcache\windows\Python\3.12.10\x64\Lib\site-packages\fabric_cicd\_common\_fabric_endpoint.py", line 118, in _refresh_token
    raise TokenError(msg, logger) from e
fabric_cicd._common._exceptions.TokenError: Failed to acquire AAD token. DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
	EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
    ... (full list of DefaultAzureCredential failures as seen in the pipeline logs) ...

Observations:

• The auth-api.py script successfully acquires the AAD token and logs "Successfully acquired access token." This has been verified through pipeline logs.

• The FABRIC_TOKEN pipeline variable is correctly set by auth-api.py and is confirmed to be available to the deploy.py step as an environment variable (FABRIC_TOKEN or FABRIC_CLIENT_TOKEN in various attempts), verified with a diagnostic PowerShell step that echoes the masked variable.

• The error points specifically to fabric_cicd._common._fabric_endpoint.py and its _refresh_token() method, which seems to unconditionally attempt to acquire a token using DefaultAzureCredential.

Troubleshooting Steps Already Taken:

1. Verified Azure DevOps Pipeline Variables: Confirmed TENANT_ID, CLIENT_ID, USERNAME, PASSWORD, FABRIC_WORKSPACE_ID are correctly set as secret pipeline variables.

2. Verified Entra ID Service Account Permissions: Confirmed the service account is active, password is correct, MFA is disabled, and it has "Admin" or "Contributor" permissions within the target Fabric workspace. No Conditional Access Policies are known to be blocking access.

3. Attempted Direct Token Injection in deploy.py (via token parameter): Modified deploy.py to pass the token directly to the FabricWorkspace constructor (FabricWorkspace(..., token=token)). This did not prevent the DefaultAzureCredential error.

4. Attempted Environment Variable FABRIC_CLIENT_TOKEN: Modified deployment-pipeline.yaml to expose the acquired token as FABRIC_CLIENT_TOKEN: $(FABRIC_TOKEN) to the deploy.py step, hoping fabric-cicd would pick it up implicitly. This also did not prevent the error.

5. Attempted Direct TokenAuthentication import: Tried directly setting workspace._endpoint._authentication = TokenAuthentication(token), but this resulted in ModuleNotFoundError.

6. Verified fabric-cicd version: Using fabric-cicd==0.1.19.

Conclusion of Troubleshooting:

It appears the fabric-cicd library's FabricEndpoint component, in version 0.1.19, has an internal mechanism that attempts to acquire a token via DefaultAzureCredential during initialization, regardless of whether a bearer token has been provided through standard means (token parameter to constructor or set_bearer_token() method). This behavior is blocking our CI/CD pipeline, as DefaultAzureCredential methods are not suitable for our pipeline's authentication context (Entra ID service account with username/password due to Data Pipeline API limitations).

Request:

We request assistance in understanding how to correctly pass a pre-acquired bearer token to fabric-cicd (version 0.1.19) to prevent it from attempting to use DefaultAzureCredential, or confirmation if this is a known bug in this version of the library. If this is a bug, please provide guidance on a fix or a workaround.

[burakkaragoz]

Hi @Shiva3 ,

Thank you for providing such a thorough description and all your troubleshooting steps—it really helps in narrowing down the potential root cause.

Based on your explanation, it seems the main issue is that the fabric-cicd Python library (v0.1.19) fails to use a pre-acquired bearer token passed from your CI/CD pipeline, and instead, it defaults to DefaultAzureCredential (which then fails due to missing environment variables).

A few thoughts and suggestions:

1. Library Limitation (v0.1.19):
As you mentioned, in v0.1.19, the library’s internal credential chain appears to prioritize DefaultAzureCredential over any explicitly provided bearer token, unless the bearer token is passed in a very specific way (such as an environment variable or a particular function parameter). This is a known issue in some Azure SDKs/libraries where the credential flow cannot be easily overridden. It’s possible that this is either a bug or a limitation in this version of fabric-cicd.

2. Possible Workarounds:

• Environment Variable Injection:
  Try explicitly setting the AZURE_ACCESS_TOKEN or similar environment variable (if supported by the library) right before the deploy/apply steps in your pipeline, so the library picks it up as part of its credential chain.
• Monkey-Patching or Custom Auth:
  If the library allows, try manually injecting the token into the authentication header or overriding the credential resolution inside your deployment script. Sometimes, patching the HTTP client or session used by the library allows you to force usage of your token.
• Check for Updates:
  Look for newer releases of the fabric-cicd library or related GitHub issues/PRs. This is a common enough scenario that the maintainers may have addressed it or provided a workaround in a newer version.
• Log Level Debugging:
  Run your deployment with full debug logging enabled to see exactly where and why the credential fallback happens. This might reveal if there’s a code path or config you can override.

3. Additional Checks:

• Confirm that the bearer token you’re passing is valid for the specific scope required by the Fabric API (sometimes, tokens with the wrong audience/scope will be ignored, causing the library to fall back).
• Double-check that there is no typo or mismatch in the way the token is injected or referenced in your deployment environment.

4. Contacting the Maintainers:
Since you’ve already validated most of the usual suspects, if none of the above workarounds help, I would recommend raising this as a GitHub issue in the repository for the fabric-cicd library (or wherever it’s maintained). Include your findings and steps above—it’s very likely the maintainers will want to know about this, and may provide a hotfix or further guidance.

Summary:

• This looks like a limitation or bug in how fabric-cicd (v0.1.19) prioritizes credentials.
• Try injecting the token as an environment variable or monkey-patching if possible.
• Check for library updates or open an issue with the maintainers if no workaround is effective.

If you do find a workaround or hear back from the maintainers, please share it here—it will help others facing the same challenge!

Good luck, and let us know how it goes.