Credentials used for Azure keyvault in notebooks

[ag_b]

I have a notebook that utilizes notebookutils.credentials.getSecret to fetch secrets from azure keyvault. As far as I understand the credentials of the person running the notebook are used to fetch the secrets, is this still the case when dealing with scheduled notebooks?

From this post https://community.fabric.microsoft.com/t5/General-Discussion/Fabric-Notebooks-RunAs-User-and-Permiss... I understand that any scheduled notebooks and pipelines are tied to the tenant and workspace, and not to any individual user. Then which credentials are used to fetch the secrets? Will the scheduled notebooks fetching secrets stop working if my user no longer has access to the keyvault?

Thanks in advance for any help!

[v-zhengdxu-msft]

Hi @ag_b 

When you set up a scheduled job or a pipeline, you would typically configure a service principal or managed identity that has the necessary permissions to access the Azure Key Vault. This identity is different from your personal user credentials. For scheduled notebooks, the service principal or managed identity's credentials are used to authenticate and fetch secrets from the Azure Key Vault, not the credentials of the person who originally ran the notebook.

If your user account loses access to the Azure Key Vault, it does not necessarily mean that the scheduled notebooks will stop working, as long as the service principal or managed identity used by the job still has the required permissions.

If the service principal or managed identity's access to the Key Vault is revoked, then the scheduled notebook will no longer be able to fetch secrets from the Key Vault, and the job may fail.

In summary, the credentials used to fetch secrets in scheduled notebooks are those of the service principal or managed identity configured for the job, not the individual user's credentials. As long as this identity has the correct permissions, your scheduled notebooks should continue to function even if your personal access is revoked.

Best Regards

Zhengdong Xu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[ag_b]

Hi @v-zhengdxu-msft , thanks for the reply!

I'm looking into configuring a service principal or managed identity whose credentials I can use in the notebook. I am however having trouble finding straigtforward documentation on how to achieve this.

When I look up using a service principal whose credentials I can use to retrieve secrets from the key vault I, from my understanding, create the service principal by creating an app that has access to the key vault. I then run into the problem of needing to store the apps credentials someplace safe other than the key vault.

When I look up using managed identity the closest solution applicable to MS fabric is using workspace identity, this is however not yet ready to be used for connecting to resources that support Entra id, meaning I can't use it to connect to the key vault.

What am i missing in how this should be correctly configured?

Best regards

ag_b

[v-zhengdxu-msft]

Hi @ag_b 

Sorry for the late reply.

Configuring a service principal or managed identity for use in a notebook can indeed be a bit tricky. Here I find some documents that can help:

Using a Managed Identity with Databricks to Run Notebooks Through a Web App (bart.je)

Use service principals & managed identities - Azure DevOps | Microsoft Learn

Set up and use Azure managed identities authentication for Azure Databricks automation - Azure Datab...

Manage service principals - Azure Databricks | Microsoft Learn

Best Regards

Zhengdong Xu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[delks]

Hi @ag_b @v-zhengdxu-msft,

Did you manage to find any breakthrough related to this?

I am also trying to do something similar to this where I am trying to extract secrets from Azure Key Vault but I do not want to store secrets in the code.

Kind regards

Delekson