Forum Discussion
Use KV secret value in pipeline code
I am trying to use KV secrets in notebook using set variable activity output.
The variable value( sceret value) shows up in notebook logs if Toggle Param cell option is used and dbutils.widget library is not found in fabric to read the variable activity output directly in notebook code.
please recommend a way to read secrets.
I dont want to call KV API from notebook directly using the context of the user running the code and If i use service principal , where should i store cleint secret of the SP when making the call from the notebook. I dont want to expose client secret of SP in notebook.
Hi SBSlalom ,
Thank you for reaching out to the Microsoft Community Forum.
To access Azure Key Vault secrets in Microsoft Fabric notebooks without exposing sensitive credentials like the client secret of a service principal.
Solution: Use mssparkutils.credentials.getSecret with Service Principal.
Microsoft Fabric notebooks support accessing Azure Key Vault secrets using the built-in utility.
secret_value = mssparkutils.credentials.getSecret('https://<keyvault-name>.vault.azure.net/', 'secret-name')
How to Securely Use a Service Principal
Solution: To avoid exposing the client secret in the notebook, Please follow below steps:
1. Create a Service Principal in Azure. Register an app in Azure AD. Generate a client secret. Assign the following roles to the SP, Key Vault Secrets User to read secrets and Key Vault Contributor, if you need to manage secrets.
2. Instead of hardcoding the client secret in the notebook, Store the client ID, tenant ID, and client secret in Azure Key Vault. Use a managed identity or pipeline parameterization to retrieve these credentials securely.
3. In Fabric pipeline, use a Set Variable activity to retrieve the SP credentials from Key Vault. Pass these variables as base parameters to the notebook.
In the notebook, access them using with sample below code.
import sys
client_id = sys.argv[1]
tenant_id = sys.argv[2]
client_secret = sys.argv[3]
Note: Check the notebook parameter cell is enabled and the pipeline passes the variables correctly.Please refer below links.
Solved: Use Keyvault secret in copy activity via notebook ... - Microsoft Fabric Community
Solved: Re: Accessing Azure KeyVault secret for Fabric Cop... - Microsoft Fabric Community
Solved: Handling secrets in Fabric Notebooks - Microsoft Fabric Community
Solved: Access Key-vault in notebooks - Microsoft Fabric Community
Solved: Accessing Azure KeyVault secret for Fabric CopyDat... - Microsoft Fabric Community
How to use notebooks - Microsoft Fabric | Microsoft Learn
I hope this information helps. Please do let us know if you have any further queries.
Regards,
Dinesh
4 Replies
v-dineshyaCommunity Support
Hi SBSlalom ,
Thank you for reaching out to the Microsoft Community Forum.
To access Azure Key Vault secrets in Microsoft Fabric notebooks without exposing sensitive credentials like the client secret of a service principal.
Solution: Use mssparkutils.credentials.getSecret with Service Principal.
Microsoft Fabric notebooks support accessing Azure Key Vault secrets using the built-in utility.
secret_value = mssparkutils.credentials.getSecret('https://<keyvault-name>.vault.azure.net/', 'secret-name')
How to Securely Use a Service Principal
Solution: To avoid exposing the client secret in the notebook, Please follow below steps:
1. Create a Service Principal in Azure. Register an app in Azure AD. Generate a client secret. Assign the following roles to the SP, Key Vault Secrets User to read secrets and Key Vault Contributor, if you need to manage secrets.
2. Instead of hardcoding the client secret in the notebook, Store the client ID, tenant ID, and client secret in Azure Key Vault. Use a managed identity or pipeline parameterization to retrieve these credentials securely.
3. In Fabric pipeline, use a Set Variable activity to retrieve the SP credentials from Key Vault. Pass these variables as base parameters to the notebook.
In the notebook, access them using with sample below code.
import sys
client_id = sys.argv[1]
tenant_id = sys.argv[2]
client_secret = sys.argv[3]
Note: Check the notebook parameter cell is enabled and the pipeline passes the variables correctly.Please refer below links.
Solved: Use Keyvault secret in copy activity via notebook ... - Microsoft Fabric Community
Solved: Re: Accessing Azure KeyVault secret for Fabric Cop... - Microsoft Fabric Community
Solved: Handling secrets in Fabric Notebooks - Microsoft Fabric Community
Solved: Access Key-vault in notebooks - Microsoft Fabric Community
Solved: Accessing Azure KeyVault secret for Fabric CopyDat... - Microsoft Fabric Community
How to use notebooks - Microsoft Fabric | Microsoft Learn
I hope this information helps. Please do let us know if you have any further queries.
Regards,
Dinesh
- v-dineshya
Hi SBSlalom ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. And, if you have any further query do let us know.
Regards,
Dinesh
- v-dineshya
Hi @SBSlalom ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. And, if you have any further query do let us know.
Regards,
Dinesh
Hi SBSlalom ,
May I check if this issue has been resolved? If not, Please feel free to contact us if you have any further questions.
Thank you
