Forum Discussion

Ms_Techzill's avatar
Ms_Techzill
Regular Visitor
8 months ago
Solved

How to Access Azure Key Vault Secrets in Fabric Notebook Using Managed Identity

I am trying to retrieve secrets from Azure Key Vault in a Microsoft Fabric Notebook using:

 

from notebookutils import mssparkutils

# Use the reference alias and secret names
CONNECTION_STRING = mssparkutils.credentials.getSecret("emailcommunicationkey", "ACS-CONNECTION-STRING")
SENDER = mssparkutils.credentials.getSecret("emailcommunicationkey", "SENDER-EMAIL")

print(CONNECTION_STRING[:15] + "*****")

However, I keep getting this error:

Py4JJavaError Traceback (most recent call last) Cell In[16], line 4 1 from notebookutils import mssparkutils 3 # Use the reference alias and secret names ----> 4 CONNECTION_STRING = mssparkutils.credentials.getSecret("emailcommunicationkey", "ACS-CONNECTION-STRING") 5 SENDER = mssparkutils.credentials.getSecret("emailcommunicationkey", "SENDER-EMAIL") 7 print(CONNECTION_STRING[:15] + "*****") File ~/cluster-env/trident_env/lib/python3.11/site-packages/notebookutils/mssparkutils/credentials.py:27, in getSecret(akvName, secret, linkedService) 25 def getSecret(akvName, secret, linkedService=''😞 26 if linkedService == '': ---> 27 return creds.getSecret(akvName, secret) 28 else: 29 return creds.getSecret(akvName, secret, linkedService)
data science
Notebook

6 Replies

  • v-hashadapu's avatar
    v-hashadapu
    Community Support
    8 months ago

    Hi Ms_Techzill , Thank you for reaching out to the Microsoft Community Forum.

     

    Mauro89  was right about the core issue, the notebook runs under the user who executes it and that user must have Key Vault secret-read permissions. If they don’t, getSecret() throws the Py4JJavaError you’re seeing.

    Your call is also failing because mssparkutils.credentials.getSecret() expects either a properly created linked service alias or the full Key Vault URL. Most of the time the alias doesn’t resolve, so call the vault directly using its endpoint and make sure the notebook runner has a Key Vault secrets role. Once you pass the URL and the user has permissions, the secret loads cleanly.

     

    Introduction to Microsoft Spark utilities - Azure Synapse Analytics | Microsoft Learn

    Grant permission to applications to access an Azure key vault using Azure RBAC | Microsoft Learn

    NotebookUtils (former MSSparkUtils) for Fabric - Microsoft Fabric | Microsoft Learn

    Microsoft Spark Utilities (MSSparkUtils) for Fabric - Microsoft Fabric | Microsoft Learn

  • Ms_Techzill's avatar
    Ms_Techzill
    Regular Visitor
    8 months ago

    Mauro89 , thank you for your response.  I have watched the video and when I try to run the api_key from my part it shows ["redacted"] just as that of the video. However, when I included it in my code I have the below
    code



    from notebookutils import mssparkutils
    import pandas as pd
    import io
    import base64
    from azure.communication.email import EmailClient
    from azure.keyvault.secrets import SecretClient



    # ====== Load secrets from Azure Key Vault ======

    #  Replace "kv-emailcomm" with your actual Key Vault connection name in Fabric
    CONNECTION_STRING = mssparkutils.credentials.getSecret("emailcommunicationkey", "ACS-CONNECTION-STRING")
    SENDER = mssparkutils.credentials.getSecret("emailcommunicationkey", "SENDER-EMAIL")



    # Initialize EmailClient
    client = EmailClient.from_connection_string(CONNECTION_STRING)

    # ====== Load churn data from Spark ======
    df = spark.read.load(
        ""
    )
    churn_alert = df.toPandas()

    # ====== Filter for test email (optional) ======
    # If you want to send all data, skip this filter
    # If you want only rows for girl, uncomment below:
    # churn_alert = churn_alert[churn_alert['email'] == ' ']

    # Create CSV attachment
    csv_buffer = io.StringIO()
    churn_alert.to_csv(csv_buffer, index=False)
    csv_base64 = base64.b64encode(csv_buffer.getvalue().encode()).decode()

    # Email Content
    plain_text_content = (
        f"Hello friend,\n\n"
        "Hope you’re doing well. This is a test\n\n"
        "Best regards,\ntesting team"
    )

    # Prepare message
    message = {
        "senderAddress": SENDER,
        "recipients": {"to": [{"address": " "}]},
        "content": {
            "subject": "Churn Alert: Your Customers at Risk",
            "plainText": plain_text_content
        },
        "attachments": [
            {
                "name": "churn_report.csv",
                "contentType": "text/csv",
                "contentInBase64": csv_base64
            }
        ]
    }

    # Send email
    try:
        poller = client.begin_send(message)
        result = poller.result()
        print(f"✅ Email sent to " " | Message ID: {result['id']}")
    except Exception as e:
        print(f"❌ Failed to send email: {e}")


    this is the output
    --------------------------------------------------------------------------- Py4JJavaError Traceback (most recent call last) Cell In[49], line 14 7 from azure.keyvault.secrets import SecretClient 11 # ====== Load secrets from Azure Key Vault ====== 12 13 # Replace "kv-emailcomm" with your actual Key Vault connection name in Fabric ---> 14 CONNECTION_STRING = mssparkutils.credentials.getSecret("emailcommunicationkey", "ACS-CONNECTION-STRING") 15 SENDER = mssparkutils.credentials.getSecret("emailcommunicationkey", "SENDER-EMAIL") 19 # Initialize EmailClient File ~/cluster-env/trident_env/lib/python3.11/site-packages/notebookutils/mssparkutils/credentials.py:27, in getSecret(akvName, secret, linkedService) 25 def getSecret(akvName, secret, linkedService=''😞 26 if linkedService == '': ---> 27 return creds.getSecret(akvName, secret) 28 else: 29 return creds.getSecret(akvName, secret, linkedService) File ~/cluster-env/trident_env/lib/python3.11/site-packages/py4j/java_gateway.py:1322, in JavaMember.__call__(self, *args) 1316 command = proto.CALL_COMMAND_NAME +\ 1317 self.command_header +\ 1318 args_command +\ 1319 proto.END_COMMAND_PART 1321 answer = self.gateway_client.send_command(command) -> 1322 return_value = get_return_value( 1323 answer, self.gateway_client, self.target_id, self.name) 1325 for temp_arg in temp_args: 1326 if hasattr(temp_arg, "_detach"😞

    • Mauro89's avatar
      Mauro89
      Super User
      8 months ago

      Hi Ms_Techzill,

       

      I guess your Key Vault URI is not correct. Thats where you wrote "emailcommunicationkey", so the first parameter in the "get_secret" function. 

      Here acutally your "Vault URI" needs to be passed. This you can find if you open you Key Vault and go to the overview (marked in red):

       

      Then the second parameter as being the acutal name of the secret.

       

      This should work then as expected.

       

      Best regards!

       

      • Ms_Techzill's avatar
        Ms_Techzill
        Regular Visitor
        8 months ago

        Hello Mauro89 , thanks so much. I eventually changed it to the vault uri and it worked. Thank you

  • Ms_Techzill's avatar
    Ms_Techzill
    Regular Visitor
    8 months ago

    --------------------------------------------------------------------------- Py4JJavaError Traceback (most recent call last) Cell In[49], line 14 7 from azure.keyvault.secrets import SecretClient 11 # ====== Load secrets from Azure Key Vault ====== 12 13 # Replace "kv-emailcomm" with your actual Key Vault connection name in Fabric ---> 14 CONNECTION_STRING = mssparkutils.credentials.getSecret("emailcommunicationkey", "ACS-CONNECTION-STRING") 15 SENDER = mssparkutils.credentials.getSecret("emailcommunicationkey", "SENDER-EMAIL") 19 # Initialize EmailClient File ~/cluster-env/trident_env/lib/python3.11/site-packages/notebookutils/mssparkutils/credentials.py:27, in getSecret(akvName, secret, linkedService) 25 def getSecret(akvName, secret, linkedService=''): 26 if linkedService == '': ---> 27 return creds.getSecret(akvName, secret) 28 else: 29 return creds.getSecret(akvName, secret, linkedService)