Forum Discussion
pass KeyVault values OUT OF Notebook to pipeline to consume - securely
Hi Maverikk ,
I completely understand the frustration here. What you are seeing with the notebook returning the value as REDACTED is the expected behavior in Fabric because the platform will not allow a secret retrieved inside a notebook to be passed back out to the pipeline. Anything Fabric detects as sensitive is automatically masked so that it never leaves the secure execution boundary. This means that even though the notebook can authenticate with Workspace Identity and obtain the token, there is no supported way to expose that token for a downstream activity to use. At the moment, Fabric does not provide an end to end pattern where a notebook retrieves a secret and then hands it back to a pipeline activity such as Copy. The only supported approach today is to have the pipeline itself retrieve the token through a connection that uses a service principal, because that is the identity type pipelines are currently able to authenticate with. I know that is not the workflow you were hoping for, but it is the secure and supported path with the capabilities that exist right now.
Thank you.
Super UserHi Maverikk
You should think about Notebooks because everybody in the world uses Linux System. Use Spark and bit of study..
Python --> Data Lake ( Aache Spark ) --> Delta Lake. It is just using Python, and it is open source.
Thanks BhaveshPatel , I am not sure you have answered my question tbh. I am saying that I AM using a Notebook and Python, but am struggling to pass the [REDACTED] OUT of the Notebook and back into a Fabric pipeline.
Hi Maverikk ,
You’re absolutely right to point out that the issue isn’t about using notebooks or Spark in general, but about how to securely pass a value retrieved from Key Vault out of a Fabric notebook so the pipeline can use it. The behavior you’re seeing where the value is replaced with [REDACTED] is expected, as Fabric intentionally masks sensitive information to prevent secrets from being exposed in pipeline logs or variables.
This means values obtained using Workspace Identity inside a notebook can’t be passed directly to a pipeline output. The secure and supported approach is to use the secret within the same notebook where it’s retrieved, or have each pipeline activity that needs it access Key Vault directly through a managed identity or linked service. That way, the secret never leaves a protected environment and your pipeline remains fully compliant with Fabric’s security model.
Best Regards,
Tejaswi.
Community SupportHi Maverikk ,
I wanted to follow up and see if you had a chance to review the information shared. If you have any further questions or need additional assistance, feel free to reach out.
Thank you.
thankyou Anonymous for your response and your input. Can I ask what would be the appropriate way inside Fabric for me to retrieve a Bearer Token for an API call (from Keyvault), and then use that Bearer Token for a Copy Activity [in a pipeline]? You have mentioned: "pipeline activity that needs it access Key Vault directly through a managed identity", I am not clear how this can be achieved with a copy activity. Ideally I would NOT be using a Notebook to retrieve the values as I am trying to reduce cu's, where not needed. Perhaps I am missing something? Thankyou for your time.
Hi Maverikk ,
That’s an excellent follow-up question, and your focus on secure handling and minimizing CU usage is well placed. In Fabric, it is essential to ensure that secrets and tokens remain within secure contexts and are not written to logs. To avoid using a notebook, I recommend configuring the pipeline to manage both Key Vault retrieval and API authorization directly. You can achieve this by adding a Web activity before the Copy Activity to obtain the token from your authentication endpoint or Key Vault using a managed identity.
The token output from the Web activity can then be referenced in the Copy Activity’s Authorization header, for example, using activity GetToken output.access_token. This method ensures the token is securely used at runtime and not exposed or stored in plain text. If your environment allows, grant the pipeline’s managed identity access to Key Vault secrets, enabling direct retrieval without a notebook. This approach maintains security, reduces CU costs, and aligns with Fabric’s best practices for secret management.
Thank you.
Hi Anonymous ,
I have another situation,
I have a master notebook that use RunMultiple notebook from notebookutils, this master notebook get secret token value from AZ Keyvault and then passing this token into child notebooks. But instead of REDACT this token value in the notebook snapshot, it showed completely all the token value. Is it an expected behavior or a known limitation for this situation. And are there another approaches to securely pass my token into child notebooks from master notebook?
Best Regards,
Khoa Nguyen.Hi lukadafoo ,
Thank you for providing the additional information and screenshot. The behavior you are observing is consistent with the security design in Fabric. When a secret or token is retrieved from Azure Key Vault within a Fabric notebook, the platform automatically redacts the value when it appears outside the notebook’s secure execution boundary. This prevents sensitive information from being exposed in outputs, logs, pipeline results, or notebook snapshots.
In your case, the master notebook retrieves the secret and passes it as a parameter to child notebooks using RunMultiple. When these parameters are included in notebook snapshots or pipeline outputs, Fabric may display the value rather than mask it, depending on how the snapshot mechanism records the parameter. This is due to the way parameters are stored for reproducibility and debugging, and it does not apply the same masking logic used for values returned from a notebook to a pipeline activity. As a result, secrets passed as notebook parameters may appear in snapshot views.
For enhanced security, it is recommended that each notebook or activity retrieves secrets directly from Azure Key Vault using the workspace or managed identity, rather than passing them as parameters. This ensures secret retrieval remains within the execution context and reduces the risk of exposure.
If multiple child notebooks require the same token, they should each call Azure Key Vault at runtime to obtain the secret. This approach aligns with Fabric’s security model and prevents sensitive values from appearing in snapshots or logs.
If you can share more details about how the child notebooks use the token, we can help suggest a secure workflow that meets your requirements.
Thank you.
