Forum Discussion
pass KeyVault values OUT OF Notebook to pipeline to consume - securely
Hi Maverikk ,
I completely understand the frustration here. What you are seeing with the notebook returning the value as REDACTED is the expected behavior in Fabric because the platform will not allow a secret retrieved inside a notebook to be passed back out to the pipeline. Anything Fabric detects as sensitive is automatically masked so that it never leaves the secure execution boundary. This means that even though the notebook can authenticate with Workspace Identity and obtain the token, there is no supported way to expose that token for a downstream activity to use. At the moment, Fabric does not provide an end to end pattern where a notebook retrieves a secret and then hands it back to a pipeline activity such as Copy. The only supported approach today is to have the pipeline itself retrieve the token through a connection that uses a service principal, because that is the identity type pipelines are currently able to authenticate with. I know that is not the workflow you were hoping for, but it is the secure and supported path with the capabilities that exist right now.
Thank you.
Super UserHi Maverikk Microsoft Fabric specifically prevent any secrets obtained in a Notebook to be leaked outside of the Notebook. You can read more about that technique here https://learn.microsoft.com/en-us/fabric/data-engineering/author-execute-notebook#secret-redaction.
This answer does not solve your questions, unfortunately, but this is not a problem with your solution but a current technical limitation in Fabric. Fabric allows you to create a Key Vault reference but it's not supported in data pipelines and it also does not support yet a private networking for Azure Key Vaults.
So, there is no generic recommended approach yet for your question. We maight be able to find a solution for your specific scenario, but you need to provide an explanation on why do you need to use a secret in a pipeline.
I personally always start from trying to avoid using secrets in the data pipeline in Fabric.
Hopefully that explains the situation even though does not answer your question. Feel free to continue this discussion but please explain your scenario.
Thankyou apturlov for taking the time to reply. I am trying to securely store my API Bearer Token inside KeyVault, and then access and use it inside a CopyActivity for an API call. The API call holds commercially sensitive data, so naturally I want to keep the "secret" under wraps. Perhaps you can suggest a way that you would approach this, maybe there is a better methodology than I am using? Thankyou for your time
