Protecting API keys and Credentials

Hi,

I have not seen a single tidy answer to this one, so let me frame the options first. Two questions shape the right choice. Where does the key get used: a notebook, a pipeline, or Dataflow Gen2? Who else holds Contributor or higher on the shared workspace?

The core problem: in a shared workspace, anyone with Contributor, Member, or Admin access is able to open and run your notebooks, read Lakehouse files, and view environment settings. Any secret stored inside an item in plain form is readable by those people. Fabric has no built-in secret store for notebooks apart from Azure Key Vault. Without Key Vault, the goal shifts. Keep the secret out of item definitions, and control who uses the credential.

Options, strongest first

1. Fabric connections for pipelines and Dataflow Gen2 (durable fix). Go to Settings gear > Manage connections and gateways > New > Cloud. Enter the endpoint and the key under the matching authentication type. The credential stays encrypted in the connection and never appears in the pipeline definition. Under Manage users, grant access only to the people who need to run the item. In the pipeline, reference the connection from a Web or Copy activity.
2. Fabric connection inside a notebook (preview, workaround). In the notebook, open the Connections pane > Add connection, or bind an existing connection. Read the credential with:
   cred = notebookutils.connections.getCredential(connection_id)
   Account key and token authentication are supported for cloud sources. One caution: the value lands in notebook memory at runtime. Anyone able to edit and run a notebook bound to the connection is able to print the value. This reduces exposure. This is not a vault. Preview status also means no production support commitment.
3. Identity instead of keys. Where the target system supports Entra ID, drop the key entirely. Workspace identity covers trusted workspace access to firewall-enabled ADLS Gen2 and several connection types. A service principal still needs a secret, so this route helps only with sources supporting identity-based access.
4. Isolate the credential (durable structural fix). Move the ingestion notebook or pipeline into a separate workspace with two or three members. Land the output in a Lakehouse there and expose the tables to the shared workspace through a OneLake shortcut. The wider team gets the data. Nobody outside the small group touches the item holding the credential.

What to avoid

Hardcoding keys in notebook cells, config files in Lakehouse Files, Variable libraries, environment Spark properties, and pipeline parameters. All of these are visible to workspace members. With Git integration enabled, notebook code also lands in your repository history.

My honest view: options 1 and 4 reduce risk to an acceptable level for most teams. None of the four replaces Key Vault. A request for one vault with the Key Vault Secrets User role on your account is a small ask. Standard tier pricing runs around USD 0.03 per 10,000 operations, so cost rarely blocks approval. Frame the request around audit logs and secret rotation, since those are the points security teams weigh. Keep one limit in mind: notebookutils.credentials.getSecret runs under the identity executing the notebook, so scheduled runs use the notebook owner's access.

Next step: reply with the item type calling the API and the auth method the API expects (API key header, OAuth client credentials, or basic). I will outline the exact setup from there.

References
Fabric connection with notebook: https://learn.microsoft.com/fabric/data-engineering/fabric-connection-with-notebook
NotebookUtils for Fabric: https://learn.microsoft.com/en-us/fabric/data-engineering/notebook-utilities
Workspace identity authentication: https://learn.microsoft.com/en-us/fabric/security/workspace-identity-authenticate

Thank you!
Proud to be a Super User!
🏷️ Need more help?
✅ Don’t forget to Accept as Solution if this guidance worked for you.
❤️Your Like motivates me to keep helping