Forum Discussion
Environment File in Notebook
- 1 year ago
Hi Anonymous , thank you for reaching out to the Microsoft Fabric Community Forum.
Microsoft Fabric does not natively support attaching .env files to notebooks. However, you can manage environment variables in different ways:- Fabric provides NotebookUtils (mssparkutils), which allows you to store and retrieve environment variables but this works only for variables already configured in Fabric.:
from notebookutils import mssparkutils
# Get an environment variable
value = mssparkutils.env.getVariable("MY_ENV_VAR")
print(value)
- If you need to customize values per environment (Dev, Test, Prod), you can use Deployment Rules but this is not a direct .env file solution.:
- Navigate to Deployment Pipelines.
- Create a Deployment Rule to set specific values.
- This allows modifying Lakehouse paths, parameters, etc. dynamically.
- Store a .env File in a Lakehouse and Read It in the Notebook. This works for custom environment variables but less secure if the .env file contains sensitive data:
import os# Read .env file from Lakehouse Files
file_path = "/lakehouse/default/Files/config/.env"
with open(file_path, "r") as f:
for line in f:
key, value = line.strip().split("=")
os.environ[key] = value
# Access an environment variable
print(os.getenv("MY_CUSTOM_VAR"))
- If your .env file contains API keys, credentials, or secrets, store them in Azure Key Vault and retrieve them in your notebook. This way you can have Secure credentials & secrets but Requires Azure Key Vault setup
from notebookutils import mssparkutils
# Get a secret from Azure Key Vault
secret_value = mssparkutils.credentials.getSecret("my-key-vault", "MY_SECRET_KEY")
print(secret_value)
If this helps, please consider marking it 'Accept as Solution' so others with similar queries may find it more easily. If not, please share the details.
Thank you. - 1 year ago
Hi Anonymous , thank you for reaching out to the Microsoft Fabric Community Forum.
To attach an ENV file to notebooks when migrating through both deployment pipeline and DevOps, and to re-tag different environments in the notebook, you can follow these steps:
- Ensure your ENV file is included in your repository. This file should contain the necessary environment variables. In your deployment pipeline, add a step to copy the ENV file to the appropriate location in your notebook's directory. This can be done using a script or a task in your pipeline configuration.
- To re-tag different environments in the notebook, you can use deployment rules to specify environment-specific configurations. For example, you can define different configurations for development, staging, and production environments. In your deployment pipeline, use environment variables or parameters to dynamically set the environment tags. This can be done by modifying the notebook's metadata or configuration files during the deployment process.
- In your deployment rules, specify the environment-specific configurations.
- In your DevOps pipeline configuration, add steps to handle the ENV file and set environment variables. This can be done using tasks or scripts in your pipeline YAML file.
If this helps, please consider marking it 'Accept as Solution' so others with similar queries may find it more easily. If not, please share the details.
Thank you.
Hi Anonymous
Microsoft Fabric does not have native support for attaching .env files directly to notebooks when migrating through deployment pipelines or DevOps. However, there are a few approaches you can consider for managing environment variables and secrets in Fabric notebooks:
Use NotebookUtils: Fabric provides NotebookUtils for working with environment variables. You can use the following code to access environment variables:
from notebookutils import mssparkutils
# Get an environment variable
value = mssparkutils.env.getVariable("VARIABLE_NAME")
Deployment Rules: When using deployment pipelines, you can create deployment rules to parameterize certain values, such as the default lakehouse, for each notebook instance. This allows you to change specific configurations when deploying across different environments.
Environment Resources: Fabric notebooks support an Environment Resources folder, which is a shared repository for collaboration across multiple notebooks. You could potentially store configuration files here, although it’s not recommended for sensitive information.
please accept this solution and give kudos if this is helpful.
thanks
Nilendra
- Anonymous1 year agoNot applicable
Hi,
I am looking to call fabric env files which has built in libraries and allows public/custom libraries.