How to Set Custom Artifact Path for MLflow Experiment in Microsoft Fabric?

[Mikel317]

Hi,

I'm trying to create an MLflow experiment in Microsoft Fabric and store artifacts in my current workspace in specific folder in Workspace (workspace/Path/to/my/folder/Experiments). How to use relative(or absolute) path to this folder, so I can use it in create_experiment("experiment_name", artifact_location=path)?

Thank you for answer.

[v-tsaipranay]

Hi @Mikel317 ,

Thanks for reaching out to the Microsoft fabric community forum.

To create an MLflow experiment in Microsoft Fabric and store artifacts in a specific folder, use mlflow.create_experiment(), specifying either an absolute or relative artifact_location.

For local storage, use a path like "file:///absolute/path/to/folder/Experiments", ensuring proper permissions.

For cloud storage (e.g., Azure Blob Storage), use a valid URI such as "wasbs://<container>@<account>.blob.core.windows.net/path/to/folder/Experiments" with authentication.

After creation, set the active experiment using mlflow.set_experiment(experiment_id). Absolute paths are recommended for consistency.

For further details, refer to the below documentation:

https://learn.microsoft.com/en-us/fabric/data-science/machine-learning-experiment

If you need any further assistance, feel free to reach out.

If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.

Thank you. 

[Mikel317]

Hello,

I would like to store the data locally using notebook, but unfortunately, the path "file:///absolute/path/to/folder/Experiments" doesn't work. I can't seem to get the relative or absolute path right.

What is the correct format for specifying a relative (or absolute) path in this case? Every solution I’ve found so far creates the file inside the root workspace rather than the specified folder I want.

Thank you for your help!

[v-tsaipranay · Accepted solution]

Hi @Mikel317  ,

To store MLflow experiment artifacts in a specific folder within Microsoft Fabric using notebooks, the path "file:///absolute/path" will not work, as Microsoft Fabric uses Lakehouse storage instead of the local filesystem.

In Fabric, you need to specify the Lakehouse path where your files should be stored. Please use the following format:

import mlflow

# Set the artifact location within Lakehouse

artifact_location = '/lakehouse/default/Files/Path/to/your/folder/Experiments'

# Create MLflow experiment

experiment_id = mlflow.create_experiment("experiment_name", artifact_location=artifact_location)

# Set the active experiment

mlflow.set_experiment("experiment_name")

Ensure that the specified folder path already exists in your Lakehouse Files section before creating the MLflow experiment. In Microsoft Fabric notebooks, relative paths are not supported for storing artifacts. It is recommended to use absolute Lakehouse paths that start with /lakehouse/default/Files/ to correctly reference the desired folder location. This approach will ensure that the experiment artifacts are stored in the specified folder without defaulting to the root workspace.

If this helps, then please Accept it as a solution and dropping a "Kudos" so other members can find it more easily.

Thank you.

[v-tsaipranay]

Hi @Mikel317 ,

I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.

Thank you.