topic Re: detect runtime in Data Engineering

detect runtime

smpa01 — Wed, 12 Mar 2025 14:33:36 GMT

Is there any way for a code to return the environment in notebook;

//pseudocode print(os.environ) // detects either Fabric or Databricks runtime

thank you in advance

Re: detect runtime

lbendlin — Thu, 13 Mar 2025 01:30:07 GMT

This might be useful

Gathering useful notebook and environment details at runtime

Re: detect runtime

V-yubandi-msft — Thu, 13 Mar 2025 10:06:27 GMT

Hi @smpa01 ,

You can utilize ‘notebookutils.runtime.context’ in Microsoft Fabric to obtain information about the current session, such as the notebook name, workspace details, and lakehouse information.

notebookutils.runtime.context

Helpful Reference:

NotebookUtils (former MSSparkUtils) for Fabric - Microsoft Fabric | Microsoft Learn

I hope this information is helpful to you.

Re: detect runtime

smpa01 — Thu, 13 Mar 2025 12:59:12 GMT

I need a code that can run both in databricks and fabric notebooks

Re: detect runtime

V-yubandi-msft — Thu, 20 Mar 2025 11:42:30 GMT

Hi @smpa01 ,

Detecting the environment in a notebook whether it's running on Microsoft Fabric or Databricks requires different methods since both platforms handle runtime details differently. Unfortunately, there's no universal way to do this across both environments.

Each platform has its own way of providing system information:

1. Microsoft Fabric Notebooks - Microsoft Fabric relies on notebookutils.runtime.context to fetch session, notebook, and workspace details. However, this API is specific to Fabric and isn't available in Databricks.

2. Databricks provides the DATABRICKS_RUNTIME_VERSION environment variable, which shows the runtime version. This variable is specific to Databricks and is not available in Microsoft Fabric.

FYI

Re: detect runtime

V-yubandi-msft — Mon, 24 Mar 2025 11:24:43 GMT

Hi @smpa01 ,

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank You.

Re: detect runtime

V-yubandi-msft — Thu, 27 Mar 2025 18:46:35 GMT

Hi @smpa01 ,

Has your issue been resolved, or do you require any further information? Your feedback is valuable to us. If the solution was effective, please mark it as 'Accepted Solution' to assist other community members experiencing the same issue.

Re: detect runtime

smpa01 — Sun, 30 Mar 2025 19:38:38 GMT

The problem requires a code that runs across platforms

sys.modules

and returns unique values related to 'fabric' or 'databricks' or a key that contains 'fabric' or 'databricks' .

I am still open for a better solution but for now I can survive with this.

Re: detect runtime

V-yubandi-msft — Wed, 02 Apr 2025 04:05:12 GMT

You managed to find a temporary solution, so it's best to mark it as accepted. This will be helpful for others facing the same issue for now.