Critical section to prevent concurrent cell execution on notebook in Fabric

[dbeavon3]

We have some resources in Fabric that need to be protected from concurrent access.  Assume there are 10 notebooks running at the exact same time for 30 mins.  Each notebook runs on its own session, but shares a custom pool.  There might be a single cell of five seconds that should NEVER be executed from two notebook sessions at the exact same time.

Has anyone tried to find an approach for ensuring the critical section is not entered by more than one session?

I am looking for a solution that can be implemented entirely in Fabric, and uses a simple, and well-known technology.  There are probably lots of ways to solve with the help of a remote SQL database , but that seems overkill.

I think the best approach is to create a file in the lakehouse, and simply take a short-term lease on that file.  That should prevent others from getting past that line of code, until the lease expires or it is explicitly released.  Does that sound like a reasonable approach?   Is there a simpler approach that I'm missing?  (eg. maybe a specialized Fabric library that is specifically built for helping with concurrency and scheduling concerns)?

[dbeavon3]

Thanks, I have already used this approach in the past.  I just wanted to make sure there wasn't something more obvious.

In general I do not really like using the file system for concerns that are not file-related.  A lock in a database would be a LOT more intuitive than a lease in a file system.  However, the LH file system is right at our fingertips, so I think it is the best option. 

Ideally they would add some related functionality to "notebookutils" one day.  (... eg. named mutexes or something like that).

[nilendraFabric · Accepted solution]

Hi @dbeavon3 

Very interesting discussion. 
I haven't implemented any such thing in Fabric yet. But I agree with your approach as it can be implemented in Fabric itself.

to ensure only one notebook at a time can enter a short critical section is to use the underlying OneLake (ADLS) support for leases on a file. When a notebook cell begins, it attempts to acquire an exclusive lease on a “lock file.” If the lease is granted, the notebook proceeds with the five‐second operation; otherwise, it retries or waits until the lease is released. Because OneLake runs on ADLS Gen2, this pessimistic concurrency pattern is directly supported. You can programmatically request and release the lease (for example, via Python or Spark code in the notebook), and no second notebook can acquire the same lease until it’s relinquished or expires.

https://learn.microsoft.com/en-us/azure/storage/blobs/concurrency-manage?toc=%2Fazure%2Fstorage%2Fbl...

No built-in Fabric library currently provides a simpler, fully internal locking mechanism across multiple Spark sessions. High concurrency mode in notebooks allows multiple sessions to share the same Spark application, but it does not itself prevent simultaneous entry into a particular code cell.

Thanks