Unable to clear lease on resource

cailen_cg — Thu, 24 Oct 2024 15:21:13 GMT

Context:

Using a Notebook in Fabric to process json files
Notebook reads json, then writes dataframe to a table, and finally uses mssparkutils to move the processed file
Fabric capacity recently hit a limit. Many jobs failed.

Our problem:

This fails:

filename = 'Files/problem.json' destination_dir = 'Files/processed/' mssparkutils.fs.mv(filename, destination_dir, True)

Here is the error message:

Py4JJavaError: An error occurred while calling z:notebookutils.fs.mv. : Operation failed: "Precondition Failed", 412, PUT, http://onelake.dfs.fabric.microsoft.com/uuid/uuid/Files/processed?timeout=90, LeaseIdMissing, "There is currently a lease on the resource and no lease ID was specified in the request. RequestId:uuid Time:2024-10-24T15:00:05.1321650Z" at org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.completeExecute(AbfsRestOperation.java:231) at org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.lambda$execute$0(AbfsRestOperation.java:191) at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDurationOfInvocation(IOStatisticsBinding.java:464) at org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.execute(AbfsRestOperation.java:189) at org.apache.hadoop.fs.azurebfs.services.AbfsClient.renamePath(AbfsClient.java:509) at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.rename(AzureBlobFileSystemStore.java:986) at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.rename(AzureBlobFileSystem.java:456) at com.microsoft.spark.notebook.msutils.impl.MSFsUtilsImpl.mvWithinFileSystem(MSFsUtilsImpl.scala:147) at com.microsoft.spark.notebook.msutils.impl.MSFsUtilsImpl.$anonfun$mv$2(MSFsUtilsImpl.scala:355) at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23) at com.microsoft.spark.notebook.msutils.impl.MSFsUtilsImpl.fsTSG(MSFsUtilsImpl.scala:223) at com.microsoft.spark.notebook.msutils.impl.MSFsUtilsImpl.$anonfun$mv$1(MSFsUtilsImpl.scala:331) at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23) at com.microsoft.spark.notebook.common.trident.CertifiedTelemetryUtils$.withTelemetry(CertifiedTelemetryUtils.scala:71) at com.microsoft.spark.notebook.msutils.impl.MSFsUtilsImpl.mv(MSFsUtilsImpl.scala:331) at mssparkutils.IFs.mv(fs.scala:24) at mssparkutils.IFs.mv$(fs.scala:20) at notebookutils.fs$.mv(utils.scala:12) at notebookutils.fs.mv(utils.scala) at jdk.internal.reflect.GeneratedMethodAccessor218.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374) at py4j.Gateway.invoke(Gateway.java:282) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:238) at java.base/java.lang.Thread.run(Thread.java:829)

Re: Unable to clear lease on resource

Anonymous — Mon, 28 Oct 2024 01:32:13 GMT

Hi @cailen_cg,

I test with your code and find it create a file in Files folder and name with processed.

For this scenario, I'd like to suggest you add the filename to the 'destination dir' parameter and try again with following code:

from notebookutils import mssparkutils filename='test.json' Source_path = 'Files/' + filename destination_dir = 'Files/processed/' + filename notebookutils.fs.mv(Source_path, destination_dir, True)

Regards,

Xiaoxin Sheng

Re: Unable to clear lease on resource

Anonymous — Wed, 30 Oct 2024 08:03:19 GMT

Hi @cailen_cg ,

Any update on this? Did the above suggestions help with your scenario? if that is the case, you can consider Kudo or Accept the helpful suggestions to help others who faced similar requirements.

If these also don't help, please share more detailed information and description to help us clarify your scenario to test.

How to Get Your Question Answered Quickly

Regards,

Xiaoxin Sheng

topic Re: Unable to clear lease on resource in Data Engineering

Unable to clear lease on resource

Re: Unable to clear lease on resource

Re: Unable to clear lease on resource