Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Get certified in Microsoft Fabric—for free! For a limited time, get a free DP-600 exam voucher to use by the end of 2024. Register now

Reply
cailen_cg
Frequent Visitor

Unable to clear lease on resource

Context: 

  • Using a Notebook in Fabric to process json files
  • Notebook reads json, then writes dataframe to a table, and finally uses mssparkutils to move the processed file
  • Fabric capacity recently hit a limit. Many jobs failed.

 

Our problem:

This fails:

 

 

 

filename = 'Files/problem.json'
destination_dir = 'Files/processed/'

mssparkutils.fs.mv(filename, destination_dir, True)

 

 

 

Here is the error message:

 

 

 

Py4JJavaError: An error occurred while calling z:notebookutils.fs.mv.
: Operation failed: "Precondition Failed", 412, PUT, http://onelake.dfs.fabric.microsoft.com/uuid/uuid/Files/processed?timeout=90, LeaseIdMissing, "There is currently a lease on the resource and no lease ID was specified in the request. RequestId:uuid Time:2024-10-24T15:00:05.1321650Z"
	at org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.completeExecute(AbfsRestOperation.java:231)
	at org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.lambda$execute$0(AbfsRestOperation.java:191)
	at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDurationOfInvocation(IOStatisticsBinding.java:464)
	at org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.execute(AbfsRestOperation.java:189)
	at org.apache.hadoop.fs.azurebfs.services.AbfsClient.renamePath(AbfsClient.java:509)
	at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.rename(AzureBlobFileSystemStore.java:986)
	at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.rename(AzureBlobFileSystem.java:456)
	at com.microsoft.spark.notebook.msutils.impl.MSFsUtilsImpl.mvWithinFileSystem(MSFsUtilsImpl.scala:147)
	at com.microsoft.spark.notebook.msutils.impl.MSFsUtilsImpl.$anonfun$mv$2(MSFsUtilsImpl.scala:355)
	at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
	at com.microsoft.spark.notebook.msutils.impl.MSFsUtilsImpl.fsTSG(MSFsUtilsImpl.scala:223)
	at com.microsoft.spark.notebook.msutils.impl.MSFsUtilsImpl.$anonfun$mv$1(MSFsUtilsImpl.scala:331)
	at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
	at com.microsoft.spark.notebook.common.trident.CertifiedTelemetryUtils$.withTelemetry(CertifiedTelemetryUtils.scala:71)
	at com.microsoft.spark.notebook.msutils.impl.MSFsUtilsImpl.mv(MSFsUtilsImpl.scala:331)
	at mssparkutils.IFs.mv(fs.scala:24)
	at mssparkutils.IFs.mv$(fs.scala:20)
	at notebookutils.fs$.mv(utils.scala:12)
	at notebookutils.fs.mv(utils.scala)
	at jdk.internal.reflect.GeneratedMethodAccessor218.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374)
	at py4j.Gateway.invoke(Gateway.java:282)
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
	at py4j.commands.CallCommand.execute(CallCommand.java:79)
	at py4j.GatewayConnection.run(GatewayConnection.java:238)
	at java.base/java.lang.Thread.run(Thread.java:829)

 

 

1 ACCEPTED SOLUTION
v-shex-msft
Community Support
Community Support

Hi @cailen_cg,

I test with your code and find it create a file in Files folder and name with processed.

For this scenario, I'd like to suggest you add the filename to the 'destination dir' parameter and try again with following code:

 

from notebookutils import mssparkutils

filename='test.json'
Source_path = 'Files/' + filename
destination_dir = 'Files/processed/' + filename
 
notebookutils.fs.mv(Source_path, destination_dir, True)

 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider accept as solution to help other members find it more quickly.

View solution in original post

2 REPLIES 2
v-shex-msft
Community Support
Community Support

Hi @cailen_cg ,

Any update on this? Did the above suggestions help with your scenario? if that is the case, you can consider Kudo or Accept the helpful suggestions to help others who faced similar requirements.

If these also don't help, please share more detailed information and description to help us clarify your scenario to test.

How to Get Your Question Answered Quickly 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider accept as solution to help other members find it more quickly.
v-shex-msft
Community Support
Community Support

Hi @cailen_cg,

I test with your code and find it create a file in Files folder and name with processed.

For this scenario, I'd like to suggest you add the filename to the 'destination dir' parameter and try again with following code:

 

from notebookutils import mssparkutils

filename='test.json'
Source_path = 'Files/' + filename
destination_dir = 'Files/processed/' + filename
 
notebookutils.fs.mv(Source_path, destination_dir, True)

 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider accept as solution to help other members find it more quickly.

Helpful resources

Announcements
November Carousel

Fabric Community Update - November 2024

Find out what's new and trending in the Fabric Community.

Live Sessions with Fabric DB

Be one of the first to start using Fabric Databases

Starting December 3, join live sessions with database experts and the Fabric product team to learn just how easy it is to get started.

November Update

Fabric Monthly Update - November 2024

Check out the November 2024 Fabric update to learn about new features.

Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Early Bird pricing ends December 9th.