Forum Discussion
Trying to write data to Delta Table
In a regular python notebook I'm trying to write data to a delta table. It used to work but recently I either get this error in the pipeline run:
Or I get the following error when running the notebook manually:
Hi Rob95
Adding a few points specific to regular Python notebooks that usually pinpoint this:-
Pure-Python Delta writes are the most common cause. Regular Python notebooks use a small session pool and the deltalake library — once the DataFrame grows past a few hundred MB, the write fails and surfaces as this exact generic error. Running the same write in a PySpark notebook almost always resolves it.
-
Use the full ABFSS path. Default-lakehouse bindings can silently drop after workspace changes, making relative paths fail:
path = "abfss://<workspace>@onelake.dfs.fabric.microsoft.com/<lakehouse>.Lakehouse/Tables/<table>" -
Check for capacity throttling. Under CU smoothing, Fabric returns this exact "please try again later" wording instead of a proper throttle error. Open the Fabric Capacity Metrics app and check the failure window.
-
Quick isolation test: run the write with df.head(100). If it succeeds, it's a compute/resource limit, not permissions or schema.
-
Capture the real error with:
------------------------------------
import traceback
try:
df.write.format("delta").mode("append").save(path)
except Exception:
traceback.print_exc()
raise------------------------------------
If it still fails after switching to PySpark + ABFSS path, raise a Microsoft Support ticket with both TraceIds — the backend team can trace the exact failure from those.Could you confirm if it's a pure Python or PySpark notebook, and roughly the DataFrame size? That'll narrow it down quickly.
Thanks,
Srikanth Cheri
Community Support Team-
5 Replies
Hi Rob95
Adding a few points specific to regular Python notebooks that usually pinpoint this:-
Pure-Python Delta writes are the most common cause. Regular Python notebooks use a small session pool and the deltalake library — once the DataFrame grows past a few hundred MB, the write fails and surfaces as this exact generic error. Running the same write in a PySpark notebook almost always resolves it.
-
Use the full ABFSS path. Default-lakehouse bindings can silently drop after workspace changes, making relative paths fail:
path = "abfss://<workspace>@onelake.dfs.fabric.microsoft.com/<lakehouse>.Lakehouse/Tables/<table>" -
Check for capacity throttling. Under CU smoothing, Fabric returns this exact "please try again later" wording instead of a proper throttle error. Open the Fabric Capacity Metrics app and check the failure window.
-
Quick isolation test: run the write with df.head(100). If it succeeds, it's a compute/resource limit, not permissions or schema.
-
Capture the real error with:
------------------------------------
import traceback
try:
df.write.format("delta").mode("append").save(path)
except Exception:
traceback.print_exc()
raise------------------------------------
If it still fails after switching to PySpark + ABFSS path, raise a Microsoft Support ticket with both TraceIds — the backend team can trace the exact failure from those.Could you confirm if it's a pure Python or PySpark notebook, and roughly the DataFrame size? That'll narrow it down quickly.
Thanks,
Srikanth Cheri
Community Support TeamTransferring to a pyspark notebook is the easiest fix. The error code is a bit vague but thanks a lot for helping!
-
Hi Rob95 ,
This error is unfortunately quite generic and usually doesn’t point directly to the root cause. Since it was working previously and started failing recently, I'd check a few things:
1. Verify that the target Delta table is still accessible and that there haven't been any permission or workspace changes.
2. Check whether the underlying storage account/Lakehouse is available and not experiencing connectivity issues.
3. Review the Spark/job logs around the TraceId for a more detailed exception, as PythonComputeClientException is often just a wrapper around the actual error.
4. Confirm that no schema changes were made to the Delta table recently that could cause write failures.5. If this is running in a pipeline, try running the notebook on a fresh Spark session or restarting the compute environment.
If you can share the code used for the write operation and any additional Spark driver/executor logs associated with the TraceId, it may be easier to identify the exact cause. At the moment, the error message itself is too generic to pinpoint the issue.
I agree with Asmita_27 and v-csrikanth. Please provide more details or give the command that is throwing the error.
Hi Rob95
If you're using the Purepython or Python only runtime notebooks, you can use my blog post below, which I explain how to successfully write to the Lake house tables. https://www.fourmoo.com/2026/07/09/using-python-notebooks-to-merge-data-into-a-microsoft-fabric-lakehouse/
