Forum Discussion

Rob95's avatar
Rob95
New Member
1 month ago
Solved

Trying to write data to Delta Table

In a regular python notebook I'm trying to write data to a delta table. It used to work but recently I either get this error in the pipeline run: PythonComputeClientException Something went wrong w...
  • v-csrikanth's avatar
    1 month ago

    Hi  Rob95 
    Adding a few points specific to regular Python notebooks that usually pinpoint this:

    • Pure-Python Delta writes are the most common cause. Regular Python notebooks use a small session pool and the deltalake library — once the DataFrame grows past a few hundred MB, the write fails and surfaces as this exact generic error. Running the same write in a PySpark notebook almost always resolves it.

    • Use the full ABFSS path. Default-lakehouse bindings can silently drop after workspace changes, making relative paths fail:
      path = "abfss://<workspace>@onelake.dfs.fabric.microsoft.com/<lakehouse>.Lakehouse/Tables/<table>"

    •  Check for capacity throttling. Under CU smoothing, Fabric returns this exact "please try again later" wording instead of a proper throttle error. Open the Fabric Capacity Metrics app and check the failure window.

    • Quick isolation test: run the write with df.head(100). If it succeeds, it's a compute/resource limit, not permissions or schema.

    • Capture the real error with:

      ------------------------------------
      import traceback
      try:
      df.write.format("delta").mode("append").save(path)
      except Exception:
      traceback.print_exc()
      raise

      ------------------------------------
    If it still fails after switching to PySpark + ABFSS path, raise a Microsoft Support ticket with both TraceIds — the backend team can trace the exact failure from those.

     

    Could you confirm if it's a pure Python or PySpark notebook, and roughly the DataFrame size? That'll narrow it down quickly.


    Thanks,
    Srikanth Cheri
    Community Support Team