Issues writing to Warehouse from PySpark Notebook

Hi TNastyCodes ,

What your errors mean:
• saveAsTable(... 'warehouse_name.dbo.table') -> PUT not allowed on Tables path – you tried to write a Delta table directly into a Warehouse’s managed storage. Warehouses don’t accept Delta writes from Spark; all writes go through the SQL engine, not straight into OneLake folders. See “two-phase write via COPY INTO” in the Spark <-> Warehouse connector doc, and note that Warehouse tables are stored/managed by the SQL engine even though they live in OneLake. (Spark connector for Fabric Data Warehouse, Lakehouse & Warehouse “Better Together”)
• df.write.synapsesql(...)-> FabricSparkTDSWriteError ... Content ... cannot be listed – this usually indicates a staging step failure in the connector’s two-phase write (Spark stages Parquet, then the engine runs COPY INTO). Common causes: Private Link enabled (write not supported) or environment/permission issues. (doc notes & restrictions)
• JDBC -> NVARCHAR(MAX) not supported – Fabric Warehouse doesn’t support NVARCHAR types; use VARCHAR (UTF-8). VARCHAR(MAX) exists but is preview with a 1 MB limit, so the safest path is VARCHAR(n) where possible. (Warehouse data types)

Quick solution

1. Prefer the Fabric Spark connector (synapsesql) for PySpark → Warehouse.
   – Use Runtime 1.3 (or newer).
   – Disable Private Link for write scenarios (tenant/workspace).
   – Pre-create the Warehouse table with VARCHAR columns (not NVARCHAR) if you need strict typing.
   (doc with examples & constraints)
2. If you must use JDBC:
   – Pre-create the target table in the Warehouse with VARCHAR types and then TRUNCATE TABLE + mode("append") from Spark.
   – Or specify createTableColumnTypes so Spark won’t default to NVARCHAR.
   (data type support)
3. Consider SQL-first ingestion for Bronze>Silver:
   – Use Warehouse T-SQL (CTAS / INSERT-SELECT) reading from the Lakehouse SQL analytics endpoint (via shortcuts/gold views) or
   – Use COPY INTO into Warehouse from external storage. OneLake as a direct COPY source has been in preview and may vary by tenant; if it isn’t enabled, stage in ADLS Gen2.
   (Lakehouse/Warehouse integration, Warehouse ingest overview)

If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.