topic Re: Forbidden error using saveAsTable in Data Engineering

Forbidden error using saveAsTable

PeterDaniels — Thu, 22 Aug 2024 23:15:56 GMT

I have a dataframe. Trying to simply saveAsTable (delta). I get a strange Forbidden error. I am admin in the workspace and I'm the one that created the lakehouse. I've tried just using a table name, using lakehouse name dot table name, and using dbo in the middle. Baffled.

Re: Forbidden error using saveAsTable

frithjof_v — Fri, 23 Aug 2024 00:02:54 GMT

Schema enabled Lakehouse?

Same issue here, it seems: https://community.fabric.microsoft.com/t5/Data-Engineering/Error-when-writing-data-into-schema-enabled-lakehouse-using/m-p/4111069#M3708

A possible solution was proposed.

Anyway, schema enabled Lakehouses have big limitations at the moment. It's a preview feature still.

Re: Forbidden error using saveAsTable

Anonymous — Fri, 23 Aug 2024 00:44:17 GMT

Hi @PeterDaniels ,

Thank you for your reply from frithjof_v .

I was able to reproduce your error with the preview schema feature enabled, and the error occurs seemingly because of an architecture mismatch.

Using the statement “SaveAsTable” works fine, you can try it:

df = spark.read.format(“csv”).option(“header”, “false”).load( “Files/2019.csv") # Create a new table df.write.format(“delta”).saveAsTable(“test2”)

If you have any other questions please feel free to contact me.

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

Re: Forbidden error using saveAsTable

PontusPersson — Fri, 23 Aug 2024 06:50:28 GMT

I have the same issue, and I know this has worked before. It stopped working yesterday (about 16-17 hours before this post was when I noticed it). Reading tables (e.g. spark.read.table("lakehouse.schema.table") does not work, but reading using path (e.g. spark.read.format("delta").load("Tables/schema/table")) does work. And same for writing. Receiving 403 forbidden error. Only happens on schema enabled lakehouse.

Re: Forbidden error using saveAsTable

PeterDaniels — Sat, 14 Sep 2024 12:42:05 GMT

Yeah. I think the solution for now is to not use schema enabled lakehouses. Hopefully that feature will get more robust as it heads towards GA