Forum Discussion
Error : Notebook writing table into a Warehouse
Hello CMJJ
Thanks for also testing the case.
I found a solution even if it is not the best because I really think the "writing" function of msspark doesn't work.
So I used this :warehouse_url = "my_url" jdbc_url = f"jdbc:sqlserver://{warehouse_url}:1433;database={database_name}" final_table = f"[schema].[table]" token = mssparkutils.credentials.getToken("pbi") df_joined.write \ .format("jdbc") \ .option("url", jdbc_url) \ .option("dbtable", final_table) \ .option("accessToken", token) \ .option("driver", "com.microsoft.sqlserver.jdbc.SQLServerDriver") \ .mode("overwrite") \ .save()
Hello Akash_Varuna,
Thanks for your answer, I've tried several things
First I modify with "overwrite" and a schema that doesn't exist:
from com.microsoft.spark.fabric.Constants import Constants
database_name = "warehouse_name"
df = spark.read.option(Constants.DatabaseName, database_name).synapsesql("SELECT TOP 1 * FROM [sys_data].[statistic]")
final_table = rf"{database_name}.000_test.new_table_stat"
df.write.mode("overwrite").synapsesql(final_table)
So I got this error because "000_test" does not exist, so it is normal
The specified schema name "000_test" either does not exist or you do not have permission to use it.
Now I create the schema and I try again, but I get the same error than in my original post even with "overwrite", something like:
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Path 'https://i-api.onelake.fabric.microsoft.com/workspace_id/_system/artifacts/synapse-notebook-id/id/user/trusted-service-user/statistic378ec4926caf4943994e3ab45bc57c8e/*.parquet' has URL suffix which is not allowed.
So it doesn't solve my error
Hi Anonymous You have the proper permissions right ?
Hello Akash_Varuna,
The notebook is in the same workspace that the warehouse and I'm an admin of the workspace.
And if I try a TSQL query in a TSQL notebook, I can create a table.
So yes, the permissions are right