Forum Discussion
Error : Notebook writing table into a Warehouse
Hello CMJJ
Thanks for also testing the case.
I found a solution even if it is not the best because I really think the "writing" function of msspark doesn't work.
So I used this :warehouse_url = "my_url" jdbc_url = f"jdbc:sqlserver://{warehouse_url}:1433;database={database_name}" final_table = f"[schema].[table]" token = mssparkutils.credentials.getToken("pbi") df_joined.write \ .format("jdbc") \ .option("url", jdbc_url) \ .option("dbtable", final_table) \ .option("accessToken", token) \ .option("driver", "com.microsoft.sqlserver.jdbc.SQLServerDriver") \ .mode("overwrite") \ .save()
Hi Anonymous
Confirming that I am receiving this identical error also, my first time trying to use this feature. I also was using the tutorial you linked, as well as this article:
Using Spark to Read and Write Data in Microsoft Fabric Data Warehouse
I've tried overwrite option with both the table existing and not existing, but the error is identical. I also tried the append option with no luck.
The dataframe columns and target table columns are identical.
The account I'm running the notebook with is a workspace admin, and the Warehouse is in the same workspace as the Notebook.
Hello CMJJ
Thanks for also testing the case.
I found a solution even if it is not the best because I really think the "writing" function of msspark doesn't work.
So I used this :
warehouse_url = "my_url"
jdbc_url = f"jdbc:sqlserver://{warehouse_url}:1433;database={database_name}"
final_table = f"[schema].[table]"
token = mssparkutils.credentials.getToken("pbi")
df_joined.write \
.format("jdbc") \
.option("url", jdbc_url) \
.option("dbtable", final_table) \
.option("accessToken", token) \
.option("driver", "com.microsoft.sqlserver.jdbc.SQLServerDriver") \
.mode("overwrite") \
.save()Thanks Ard for the code sample. I had to make one change to your code as it was failing when the notebook runs under the identity of a service principal with your code.
Change:
token = mssparkutils.credentials.getToken("pbi")
To:
token = notebookutils.credentials.getToken('https://database.windows.net')My guess is that under the covers .synapsesql is using a pbi token and that token for service principals doesn't have the right access (whereas it does for regular user accounts) as mentioned here. Seems like a .synapsesql bug.
