topic Re: How to access Fabric Warehouse from Fabric Notebooks? in Fabric platform

How to access Fabric Warehouse from Fabric Notebooks?

Madhusudan_P — Mon, 22 Jan 2024 11:23:44 GMT

Hi,

I am trying to access Fabric Datawarehouse table from Fabric notebooks. I am copying ABFS path of Datawarehouse table from its properties as under the hood table in datawarehouse is a delta table.

And then passed below query -

df = spark.read.format("delta").load("abfss://My workspace@onelake.dfs.fabric.microsoft.com/test_dw.Datawarehouse/Tables/dbo/Date")

It's getting failed with the following error -
IllegalArgumentException: Illegal character in authority at index 8: abfss://My workspace@onelake.dfs.fabric.microsoft.com.

I tried passing below query but again same error.

df = spark.read.format("delta").load("abfss://My%20workspace@onelake.dfs.fabric.microsoft.com/test_dw.Datawarehouse/Tables/dbo/Date")

Just wondering is it possible to read and write to warehouse tables from notebooks in Fabric?
We have similar option in Synapse in which we can read/write data warehouse tables using .synapsesql option using Notebooks.

https://techcommunity.microsoft.com/t5/azure-synapse-analytics-blog/writing-data-using-azure-synapse-dedicated-sql-pool-connector/ba-p/3535930

Re: How to access Fabric Warehouse from Fabric Notebooks?

AndyDDC — Mon, 22 Jan 2024 11:35:02 GMT

You won't be able to read or write to Warehouse tables as those requests need to go through the SQL endpoint of the warehouse, it won't work with Spark.

You can use something like pyodbc or jdbc to connect to the Warehouse from within a notebook:

https://learn.microsoft.com/en-us/fabric/data-warehouse/connectivity

Re: How to access Fabric Warehouse from Fabric Notebooks?

Madhusudan_P — Wed, 31 Jan 2024 10:40:45 GMT

Hi AndyDDC ,

Thanks for your answer. However, I was curious to try the query said above without any space in the workspace name. For that, I had to create new Fabric account as I don't have new workspace creation request in existing account. It took me a while, but I am able to read DW table from Spark notebook.

df = spark.read.format("delta").load("abfss://workspace-without-space@onelake.dfs.fabric.microsoft.com/test_dw.Datawarehouse/Tables/dbo/Date")

df.write is getting failed as it says PUT request forbidden.

However, It seems like overwrite or create new table in Warehouse using Spark Notebook is not possible as of now. Pyodbc and jdbc might help.

Re: How to access Fabric Warehouse from Fabric Notebooks?

Anonymous — Mon, 29 Jan 2024 17:37:10 GMT

Hi Madhusudan_P
That's correct, you cannot write directly to the warehouse folder. All updates have to come through the warehouse engine. You can't write to DW tables from Spark or any other engine .

Re: How to access Fabric Warehouse from Fabric Notebooks?

Madhusudan_P — Tue, 30 Jan 2024 01:01:42 GMT

Hi Anonymous
Since we have the capacity in Synapse Dedicated SQL Pool to write from Synapse notebook, why this hasn't been extended to Fabric Warehouse and Notebooks. Is it expected in future?
Dedicated SQL Pool connector for Apache Spark

Re: How to access Fabric Warehouse from Fabric Notebooks?

Anonymous — Tue, 30 Jan 2024 04:03:30 GMT

Hi Madhusudan_P
Currently we do not have any information regarding the availability of this feature.

Appreciate if you could share the feedback on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.

Hope this helps. Please let me know if you have any further queries.

Re: How to access Fabric Warehouse from Fabric Notebooks?

govindarajan_d — Tue, 30 Jan 2024 13:28:25 GMT

Hi Madhusudan_P,

You can add the Warehouse as a shortcut to Lakehouse and then read it just as a table. But even in this method, you will be able to only read data from Warehouse and not write.

The approach I would suggest you is add the Lakehouse back in Warehouse and you would be able to query the data from Lakehouse within Warehouse. Kind of like a full circle.

Re: How to access Fabric Warehouse from Fabric Notebooks?

jankuc — Tue, 29 Apr 2025 08:41:40 GMT

Since 2025Q1 the following should work (even reading from different workspace)

df.write.synapsesql("<warehouse/lakehouse name>.<schema name>.<table name>") # this uses default mode - errorifexists df.write.mode("errorifexists").synapsesql("<warehouse/lakehouse name>.<schema name>.<table name>") df.write.mode("ignore").synapsesql("<warehouse/lakehouse name>.<schema name>.<table name>") df.write.mode("append").synapsesql("<warehouse/lakehouse name>.<schema name>.<table name>") df.write.mode("overwrite").synapsesql("<warehouse/lakehouse name>.<schema name>.<table name>")

https://learn.microsoft.com/en-us/fabric/data-engineering/spark-data-warehouse-connector?tabs=pyspark#supported-dataframe-save-modes

Re: How to access Fabric Warehouse from Fabric Notebooks?

jankuc — Tue, 29 Apr 2025 08:44:46 GMT

Currently since 2025Q1 the synapsesql() should work:

import com.microsoft.spark.fabric from com.microsoft.spark.fabric.Constants import Constants df = spark.read.synapsesql("<warehouse/lakehouse name>.<schema name>.<table or view name>")

https://learn.microsoft.com/en-us/fabric/data-engineering/spark-data-warehouse-connector?tabs=pyspark#supported-dataframe-save-modes

Re: How to access Fabric Warehouse from Fabric Notebooks?

stuckerj — Thu, 16 Oct 2025 20:54:32 GMT

This code works just fine. I had generated some synthetic data with time stamps and needed to fix a "no time zone" error.

# Using spark connector to warehouse
from com.microsoft.spark.fabric import Constants
from pyspark.sql.functions import col

# Make sure timestamps have time zone
for c in df.columns:
    if dict(df.dtypes)[c] == "timestamp_ntz":
        df = df.withColumn(c, col(c).cast("timestamp"))

# Write data from a DataFrame to the warehouse
df.write.mode("overwrite").synapsesql(f"{target_warehouse}.dbo.{warehouse_table_name}")

## Verify the data written to the warehouse
# df_verify = spark.read.synapsesql(f"{target_warehouse}.dbo.{warehouse_table_name}")
#display(df_verify.limit(10))