Forum Discussion
Spark Connector Issue
I tried myself now, and I am getting the following error:
Request to read failed. Reason - HTTP request forbidden.
com.microsoft.spark.fabric.tds.error.FabricSparkTDSInternalAuthError: HTTP request forbidden.
So I am not able to use it myself either.
But to be honest I don't have experience with Scala, so perhaps I did something wrong in my code, I don't know. I tried to follow this page: Spark connector for Microsoft Fabric Synapse Data Warehouse - Microsoft Fabric | Microsoft Learn
I used this code:
Cell 1 (runs fine)
%%spark
import com.microsoft.spark.fabric.tds.implicits.read.FabricSparkTDSImplicits._
import org.apache.spark.sql.functions._
Cell 2 (gives the error when I run this cell)
%%spark
val df = spark.read.synapsesql("TestWarehouse.dbo.customer_dimension")
I also tried this
Cell 3 (gives the same error when I run this cell)
%%spark
import com.microsoft.spark.fabric.Constants
val df = spark.read.option(Constants.WorkspaceId, "550e8400-e29b-41d4-a716-446655440000").synapsesql("testwarehouse.dbo.customer_dimension")
This is my table in the Fabric Warehouse. It's in the same workspace as the Lakehouse which is the Default lakehouse of my Notebook.
So this is the table I am trying to read in a Notebook by using the Spark connector for Microsoft Fabric Synapse Data Warehouse, but I am getting the error message as mentioned above.
I am the user who created (and therefore also admin) of the workspace, the lakehouse and the warehouse.