Error running a spark notebook reading a table from warehouse

Hi Anonymous 

May I check if this issue has been resolved? If not, Please feel free to contact us if you have any further questions.

Thank you

open

Hi Anonymous,

That FabricSparkTDSInternalAuthError: HTTP request forbidden means your notebook’s identity can reach the Lakehouse (OneLake/ABFS) but doesn’t have permission to the Warehouse SQL engine that sits behind spark.read.synapsesql("Warehouse.dbo.configuration"). Pipelines work because they typically run under a different identity that does have the required Warehouse SQL permissions.

1. Grant the service principal access at the Warehouse (item) level
   Give it at least “Read all data using SQL” on the Warehouse (or higher if needed). This is the Warehouse’s SQL-engine permission, separate from Lakehouse/OneLake permissions. See Microsoft’s connector permissions note here and a concise overview here.
2. Create a database user for the SPN and add it to roles (object-level)
   

   -- Read tables
   ALTER ROLE db_datareader ADD MEMBER [<your-spn-app-id-or-name>];
   
   -- Optional: allow INSERT/UPDATE/DELETE
   ALTER ROLE db_datawriter ADD MEMBER [<your-spn-app-id-or-name>];
       

   The connector requires item-level + object-level permissions to read tables. Microsoft documents this requirement here. Using a service principal with Warehouse is supported and documented here.
3. Confirm tenant setting for service principals
   In the Fabric Admin portal, ensure service principals are allowed to use Fabric (scope to your SPN’s security group if your org restricts this).

Once those are in place, your line should work:

config_df = spark.read.synapsesql("Warehouse.dbo.configuration")

why it happens

• Lakehouse vs Warehouse permissions: Lakehouse operations ride on OneLake permissions you already granted; the Warehouse connector (synapsesql) calls the SQL engine (TDS endpoint) which also needs SQL-style permissions. Without them, you get an HTTP 403/forbidden from the connector. See the Fabric Spark-to-Warehouse connector doc here.
• Pipeline works, Airflow doesn’t: Your Pipeline likely executes under an identity that already has Warehouse SQL access (for example, the user who created it or a configured app). Airflow runs with your service principal, which currently lacks Warehouse SQL permissions. Similar cases have been reported with 403/forbidden until the SPN was granted SQL engine access (community examples: example, and a recent thread here).
• Object-level matters: Even after giving “Read all data using SQL” at the item, you still need a database principal mapped and placed in db_datareader (and db_datawriter if you’ll write). A quick pattern is shown above and mirrors guidance such as this.

If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.