Forum Discussion
Notebook connect
HI Stevox,
Thank you for the follow-up.
Given your current setup, using spark.read.jdbc() to connect directly from a Microsoft Fabric notebook won't work because you only have the SQL connection string with SQL authentication and no access to configure firewall rules or create a Service Principal.
The Spark engine in notebooks uses separate outbound IP addresses, which are not permitted by default in Azure SQL firewall settings, causing the connection failure. Although Copy Data and Dataflows succeed, they use Fabric's managed integration runtimes with internal network access. Unfortunately, workspace connections created in Fabric cannot be accessed directly from notebooks at this time and are only available to UI-based tools like Dataflows, Pipelines, and Copy Activity.
As a workaround, you can use Copy Data or Dataflows to load filtered or minimal required data into a staging Lakehouse table, then run the incremental logic from your notebook. This keeps your notebook-based workflow intact without re-ingesting full datasets each time. Until Microsoft enables native notebook access to workspace connections or shared credentials, this workaround remains the most reliable with your current access level.Hope my suggestion gives you good idea, if you have any more questions, please feel free to ask we are here to help you.
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.Best regards,
Sahasra.
Hi sahasra,
Thank you for the clarification.
In my case, I only have the Azure SQL connection string using SQL authentication (username and password), and I don’t have access to configure the firewall or set up a Service Principal. While I’m able to connect successfully using Copy Data and Dataflows within Fabric, any attempt to connect using Spark’s JDBC connector in a notebook results in a firewall error — even though I’ve added the connection under my workspace connections.
The notebook I’m working with performs custom incremental loading across multiple tables, so direct access via Spark would be ideal for maintaining that workflow.
Is there a way to use the registered workspace connections directly within the notebook without using the copy or dataflow activity to copy the entire dataset into a staging table every time? Or any alternative method that works with my current access level?
Appreciate your guidance on whether there’s a way to go about this within the current Fabric limitations.
Thanks,
Community SupportHI Stevox,
Thank you for the follow-up.
Given your current setup, using spark.read.jdbc() to connect directly from a Microsoft Fabric notebook won't work because you only have the SQL connection string with SQL authentication and no access to configure firewall rules or create a Service Principal.
The Spark engine in notebooks uses separate outbound IP addresses, which are not permitted by default in Azure SQL firewall settings, causing the connection failure. Although Copy Data and Dataflows succeed, they use Fabric's managed integration runtimes with internal network access. Unfortunately, workspace connections created in Fabric cannot be accessed directly from notebooks at this time and are only available to UI-based tools like Dataflows, Pipelines, and Copy Activity.
As a workaround, you can use Copy Data or Dataflows to load filtered or minimal required data into a staging Lakehouse table, then run the incremental logic from your notebook. This keeps your notebook-based workflow intact without re-ingesting full datasets each time. Until Microsoft enables native notebook access to workspace connections or shared credentials, this workaround remains the most reliable with your current access level.
Hope my suggestion gives you good idea, if you have any more questions, please feel free to ask we are here to help you.
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Best regards,
Sahasra.