Enable On-Prem and Virtual Data Gateway Connections in Fabric Notebooks

[change007]

Problem:

• Notebook Users are restricted to cloud sources, making it impossible to connect interactively to on-prem databases or services.
• Workaround approaches (shortcuts, data pipelines) add complexity, latency, and maintenance overhead.

Proposed Enhancement:

Integrate gateway support into the Fabric Notebook runtime so that:

• Users can select existing on-prem or virtual gateway connections when configuring data sources in a notebook.
• Queries issued from Spark in notebooks are automatically proxied through the gateway without extra pipeline configuration.

[treyu33]

I suspect it may be more complicated with PySpark notebooks given the distributed architecture. However, I really think that it should be at least possible with a single node pure Python Notebook in Fabric. Would almost certainly be more efficient for small-medium workloads in terms of CUs compared to data pipeline copy activity or copy job.

[datpbi_newbie]

This is much much-needed feature to connect the on-prem sources like Oracle or PostgreSQL db in the Notebooks.

[jvanegmond]

I was rebuilding some fabric pipelines to notebooks for data ingestion. Performance is better for notebooks and CUs spend are lower. Then I bumped into two on prem sources which I could migrate. Connecting to Fabric connections would be a great solution