Forum Discussion
Using a VNet gateway with Notebooks
- 2 years ago
Do you know if this feature is available for "On-Premises Data Gateways"?
I'm unsure what you are asking beyond the information i provided.
We have enabled the On-prem gateways so that Pipelines and Gen2 dataflows can connect to an AWS PostGreSQL RDS database. That works fine with the Pipeline. But we want to use Notebooks instead to query and pull that data in. The Notebook query template given to us by Microsoft consultants Was:
# Import necessary libraries
from pyspark.sql import SparkSession
# Create a Spark session
spark = SparkSession.builder \
.appName("Read from PostgreSQL") \
.getOrCreate()
# Define the PostgreSQL connection properties
jdbc_url = "jdbc:postgresql://your-postgres-hostname:5432/your-database"
connection_properties = {
"user": "your-username",
"password": "your-password",
"driver": "org.postgresql.Driver"
}
# Read data from a table in the PostgreSQL database
table_name = "your_table_name"
df = spark.read.jdbc(url=jdbc_url, table=table_name, properties=connection_properties)
# Show the data
df.show()
# Stop the Spark session
spark.stop()However, we should be triggering the gateway and not including the username and password in the code.
- Anonymous2 years agoNot applicable
Hi eb38335 ,
I suggest you to use Azure Key Vault Secrets inorder to store your username and password.
Accessing Azure Key Vault Secrets from Fabric Notebooks (robkerr.ai)
Additional Docs to refer -
Solved: Microsoft Fabric connect to Azure Key Vault - Microsoft Fabric Community
Hope this is helpful. Please let me know incase of further queries.- eb383352 years agoRegular Visitor
Actually, that's what the gateway does. It manages the connection from Fabric to the database for the query when the database path matches what's in the gateway path.
However, I'm unsure how we invoke the gateway from the Notebook like it is invoked from the Pipeline.
- Anonymous2 years agoNot applicable
Hi eb38335 ,
Currently we can only invoke the gateway using Pipeline, we cannot invoke it using Fabric Notebooks.
Docs to refer -
VNET Data Gateway for Fabric and Power BI is now Generally Available | Microsoft Fabric Blog | Microsoft FabricWhat is a virtual network (VNet) data gateway | Microsoft Learn
Hope this is helpful. Please let me know incase of further queries.