not able to connect to fabric data warehouse from synapse notebook using pyspark

[v-vepott]

I am trying to connect fabric warehouse from synapse notebook using pyspark, but not able to connect. Also, I have tried to use the linked service in notebook and it's not working out.

Using below code

spark.conf.set("spark.storage.fabric.linkedServiceName", varLinkedServiceName) spark.conf.set("fs.azure.account.oauth.provider.type", "com.microsoft.fabric.tokenlibrary.LinkedServiceBasedTokenProvider") spark.conf.set("spark.sql.legacy.parquet.int96RebaseModeInRead", "CORRECTED") spark.conf.set("spark.sql.legacy.parquet.int96RebaseModeInWrite", "CORRECTED") spark.conf.set("spark.sql.legacy.parquet.datetimeRebaseModeInRead", "CORRECTED") spark.conf.set("spark.sql.legacy.parquet.datetimeRebaseModeInWrite", "CORRECTED")

used jdbc connector

jdbc_url = f"jdbc:sqlserver://{varADWServerName}:1433;databaseName={varADWDatabase};encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.msit-datawarehouse.fabric.microsoft.com;loginTimeout=5;" df = spark.read \ .format("jdbc") \ .option("url", jdbc_url) \ .option("dbtable", "Source.tblFilePartnerAuditEntities") \ .load()

getting below error

Py4JJavaError: An error occurred while calling o4334.load. : com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host 4a68b9ec28e9487ea59e6c86e120aa79.pbidedicated.windows.net\X6EPS4XRQ2XUDENLFV6NAEO3I4-VBO7RRI4LFQELXXXXXXXXXXX-dw (redirected from x6eps4xrq2xudenlfv6naeo3i4-vbo7rri4lfqeliq6oqbllpj56i.msit-datawarehouse.fabric.microsoft.com), port 1433 has failed. Error: "null. Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall

I am able to read the data using pipeline with the same linked service.

Could anyone help me how can we connect?

Thanks in advance,

Venkat