Forum Discussion

ackerman_chris's avatar
ackerman_chris
New Member
4 months ago
Solved

Persistent "Login Failed" using PySpark JDBC + Token Auth for Fabric SQL Database

Hi everyone,   I'm currently writing a notebook in PySpark that is attempting to write data over to a new Microsoft Fabric SQL Database. I’m trying to keep my ingestion logic strictly within Fabric...
  • deborshi_nag's avatar
    4 months ago

    Hello ackerman_chris 

     

    The best option is to use the Spark connector for SQL databases in Fabric - details in the Microsoft doc below:

    Spark connector for SQL databases - Microsoft Fabric | Microsoft Learn

     

    Please bear in mind this is still in Preview. 

     

    For getting the right connection string for your database, go to the Fabric SQL database in question, Settings > Connection Strings > JDBC. Remove "authentication=ActiveDirectoryInteractive" from the connection string. It should end with a ";".

     

    Here's a sample code:

    import com.microsoft.sqlserver.jdbc.spark
    
    
    data = [("Alice", 1), ("Bob", 2), ("Charlie", 3)]
    cols = ["Name", "Age"]
    df = spark.createDataFrame(data, cols)
    
    # 2) JDBC URL for your Fabric SQL database
    url = "jdbc:sqlserver://blah-blah.database.fabric.microsoft.com:1433;database={blah};encrypt=true;trustServerCertificate=false;" 
    
    
    (df.write
       .mode("overwrite")                 # overwrite | append | errorifexists | ignore
       .option("url", url)
       .mssql("dbo.publicExample"))
    
    #  3) Read it back to verify
    spark.read.option("url", url).mssql("dbo.publicExample").show()