Forum Discussion

chetanhiwale's avatar
chetanhiwale
Resolver I
6 months ago
Solved

Synapse SQL connector for warehouse failed to write data.

Hello,  I am trying to write a dataframe to Fabric Warehouse using SynapseSQL connector. Somehow it is throwing attribute error.  Is anyone facing the same issue.  Code Snippet : Note :...
data engineering
Notebook
  • chetanhiwale's avatar
    chetanhiwale
    6 months ago

    Hi Folks, 

    The same code is working now. I havent changed anything but now it is running. Seems to be something internal issue. 

BalajiL's avatar
BalajiL
Resolver I
6 months ago

I dont think your schema will be same in df and table. 

Make sure your table 'ok' has integer type for column id. Then execute the below code it should able to execute in warehouse. 

 

import com.microsoft.spark.fabric
from com.microsoft.spark.fabric.Constants import Constants

from pyspark.sql.types import StructType, StructField, IntegerType

schema = StructType([
    StructField("id", IntegerType(), True)
])

data = [(1,),(2,),(3,)]

df = spark.createDataFrame(data, schema)
chetanhiwale's avatar
chetanhiwale
Resolver I
6 months ago

HI BalajiL , 
Both schemas are same. As the spark is not able to find the synapsesql connector, more or less it seems issue with Spark connector / Spark runtime not with the schema.