Forum Discussion

cw88's avatar
cw88
Icon for Helper IV rankHelper IV
1 year ago
Solved

Notebook: Write data from snowflake (oauth) to lakehouse

I try to get data from snowflake (with oauth authentication) and write it to the default lakehouse with Pyspark:   conn = http.client.HTTPSConnection("XXX") payload = f"XXX" headers = {'content-t...
data engineering
lakehouse
notebook
Anonymous's avatar
Anonymous
Not applicable
1 year ago

HI cw88,

Did any changes or update that applied to the credentials you used? How long did these operation processed? BTW, how did you configure the spark session and time out? Can you please share some more detail?

Regards,

Xiaoxin Sheng

cw88's avatar
cw88
Icon for Helper IV rankHelper IV
1 year ago

hello Anonymous ,

Did any changes or update that applied to the credentials you used? --> No

How long did these operation processed? --> Processing takes 2-3 seconds

BTW, how did you configure the spark session and time out? Can you please share some more detail? --> No custom settings

 

  • Anonymous's avatar
    Anonymous
    Not applicable
    1 year ago

    HI cw88,

    It seems like nothing special changes on your configurations.
    For this scenario, I suppose this may related to the http.client method. Have you tried to other methods(e.g. snowflake.connector) or data driver(e.g. jdbc) to getting data from snowflake?

    Regards,

    Xiaoxin Sheng