Forum Discussion

mkjit256's avatar
mkjit256
Advocate II
1 year ago
Solved

Concatenate using || failing in pyspark spark.sql(sql_string)

Hello  I am trying to execute the following code in Fabric notebook (Pyspark cell): sql_string = """ INSERT INTO dummy_Table (Schema_Name, Table_Name, SQL_Statement) ('schema1', 'table1', "SELECT ...
  • SachinNandanwar's avatar
    1 year ago

    Try either of this 

    sql_string = """
    INSERT INTO dummy_Table (Schema_Name, Table_Name, SQL_Statement)
    VALUES ('schema1', 'table1', "SELECT col1, col2, col3, col4, col5, col6, col5 || '_' || col6 as con_col FROM schema1.table1")"""
    
    spark.sql(sql_string)

     

    sql_string = """
    INSERT INTO dummy_Table (Schema_Name, Table_Name, SQL_Statement)
    VALUES ('schema1', 'table1', 'SELECT col1, col2, col3, col4, col5, col6, col5 || ''_'' || col6 as con_col FROM schema1.table1')"""
    
    spark.sql(sql_string)
  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi mkjit256 ,

     

    Thanks for the reply from SachinNandanwar .

     

    Please try this code, it works fine for me:

     

    Replace the schema name, table name, and column name with your own.

    sql_string = “””
    INSERT INTO dummy_Table (Schema_Name, Table_Name, SQL_Statement)
    VALUES ('dbo', 'products', “SELECT ProductID, ProductName, Category, ListPrice, Date, Month, Date || ‘_’) | | Month as con_col FROM dbo.products")
    “"”
    
    spark.sql(sql_string)

     

    The values keyword is used instead of inserting values directly.

     

    If you have any other questions please feel free to contact me.

     

    Best Regards,
    Yang
    Community Support Team

     

    If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
    If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!