Forum Discussion

ToddChitt's avatar
ToddChitt
Icon for Super User rankSuper User
2 years ago
Solved

PySpark to rename a column in a dataframe

Hello. I am quite new to Spark Notebooks. I am using one to extract JSON data to save to tables in a Lakehouse. It works, but there are some slight issues. The data, being JSON, has nexted objects. I...
data engineering
  • AndyDDC's avatar
    AndyDDC
    2 years ago

    Hi ToddChitt @can you try aliasing the column when using .select

     

    df_renamed = df.select(col("Name").alias("EmployeeName"), col("Department").alias("Dept")) 
df_renamed.show()
AndyDDC's avatar
AndyDDC
Icon for Most Valuable Professional rankMost Valuable Professional
2 years ago

Hi ToddChitt @can you try aliasing the column when using .select

 

df_renamed = df.select(col("Name").alias("EmployeeName"), col("Department").alias("Dept")) 
df_renamed.show()