Forum Discussion

Jeanxyz's avatar
Jeanxyz
Power Participant
1 year ago
Solved

how to write a spark dataframe into csv file without losing column headers

I have created a notebook and a spark data frame, when I write the data frame into a csv file, the column header is not written into csv. Is that normal? *** yearlySales = df.select(year(col("Orde...
data engineering
  • nilendraFabric's avatar
    nilendraFabric
    1 year ago

    Hi Jeanxyz 

     

    yearlySales.write.mode("overwrite").option("header", True).format("csv").save("Files/yearlySales.csv")

     

    option(“header”, True): Ensures that the column headers are written as the first row in the CSV output.

     

nilendraFabric's avatar
nilendraFabric
Super User
1 year ago

Hi Jeanxyz 

 

yearlySales.write.mode("overwrite").option("header", True).format("csv").save("Files/yearlySales.csv")

 

option(“header”, True): Ensures that the column headers are written as the first row in the CSV output.