Forum Discussion

Ostrzak's avatar
Ostrzak
Helper II
2 years ago
Solved

Overwriting a csv using Spark notebook creates artifacts

Hi everyone,   It might be a bit stupid of a question, but is there a way to overwrite csv files in a lakehouse (using Pyspark notebook) without creation of additional folders/artifacts?   Right ...
data engineering
  • AndyDDC's avatar
    AndyDDC
    2 years ago

    Hi Ostrzak what is the value in "file_path"?  If I specify a Files folder to save the CSV to, it replaces the current CSV with a new version - changing the CSV filename in the process.  But the old one has been removed.

     

     

     

     

AndyDDC's avatar
AndyDDC
Most Valuable Professional
2 years ago

Hi Ostrzak what is the value in "file_path"?  If I specify a Files folder to save the CSV to, it replaces the current CSV with a new version - changing the CSV filename in the process.  But the old one has been removed.