Forum Discussion
Lakehouse Add or Remove columns from table
Not sure if this is the right forum or not - but here is the issue.
We are loading data to a Lakehouse using gen 2 data flows (for now they are just pointing at exisitn gen 1 dataflows then doi...
coolie
Helper I
Helper Icould you share how to do this in python?
import pyspark.pandas as ps
import pandas as pd
import numpy as np
from pyspark.sql import *
psdf = ps.read_delta('path_to_table')
psdf.head(10)
psdf['new_col'] = ''
psdf.head(10)
sdf = psdf.to_spark()
sdf.write.mode('overwrite').saveAsTable('existing_table')
Once can use a spark dataframe in lieu of pandas.
How ever the easiest option now is to use a SQL notebook and add a column. Please refer