Forum Discussion
Solved
upsert in lakehouse table using python not pyspark
Hi, I have a small dataset that I am using in my project and trying to perform upserts in a lakehouse table using python in fabric notebooks, but couldn't figure out a way to do that. I can overwrit...
I am using the below code and it is working,
if os.path.exists("/lakehouse/default/Tables/test_pandas") :dt = DeltaTable("/lakehouse/default/Tables/test_pandas",storage_options={"allow_unsafe_rename": "true"})(dt.merge(source=expanded_df,predicate="s.row_hash = t.row_hash",source_alias="s",target_alias="t",).when_matched_update_all().when_not_matched_insert_all().execute())print("Table updated : test_pandas")else:write_deltalake(table_path, expanded_df, mode = 'overwrite', schema_mode='merge', engine = 'rust', storage_options=storage_options)print("Table created for the first time")Thanks
I am using the below code and it is working,
if os.path.exists("/lakehouse/default/Tables/test_pandas") :
dt = DeltaTable("/lakehouse/default/Tables/test_pandas",storage_options={"allow_unsafe_rename": "true"})
(
dt.merge(
source=expanded_df,
predicate="s.row_hash = t.row_hash",
source_alias="s",
target_alias="t",
)
.when_matched_update_all()
.when_not_matched_insert_all()
.execute()
)
print("Table updated : test_pandas")
else:
write_deltalake(table_path, expanded_df, mode = 'overwrite', schema_mode='merge', engine = 'rust', storage_options=storage_options)
print("Table created for the first time")
Thanks
Hi fabric_1 ,
We really appreciate your efforts and for letting us know the update on the issue.
Please continue using fabric community forum for your further assistance.
If this is the solution that has worked for you please accept your reply as solution so as to help other community members who may face similar issue in the future.
Thank you for reaching out to us on the Microsoft Fabric Community Forum.
