Forum Discussion
Helper IUnable to use a hardcoded date column in Delta table merge api
Hi
I can confirm that the merge condition is right, When i remove the date columns from update_columns or create_columns then the merge is successful.
Also when i use the same date or timestamp column in withColumn then i see no issues.
Hi priyankabis ,
It looks like you’re trying to perform a merge operation on a Delta table with hardcoded data, but encountering errors. Here are a few things to check and consider:
Firstly, ensure that the data types and formats of the columns in update_columns and create_columns match those in the Delta table schema. For example, to_date and to_timestamp should be correctly formatted.
The condition in the merge function should be a valid SQL expression. Ensure that target.entity_id = source.merge_key and target.latest = 1 is correctly referencing the columns.
Finally you need to make sure that the column names in update_columns and create_columns are exactly the same as those in the Delta table.
Best Regards
Yilong Zhou
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Helper IHi
I can confirm that the merge condition is right, When i remove the date columns from update_columns or create_columns then the merge is successful.
Also when i use the same date or timestamp column in withColumn then i see no issues.