Basically, we should be able to edit the final output (data destination) columns directly in the Dataflow Gen2, without needing to go to the Warehouse itself to add the new columns (using T-SQL).
This should be possible to do in the Dataflow user interface both for the Append and Replace options.
The settings in the dataflow outputs should be applied to the destination (Lakehouse, Warehouse) without needing to go into the Warehouse or Lakehouse to add the columns.
This should work regardless of writing to a New or Existing table.
This should just work.
We should be able to choose which columns to write to the destination, and add more columns to an existing table, directly in the Dataflow user interface, without having to go to the Lakehouse or Warehouse to make any changes.
2 Comments
- For Replace mode: use similar logic as Spark .mode("overwrite").option("overwriteSchema", "true"). For Append mode: use similar logic as Spark .mode("append").option("mergeSchema", "true")
- Status changed:NewtoNeeds Votes
