Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Dataflows to Lakehouse does not overwrite table, but only the records existing in the new table

Dataflows to Lakehouse (replace) does not overwrite table (first delete all records, then append new ones) , but only the records existing in the new table will be updated. Is this a bug or expected?...
Dataflows
Anonymous's avatar
Anonymous
Not applicable
2 years ago

Well it doesn't replace all the data, because some of it still exists. Apparently sometimes it does, but in my case it didn't.

frithjof_v's avatar
frithjof_v
Icon for Community Champion rankCommunity Champion
2 years ago

Are the unwanted rows visible in the SQL Analytics Endpoint, Lakehouse Explorer and Notebook?

 

Or are the unwanted rows only visible in some of these user interfaces?

 

I had an experience where Notebook was showing me some old rows which should have been deleted by Dataflow Gen2 replace.

However the SQL Analytics Endpoint was showing me the correct rows only.

  • miguel's avatar
    miguel
    Icon for Community Admin rankCommunity Admin
    2 years ago

    the scenario that you're bringing up might be unrelated to the discussion of this thread. I'd recommend opening up a new topic / thread and share some repro steps to see the behavior or reaching out to our customer support team by opening a support ticket so an engineer can take a closer look at your specific scenario.