Duplicate rows in Lakehouse Table with Primary Key with DataFlow Gen2

Hi lbendlin ,

Thanks for responding.

Due to the nature of the data, I cannot use an Incremental refresh in the way documented since a "ticket" in the ITSM may last accross several refresh cycles, and change state during that time.  For reporting purposes I need to make sure that I have the latest state of the ticket, rather than freezing it on initial import, if that makes sense.

From reading several articles (not specific to DataFlow Gen2) for PowerBI/Fabric, it suggested at setting a Primary Key would result in an UPSERT if a matching entry was found. Is this not the case for Lakehouse/DataFlow Gen2?

For now I am attempting to deduplicate the data via other means, however, if this process can be simplified, that would be much preferred.

Thanks.

JohnB