Vishal_Arora's avatar
Vishal_Arora
New Member
2 years ago
Status:
Needs Votes

Mapping DF Gen2 Incremental Load

Right now in mapping data flows there are two options Append or Replace. Append keeps on adding. And replace just truncate and fill it from scratch. I honestly believe in Append we can have an option for some key/s so that on that specific key or keys basis it should append, because right now just append is not making any sense. We have to integrate the logic inside the mapping data flows use multiple transformations, this was not expected as Fabric basic is to make things simpler but it has make incremental loading complex.


And Secondly for Join, we have the merge condition. Suppose we have around 20+ merges required, then we have to use the transformations 20 times. Right now for this I don't have any solution in mind but if that can be done using one merge then it would be great otherwise DF will be messy and it's pretty difficult to use..

Recent ideas