Forum Discussion

waltheed's avatar
waltheed
Icon for Impactful Individual rankImpactful Individual
3 years ago
Solved

Fabric Data Factory Pipeline: Incremental load

How can I create an incremental load in Data Factory Pipeline?   I only see the Table Actions append or overwrite. But I want to be able to add new rows, update existing ones and mark deleted rows....
Need Help
pipelines
  • ajarora's avatar
    ajarora
    3 years ago

    Your table action options imply that you are using pipelines. If so, good news, we have just added a new feature that allows you to "upsert" the new rows into existing tables instead of append/overwrite only.

    Its in private preview, please ping me with your workspace id, and we can enable it for you along with instructions. cc makromer 

ajarora's avatar
ajarora
Icon for Microsoft Employee rankMicrosoft Employee
3 years ago

Your table action options imply that you are using pipelines. If so, good news, we have just added a new feature that allows you to "upsert" the new rows into existing tables instead of append/overwrite only.

Its in private preview, please ping me with your workspace id, and we can enable it for you along with instructions. cc makromer 

matkvaid's avatar
matkvaid
Icon for Helper III rankHelper III
2 years ago

Hi, is it still possible to try this feature? I have set up incremental refresh with stored procedures etc, but want to have that setup for first step from on prem sql to lakehouse, and would not want to recreate everything again or leave full refreshes for prepared data from lakehouse to warehouse 🙂