topic Re: Incremental update using dataflow gen2 in Dataflow

Incremental update using dataflow gen2

trivediasit — Mon, 18 Sep 2023 05:23:28 GMT

I have gone through the above link and have a few questions:

- What will happen in case there is no incremental data? Will the dataflow fail?

- What is the best way to manage the issue?

Look forwrad to suggestions.

Regards

Asit

LuitwielerMSFT — Mon, 18 Sep 2023 13:05:17 GMT

Thanks for your questions on this topic, let me adress those.

If there is no incremental data the dataflow will still refresh fine but just not append any new data to the data destination.
Failures can happen because of many reasons. The great thing is that with failures in the data pipeline you can setup yourself a way to notify or retry the data pipeline. Just link the activity to the failed output in data pipelines and define your own way of trouble shooting your pipeline. Module 3 - Automate and send notifications with Data Factory - Microsoft Fabric | Microsoft Learn