Forum Discussion
[Fabric] How can I load incremental data from a PostgreSQL database using Dataflows or Data Factory?
Hi luis_quiroz , to add to the post of Anonymous;
I am a PM on the Data Factory team and focus on Dataflows Gen2.
Within Dataflows Gen2 we will be providing a native way to do incremental refresh, public preview is planned for later this year. In the mean time, we have an other solution that may work for you.
Have you looked into this solution? Pattern to incrementally amass data with Dataflow Gen2 - Microsoft Fabric | Microsoft Learn This solution is designed to check your data destination for the latest value for a specific column and leverage that as a filter on the source to trickle feed changes from your source into the destination, is that something that would help you forward?
Thank you for your response. We appreciate the information provided. We implemented the append method (Pattern to incrementally amass data with Dataflow Gen2 - Microsoft Fabric | Microsoft), but we did not see any difference compared to the replace method when bringing in data from PostgreSQL.
