Forum Discussion
Dataflow Gen2 Incremental Refresh and Data Refresh
Hi!
My name is Jeroen Luitwieler, and I’m a Senior Product Manager on the Dataflow team. While this guide shows how to use an OData source, it’s just an example. You can use this method with your on-premises SQL Server without any issues. If you want to perform a full refresh of your data every time you refresh, you can change the current data destination setting from “append” to “replace”.
Hope this resolves your issue! 😁
Hi!
My name is Jeroen Luitwieler, and I’m a Senior Product Manager on the Dataflow team. While this guide shows how to use an OData source, it’s just an example. You can use this method with your on-premises SQL Server without any issues. If you want to perform a full refresh of your data every time you refresh, you can change the current data destination setting from “append” to “replace”.
Hope this resolves your issue! 😁
I can confirm that it works - with that example i made incremental refreshes from on prem sql server: query warehouse for max date (or entry no, if Your tables have them) and filter source sql server, check that it works correctly and query folding works. I have also implemented pipelines and stored procedures to delete some data from warehouse and refresh a week or a month old of data.