Forum Discussion
PowerBITestingG
Resolver I
Resolver IDataflow Gen2 incremental refresh alternative to modified date
We need hourly updates in a dataflow that writes to warehouse , but the source has no “modified date,” so Gen2 incremental refresh can’t detect changes. Full table refresh every hour is too heavy. A...
v-veshwara-msft
Community Support
Community SupportHi PowerBITestingG ,
Thanks for posting in the Microsoft Fabric Community.
Your approach works well when there is no modified date to use for incremental refresh. The filters you mentioned for separating historical and recent data are a good way to avoid duplicates, and the union view in the warehouse can combine them.
If your source can have late-arriving records, you can keep a small overlap in the date ranges and then remove any duplicates in the union view using a unique key.
Thanks for sharing here as this helps others in the community, and please reach out for any further assistance.