Incremental Refresh 'Detect Data Changes' Error

[uzuntasgokberk]

Hello community,

I've faced problem about Incremental refresh error in Dataflow. You can follow with the ticket:https://community.fabric.microsoft.com/t5/Service/Incremental-Refresh-Problem/m-p/4260816#M255106

I hope this can be fixed.

Best Regards,
Gökberk Uzuntaş

LinkedIn: https://www.linkedin.com/in/g%C3%B6kberk-uzunta%C5%9F-b43906198/

Medium: https://medium.com/@uzuntasgokberk

[Anonymous]

Hi @uzuntasgokberk ,

Please ensure that the M query for your table includes a filter based on RangeStart and RangeEnd. The column returned by the polling query, such as MaxREFRESH_BOOKMARK, should indicate when the latest data is refreshed. If there's a retro change to the data, the MaxREFRESH_BOOKMARK should differ from the last processed time.

Select Detect data changes to specify a date/time column used to identify and refresh only the days where the data has changed. A date/time column must exist, usually for auditing purposes, at the data source. This column should not be the same column used to partition the data with the RangeStart and RangeEnd parameters. The maximum value of this column is evaluated for each of the periods in the incremental range.

If you need to capture update made to data created/inserted in the year 2022, the incremental refresh policy needs to refresh past 3 years data.

Best regards.
Community Support Team_Caitlyn

[uzuntasgokberk]

Hello @Anonymous,

I've made everything about what you said. Let me show with visualization.

I have below the data in Azure SQL 

Incremental Refresh in Dataflow the importing thing is 'Detect Data Changes' which means when ETL_Date is incresing that means dataflow refresh automatically.

Dataflow gen1 understand ıncremental refresh and made in table

ı added a new row in my Azure SQL. Also you can see change etl_date.

However dataflow isn't automatically refreshed. Also, I checked with connect from Power BI Desktop too. When I check 'Detect Change Data' that means ı don't need to schedule. However, without creating schedule it seems doesn't work.

Kind Regards,
Gökberk Uzuntaş

🔗 Stay Connected:
📘 Medium |
📺 YouTube |
💼 LinkedIn |
📷 Instagram |
🐦 X |
👽 Reddit |
🌐 Website |
🎵 TikTok |