Unclear about incremental refresh and detect data change

[KH_Mike]

Hello All,

I would like to apply the incremental refresh on my dataset but there are some unclear points would like to clarify.

Backgroup Setting

For the "RangeStart" and "RangeEnd". If I apply it on "Create_Date" and select "Archive Date" starting 1 Year before refresh date. Incrementally Refresh Data 3 months before refresh date.

Question 1:

If the daily refresh on Jan-2025, then the "Acrhive Date" will start from 1-Jan-2024 & Incremental Refresh the "Create_Date" between 1-Nov-2024 & 31-Jan-2025. When we are in Feb-2025, then the "Acrhive Date" will start from 1-Feb-2024 & Incremental Refresh the "Create_Date" between 1-Dec-2024 & 28-Feb-2025. Is my understanding correct?

Question 2:

If in "Detect Data Change" and select "Last_Edit_Date". Then on 15-Jan-2025, there are two edited records (003 & 005). Then both records will be refreshed or only 005 will be refresh?

Rows_No	Create_Date	Last_Edit_Date	Last_Edit_Date on 15-Jan-2025
001	1-Jan-2024	2-Jan-2024	2-Jan-2024
002	1-Jun-2024	2-Jun-2024	2-Jun-2024
003	1-Sep-2024	2-Sep-2024	15-Jan-2025
004	1-Dec-2024	2-Dec-2024	2-Dec-2024
005	1-Jan-2024	2-Jan-2025	15-Jan-2025

Question 3:

If in "Detect Data Change" and select "Last_Edit_Date". Then only the "Last_Edit_Date" have change compare with previous record will be refreshed? The rest will not be refreshed even within the "Incremental Refresh" period?

Question 4:

In some article, it mentioned that the RangeStart and before RangeEnd should have an equal to (=) on either RangeStart or RangeEnd, but not both. Otherwise, it will have duplication in records. I don't really understand the detail. Any example can illustrate?

Thank you.

[Kedar_Pande · Accepted solution]

@KH_Mike 

1. Question 1
   Yes, your understanding is correct:
2. Question 2
   If "Detect Data Change" is enabled on Last_Edit_Date, both records (003 and 005) will be refreshed because their Last_Edit_Date changed on 15-Jan-2025.
3. Question 3
   Only rows where Last_Edit_Date has changed will be refreshed, even if they are within the "Incremental Refresh" period. Rows with unchanged Last_Edit_Date are not refreshed.

4. Question 4
   To avoid duplication, ensure:

   One boundary (either RangeStart or RangeEnd) includes = (e.g., >= RangeStart and < RangeEnd).
   Example: If you use >= RangeStart and <= RangeEnd, overlapping rows on RangeEnd could result in duplicate data.

💡 If this helped, please give Kudos 👍 or mark it as a Solution ✅.
Best regards,
Kedar
🌐 Connect on LinkedIn

[Anonymous]

Hi @KH_Mike ,

Thanks for reaching out.

Pls has your problem been solved? If so, accept the reply as a solution. This will make it easier for the future people to find the answer quickly.

If not, please provide a more detailed description, preferably some virtual sample data, and the expected results.

Best Regards,

Stephen Tao

[uzuntasgokberk · Accepted solution]

Hello @KH_Mike ,

First of all, ı've written about incremental refresh detailly in my medium post and I belive it will be usefull for you.
https://medium.com/microsoft-power-bi/power-bi-hidden-treasure-incremental-refresh-7e91ac79fc6a

But for you questions:
Q1: Yes
Q2: Both will be refresh.

Q3: When Last_Edit_Date increased, new row will be added based on last_edit_date
Q4: İt is rule that needs to be use like this. Other wise it can be duplicated

Kind Regards,
Gökberk Uzuntaş

📌 If this post helps, then please consider Accepting it as a solution and giving Kudos — it helps other members find answers faster!

🔗 Stay Connected:
📘 Medium |
📺 YouTube |
💼 LinkedIn |
📷 Instagram |
🐦 X |
👽 Reddit |
🌐 Website |
🎵 TikTok |

[Kedar_Pande · Accepted solution]

@KH_Mike 

1. Question 1
   Yes, your understanding is correct:
2. Question 2
   If "Detect Data Change" is enabled on Last_Edit_Date, both records (003 and 005) will be refreshed because their Last_Edit_Date changed on 15-Jan-2025.
3. Question 3
   Only rows where Last_Edit_Date has changed will be refreshed, even if they are within the "Incremental Refresh" period. Rows with unchanged Last_Edit_Date are not refreshed.

4. Question 4
   To avoid duplication, ensure:

   One boundary (either RangeStart or RangeEnd) includes = (e.g., >= RangeStart and < RangeEnd).
   Example: If you use >= RangeStart and <= RangeEnd, overlapping rows on RangeEnd could result in duplicate data.

💡 If this helped, please give Kudos 👍 or mark it as a Solution ✅.
Best regards,
Kedar
🌐 Connect on LinkedIn