Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Don't miss out! 2025 Microsoft Fabric Community Conference, March 31 - April 2, Las Vegas, Nevada. Use code MSCUST for a $150 discount. Prices go up February 11th. Register now.

Reply
KH_Mike
Helper III
Helper III

Unclear about incremental refresh and detect data change

Hello All,

 

I would like to apply the incremental refresh on my dataset but there are some unclear points would like to clarify.

 

Backgroup Setting

For the "RangeStart" and "RangeEnd". If I apply it on "Create_Date" and select "Archive Date" starting 1 Year before refresh date. Incrementally Refresh Data 3 months before refresh date.

 

Question 1:

If the daily refresh on Jan-2025, then the "Acrhive Date" will start from 1-Jan-2024 & Incremental Refresh the "Create_Date" between 1-Nov-2024 & 31-Jan-2025. When we are in Feb-2025, then the "Acrhive Date" will start from 1-Feb-2024 & Incremental Refresh the "Create_Date" between 1-Dec-2024 & 28-Feb-2025. Is my understanding correct?

 

Question 2:

If in "Detect Data Change" and select "Last_Edit_Date". Then on 15-Jan-2025, there are two edited records (003 & 005). Then both records will be refreshed or only 005 will be refresh?

 

Rows_NoCreate_DateLast_Edit_DateLast_Edit_Date on 15-Jan-2025
0011-Jan-20242-Jan-20242-Jan-2024
0021-Jun-20242-Jun-20242-Jun-2024
0031-Sep-20242-Sep-202415-Jan-2025
0041-Dec-20242-Dec-20242-Dec-2024
0051-Jan-20242-Jan-202515-Jan-2025

 

Question 3:

If in "Detect Data Change" and select "Last_Edit_Date". Then only the "Last_Edit_Date" have change compare with previous record will be refreshed? The rest will not be refreshed even within the "Incremental Refresh" period?

 

Question 4:

In some article, it mentioned that the RangeStart and before RangeEnd should have an equal to (=) on either RangeStart or RangeEnd, but not both. Otherwise, it will have duplication in records. I don't really understand the detail. Any example can illustrate?

 

Thank you.

2 ACCEPTED SOLUTIONS
Kedar_Pande
Super User
Super User

@KH_Mike 

  1. Question 1
    Yes, your understanding is correct:
  2. Question 2
    If "Detect Data Change" is enabled on Last_Edit_Date, both records (003 and 005) will be refreshed because their Last_Edit_Date changed on 15-Jan-2025.
  3. Question 3
    Only rows where Last_Edit_Date has changed will be refreshed, even if they are within the "Incremental Refresh" period. Rows with unchanged Last_Edit_Date are not refreshed.
  4. Question 4
    To avoid duplication, ensure:

    One boundary (either RangeStart or RangeEnd) includes = (e.g., >= RangeStart and < RangeEnd).
    Example: If you use >= RangeStart and <= RangeEnd, overlapping rows on RangeEnd could result in duplicate data.

💡 If this helped, please give Kudos 👍 or mark it as a Solution .
Best regards,
Kedar
🌐 Connect on LinkedIn

View solution in original post

uzuntasgokberk
Super User
Super User

Hello @KH_Mike ,

 

First of all, ı've written about incremental refresh detailly in my medium post and I belive it will be usefull for you.
https://medium.com/microsoft-power-bi/power-bi-hidden-treasure-incremental-refresh-7e91ac79fc6a

But for you questions:
Q1: Yes
Q2: Both will be refresh.

Q3: When Last_Edit_Date increased, new row will be added based on last_edit_date
Q4: İt is rule that needs to be use like this. Other wise it can be duplicated

uzuntasgokberk_0-1735900495304.png

 

Best Regards,
Gökberk Uzuntaş

LinkedIn: https://www.linkedin.com/in/g%C3%B6kberk-uzunta%C5%9F-b43906198/

Medium: https://medium.com/@uzuntasgokberk

 

İf this post helps, then please consider Accept it as solution and kudos to help the other members find it more quickly.

View solution in original post

3 REPLIES 3
v-stephen-msft
Community Support
Community Support

Hi @KH_Mike ,

 

Thanks for reaching out.

Pls has your problem been solved? If so, accept the reply as a solution. This will make it easier for the future people to find the answer quickly.

If not, please provide a more detailed description, preferably some virtual sample data, and the expected results.

 

Best Regards,

Stephen Tao

 

uzuntasgokberk
Super User
Super User

Hello @KH_Mike ,

 

First of all, ı've written about incremental refresh detailly in my medium post and I belive it will be usefull for you.
https://medium.com/microsoft-power-bi/power-bi-hidden-treasure-incremental-refresh-7e91ac79fc6a

But for you questions:
Q1: Yes
Q2: Both will be refresh.

Q3: When Last_Edit_Date increased, new row will be added based on last_edit_date
Q4: İt is rule that needs to be use like this. Other wise it can be duplicated

uzuntasgokberk_0-1735900495304.png

 

Best Regards,
Gökberk Uzuntaş

LinkedIn: https://www.linkedin.com/in/g%C3%B6kberk-uzunta%C5%9F-b43906198/

Medium: https://medium.com/@uzuntasgokberk

 

İf this post helps, then please consider Accept it as solution and kudos to help the other members find it more quickly.

Kedar_Pande
Super User
Super User

@KH_Mike 

  1. Question 1
    Yes, your understanding is correct:
  2. Question 2
    If "Detect Data Change" is enabled on Last_Edit_Date, both records (003 and 005) will be refreshed because their Last_Edit_Date changed on 15-Jan-2025.
  3. Question 3
    Only rows where Last_Edit_Date has changed will be refreshed, even if they are within the "Incremental Refresh" period. Rows with unchanged Last_Edit_Date are not refreshed.
  4. Question 4
    To avoid duplication, ensure:

    One boundary (either RangeStart or RangeEnd) includes = (e.g., >= RangeStart and < RangeEnd).
    Example: If you use >= RangeStart and <= RangeEnd, overlapping rows on RangeEnd could result in duplicate data.

💡 If this helped, please give Kudos 👍 or mark it as a Solution .
Best regards,
Kedar
🌐 Connect on LinkedIn

Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Prices go up Feb. 11th.

Jan25PBI_Carousel

Power BI Monthly Update - January 2025

Check out the January 2025 Power BI update to learn about new features in Reporting, Modeling, and Data Connectivity.

Jan NL Carousel

Fabric Community Update - January 2025

Find out what's new and trending in the Fabric community.