Forum Discussion

KH_Mike's avatar
KH_Mike
Icon for Helper III rankHelper III
1 year ago
Solved

Unclear about incremental refresh and detect data change

Hello All,

 

I would like to apply the incremental refresh on my dataset but there are some unclear points would like to clarify.

 

Backgroup Setting

For the "RangeStart" and "RangeEnd". If I apply it on "Create_Date" and select "Archive Date" starting 1 Year before refresh date. Incrementally Refresh Data 3 months before refresh date.

 

Question 1:

If the daily refresh on Jan-2025, then the "Acrhive Date" will start from 1-Jan-2024 & Incremental Refresh the "Create_Date" between 1-Nov-2024 & 31-Jan-2025. When we are in Feb-2025, then the "Acrhive Date" will start from 1-Feb-2024 & Incremental Refresh the "Create_Date" between 1-Dec-2024 & 28-Feb-2025. Is my understanding correct?

 

Question 2:

If in "Detect Data Change" and select "Last_Edit_Date". Then on 15-Jan-2025, there are two edited records (003 & 005). Then both records will be refreshed or only 005 will be refresh?

 

Rows_NoCreate_DateLast_Edit_DateLast_Edit_Date on 15-Jan-2025
0011-Jan-20242-Jan-20242-Jan-2024
0021-Jun-20242-Jun-20242-Jun-2024
0031-Sep-20242-Sep-202415-Jan-2025
0041-Dec-20242-Dec-20242-Dec-2024
0051-Jan-20242-Jan-202515-Jan-2025

 

Question 3:

If in "Detect Data Change" and select "Last_Edit_Date". Then only the "Last_Edit_Date" have change compare with previous record will be refreshed? The rest will not be refreshed even within the "Incremental Refresh" period?

 

Question 4:

In some article, it mentioned that the RangeStart and before RangeEnd should have an equal to (=) on either RangeStart or RangeEnd, but not both. Otherwise, it will have duplication in records. I don't really understand the detail. Any example can illustrate?

 

Thank you.

Need Help
  • Kedar_Pande's avatar
    Kedar_Pande
    1 year ago

    KH_Mike 

    1. Question 1
      Yes, your understanding is correct:
    2. Question 2
      If "Detect Data Change" is enabled on Last_Edit_Date, both records (003 and 005) will be refreshed because their Last_Edit_Date changed on 15-Jan-2025.
    3. Question 3
      Only rows where Last_Edit_Date has changed will be refreshed, even if they are within the "Incremental Refresh" period. Rows with unchanged Last_Edit_Date are not refreshed.

    4. Question 4
      To avoid duplication, ensure:

      One boundary (either RangeStart or RangeEnd) includes = (e.g., >= RangeStart and < RangeEnd).
      Example: If you use >= RangeStart and <= RangeEnd, overlapping rows on RangeEnd could result in duplicate data.

    💡 If this helped, please give Kudos 👍 or mark it as a Solution ✅.
    Best regards,
    Kedar
    🌐 Connect on LinkedIn
  • uzuntasgokberk's avatar
    uzuntasgokberk
    1 year ago

    Hello KH_Mike ,

     

    First of all, ı've written about incremental refresh detailly in my medium post and I belive it will be usefull for you.
    https://medium.com/microsoft-power-bi/power-bi-hidden-treasure-incremental-refresh-7e91ac79fc6a

    But for you questions:
    Q1: Yes
    Q2: Both will be refresh.

    Q3: When Last_Edit_Date increased, new row will be added based on last_edit_date
    Q4: İt is rule that needs to be use like this. Other wise it can be duplicated

     

    Kind Regards,
    Gökberk Uzuntaş

    📌 If this post helps, then please consider Accepting it as a solution and giving Kudos — it helps other members find answers faster!

    🔗 Stay Connected:
    📘 Medium |
    📺 YouTube |
    💼 LinkedIn |
    📷 Instagram |
    🐦 X |
    👽 Reddit |
    🌐 Website |
    🎵 TikTok |

3 Replies

  • Kedar_Pande's avatar
    Kedar_Pande
    Icon for Super User rankSuper User
    1 year ago

    KH_Mike 

    1. Question 1
      Yes, your understanding is correct:
    2. Question 2
      If "Detect Data Change" is enabled on Last_Edit_Date, both records (003 and 005) will be refreshed because their Last_Edit_Date changed on 15-Jan-2025.
    3. Question 3
      Only rows where Last_Edit_Date has changed will be refreshed, even if they are within the "Incremental Refresh" period. Rows with unchanged Last_Edit_Date are not refreshed.

    4. Question 4
      To avoid duplication, ensure:

      One boundary (either RangeStart or RangeEnd) includes = (e.g., >= RangeStart and < RangeEnd).
      Example: If you use >= RangeStart and <= RangeEnd, overlapping rows on RangeEnd could result in duplicate data.

    💡 If this helped, please give Kudos 👍 or mark it as a Solution ✅.
    Best regards,
    Kedar
    🌐 Connect on LinkedIn
  • uzuntasgokberk's avatar
    uzuntasgokberk
    Icon for Super User rankSuper User
    1 year ago

    Hello KH_Mike ,

     

    First of all, ı've written about incremental refresh detailly in my medium post and I belive it will be usefull for you.
    https://medium.com/microsoft-power-bi/power-bi-hidden-treasure-incremental-refresh-7e91ac79fc6a

    But for you questions:
    Q1: Yes
    Q2: Both will be refresh.

    Q3: When Last_Edit_Date increased, new row will be added based on last_edit_date
    Q4: İt is rule that needs to be use like this. Other wise it can be duplicated

     

    Kind Regards,
    Gökberk Uzuntaş

    📌 If this post helps, then please consider Accepting it as a solution and giving Kudos — it helps other members find answers faster!

    🔗 Stay Connected:
    📘 Medium |
    📺 YouTube |
    💼 LinkedIn |
    📷 Instagram |
    🐦 X |
    👽 Reddit |
    🌐 Website |
    🎵 TikTok |

  • Anonymous's avatar
    Anonymous
    Not applicable
    1 year ago

    Hi KH_Mike ,

     

    Thanks for reaching out.

    Pls has your problem been solved? If so, accept the reply as a solution. This will make it easier for the future people to find the answer quickly.

    If not, please provide a more detailed description, preferably some virtual sample data, and the expected results.

     

    Best Regards,

    Stephen Tao