Forum Discussion

KH_Mike's avatar
KH_Mike
Helper III
1 year ago
Solved

Unclear about incremental refresh and detect data change

Hello All,

 

I would like to apply the incremental refresh on my dataset but there are some unclear points would like to clarify.

 

Backgroup Setting

For the "RangeStart" and "RangeEnd". If I apply it on "Create_Date" and select "Archive Date" starting 1 Year before refresh date. Incrementally Refresh Data 3 months before refresh date.

 

Question 1:

If the daily refresh on Jan-2025, then the "Acrhive Date" will start from 1-Jan-2024 & Incremental Refresh the "Create_Date" between 1-Nov-2024 & 31-Jan-2025. When we are in Feb-2025, then the "Acrhive Date" will start from 1-Feb-2024 & Incremental Refresh the "Create_Date" between 1-Dec-2024 & 28-Feb-2025. Is my understanding correct?

 

Question 2:

If in "Detect Data Change" and select "Last_Edit_Date". Then on 15-Jan-2025, there are two edited records (003 & 005). Then both records will be refreshed or only 005 will be refresh?

 

Rows_NoCreate_DateLast_Edit_DateLast_Edit_Date on 15-Jan-2025
0011-Jan-20242-Jan-20242-Jan-2024
0021-Jun-20242-Jun-20242-Jun-2024
0031-Sep-20242-Sep-202415-Jan-2025
0041-Dec-20242-Dec-20242-Dec-2024
0051-Jan-20242-Jan-202515-Jan-2025

 

Question 3:

If in "Detect Data Change" and select "Last_Edit_Date". Then only the "Last_Edit_Date" have change compare with previous record will be refreshed? The rest will not be refreshed even within the "Incremental Refresh" period?

 

Question 4:

In some article, it mentioned that the RangeStart and before RangeEnd should have an equal to (=) on either RangeStart or RangeEnd, but not both. Otherwise, it will have duplication in records. I don't really understand the detail. Any example can illustrate?

 

Thank you.

Need Help
  • Kedar_Pande's avatar
    Kedar_Pande
    1 year ago

    KH_Mike 

    1. Question 1
      Yes, your understanding is correct:
    2. Question 2
      If "Detect Data Change" is enabled on Last_Edit_Date, both records (003 and 005) will be refreshed because their Last_Edit_Date changed on 15-Jan-2025.
    3. Question 3
      Only rows where Last_Edit_Date has changed will be refreshed, even if they are within the "Incremental Refresh" period. Rows with unchanged Last_Edit_Date are not refreshed.

    4. Question 4
      To avoid duplication, ensure:

      One boundary (either RangeStart or RangeEnd) includes = (e.g., >= RangeStart and < RangeEnd).
      Example: If you use >= RangeStart and <= RangeEnd, overlapping rows on RangeEnd could result in duplicate data.

    ๐Ÿ’ก If this helped, please give Kudos ๐Ÿ‘ or mark it as a Solution โœ….
    Best regards,
    Kedar
    ๐ŸŒ Connect on LinkedIn
  • uzuntasgokberk's avatar
    uzuntasgokberk
    1 year ago

    Hello KH_Mike ,

     

    First of all, ฤฑ've written about incremental refresh detailly in my medium post and I belive it will be usefull for you.
    https://medium.com/microsoft-power-bi/power-bi-hidden-treasure-incremental-refresh-7e91ac79fc6a

    But for you questions:
    Q1: Yes
    Q2: Both will be refresh.

    Q3: When Last_Edit_Date increased, new row will be added based on last_edit_date
    Q4: ฤฐt is rule that needs to be use like this. Other wise it can be duplicated

     

    Kind Regards,
    Gรถkberk UzuntaลŸ

    ๐Ÿ“Œ If this post helps, then please consider Accepting it as a solution and giving Kudos โ€” it helps other members find answers faster!

    ๐Ÿ”— Stay Connected:
    ๐Ÿ“˜ Medium |
    ๐Ÿ“บ YouTube |
    ๐Ÿ’ผ LinkedIn |
    ๐Ÿ“ท Instagram |
    ๐Ÿฆ X |
    ๐Ÿ‘ฝ Reddit |
    ๐ŸŒ Website |
    ๐ŸŽต TikTok |

3 Replies

  • Kedar_Pande's avatar
    Kedar_Pande
    Super User
    1 year ago

    KH_Mike 

    1. Question 1
      Yes, your understanding is correct:
    2. Question 2
      If "Detect Data Change" is enabled on Last_Edit_Date, both records (003 and 005) will be refreshed because their Last_Edit_Date changed on 15-Jan-2025.
    3. Question 3
      Only rows where Last_Edit_Date has changed will be refreshed, even if they are within the "Incremental Refresh" period. Rows with unchanged Last_Edit_Date are not refreshed.

    4. Question 4
      To avoid duplication, ensure:

      One boundary (either RangeStart or RangeEnd) includes = (e.g., >= RangeStart and < RangeEnd).
      Example: If you use >= RangeStart and <= RangeEnd, overlapping rows on RangeEnd could result in duplicate data.

    ๐Ÿ’ก If this helped, please give Kudos ๐Ÿ‘ or mark it as a Solution โœ….
    Best regards,
    Kedar
    ๐ŸŒ Connect on LinkedIn
  • uzuntasgokberk's avatar
    uzuntasgokberk
    Super User
    1 year ago

    Hello KH_Mike ,

     

    First of all, ฤฑ've written about incremental refresh detailly in my medium post and I belive it will be usefull for you.
    https://medium.com/microsoft-power-bi/power-bi-hidden-treasure-incremental-refresh-7e91ac79fc6a

    But for you questions:
    Q1: Yes
    Q2: Both will be refresh.

    Q3: When Last_Edit_Date increased, new row will be added based on last_edit_date
    Q4: ฤฐt is rule that needs to be use like this. Other wise it can be duplicated

     

    Kind Regards,
    Gรถkberk UzuntaลŸ

    ๐Ÿ“Œ If this post helps, then please consider Accepting it as a solution and giving Kudos โ€” it helps other members find answers faster!

    ๐Ÿ”— Stay Connected:
    ๐Ÿ“˜ Medium |
    ๐Ÿ“บ YouTube |
    ๐Ÿ’ผ LinkedIn |
    ๐Ÿ“ท Instagram |
    ๐Ÿฆ X |
    ๐Ÿ‘ฝ Reddit |
    ๐ŸŒ Website |
    ๐ŸŽต TikTok |

  • Anonymous's avatar
    Anonymous
    Not applicable
    1 year ago

    Hi KH_Mike ,

     

    Thanks for reaching out.

    Pls has your problem been solved? If so, accept the reply as a solution. This will make it easier for the future people to find the answer quickly.

    If not, please provide a more detailed description, preferably some virtual sample data, and the expected results.

     

    Best Regards,

    Stephen Tao