Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
4 years ago
Solved

How to properly implement incremental refresh with a last_update column.

Hi community!

I'm trying to implement the incremental refresh as explained in the microsoft guideline: 

https://docs.microsoft.com/en-us/power-bi/connect-data/incremental-refresh-configure

 

What I want to achieve is to archive the full history and only refresh new data or freshly modified data (could be in the scope of the incremental refresh but also the archive).

 

This is the structure of my table:

date - value1 - value2 - value3 - last_update

 

What would be the best strategy to do this job?

  1. Filter date with RangeStart and RangeEnd and check data changes base on last_update column? or
  2. Filter last_update with RangeStart and RangeEnd and don't check data changes?
  3. Filter last_update with RangeStart and RangeEnd and don't check data changes and check data changes base on last_update column?

Thanks for helping 🙂

Cheers

Interesting Links
Need Help

2 Replies

  • v-cazheng-msft's avatar
    v-cazheng-msft
    Community Support
    4 years ago

    Hi Anonymous,

     

    You may consider ticking the option Detect data changes and the best strategy would be Filter date with RangeStart and RangeEnd, then detect data changes based on last_update.

     

    You could refer to the official doc to learn more about the option Detect data changes.

    https://docs.microsoft.com/en-us/power-bi/connect-data/incremental-refresh-overview?msclkid=bd7bc441cf5c11ec8e7e3f575ccf6256#optional-settings

     

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly. If I misunderstand your needs or you still have problems on it, please feel free to let me know. Thanks a lot!

     

    Best Regards,

    Community Support Team _ Caiyun

  • Anonymous's avatar
    Anonymous
    Not applicable
    4 years ago

    Hey, v-cazheng-msft thanks for your reply.

     

    I've one more question, please.

     

    If

    • the incremental refresh setup is 7 days before the refresh date
    • someone edited the history data (let's say the date of the modified data is 1 month before the refresh).
    • the last_update of this data will turn to today

    Is my incremental refresh will update this value? I'm not sure according to the schema at the bottom of your screenshot and the explanations about the data changes.

     

    Thanks,