Forum Discussion
Incremental Refreshing
- 1 year ago
Yes you can use incremental refresh in your case. In PQ, go to the table with your data and filter the Date Modified column.
Instead of manually selecting dates, you should be able to apply a filter using the RangeStart and RangeEnd parameters. Then choose Custom Filter and set it to filter for records where Date Modified is greater than or equal to RangeStart and less than RangeEnd.After setting the filter in Power Query, close and apply the changes.
Then, set up the incremental refresh for your 2024 data. Define the period for incremental refresh (refresh the last 1 year) and specify how long you want to keep historical data (keep data for 3 years).You can configure how much data to refresh and whether to archive data. For example, you could set it to refresh the data for the past 1 year (which would cover the changing data for 2024) while keeping the historical data for 2022-2023 unchanged.
If you want to be more specific and only refresh the most recent data, you could set this to 1 month if you only expect data changes for the most recent month.
For example, if you update your data on the 1st of each month, setting this to 1 month will ensure that only the last 30 days of 2024 are refreshed, making the refresh process faster. However, this assumes that older data in 2024 does not change.
Update :
The solution was to convert the column to Datetime
Hi AmiraBedh
I just figured out why I couldn't see the parameter option. My date field that I wanted to foliter wasn't set up as Date/Time. OMG. Thanks so much for your help and quick replies. I'm going to try setting it up now.
Thanks
KatieK
If my answer helped you don't forget to accept it 🙂