Questions about incremental refresh behavior

[Lullabit13]

Hello! a need help with some questions about incremental refresh... and how it works in certain escenarios

Let's assume we have a sales table named Sales with the following fields:

ID CreationDate ModificationDate Customer Product Amount Status

101	01-May-2024	01-May-2024	Juan	Laptop	1000€	Active
102	05-May-2024	05-May-2024	María	Phone	500€	Active
103	10-May-2024	10-May-2024	Pedro	Tablet	700€	Active

The Power BI incremental refresh configuration is:

• Refresh window: Last 10 days.
• Historical data retention: Last 2 years.
• Date column used for incremental refresh: ModificationDate.

In this scenario, when a new sale is generated, both CreationDate and ModificationDate have the same value, reflecting the day the record was created.

---

Question 1: What happens when the date column used for incremental refresh is modified after the record is already in the historical partition?

1. Today (February 23, 2025): A new sale with ID = 201 is created.

   • Its CreationDate and ModificationDate are February 23, 2025, so it falls within the refresh window and is included in Power BI.

2. In 3 months (May 23, 2025): The record is updated to correct an error in the Amount field.

   • Now, the ModificationDate changes to May 23, 2025.

ID CreationDate ModificationDate Customer Product Amount Status

201

23-Feb-2025

23-May-2025

Carlos

Monitor

400€ ⬅️ Updated

Active

Question:
🔹 Since the record was already in the historical partition, will Power BI detect the change in ModificationDate and update the record?
🔹 Will the previous version of the record be overwritten, or will a new record be added to the dataset?
🔹 I know I should ideally use CreationDate, but let's assume that for some reason, whichever date we choose for incremental refresh has changed. How does this impact the dataset update?

---

Question 2: What happens when a record within the refresh window is deleted?

1. Today (February 23, 2025): A new sale with ID = 301 is created, with CreationDate = ModificationDate = February 23, 2025.

   • ✅ The record is included in Power BI because it falls within the refresh window.

2. In 3 days (February 26, 2025): The sale is canceled and deleted from the database.

3. In 4 days (February 27, 2025): The incremental refresh runs.

Question:
🔹 Will Power BI remove the record from the dataset since it no longer exists in the database?
🔹 Will the record still exist in Power BI even though it has been deleted from the source?
🔹 On the other hand, I understand that if I delete a record whose date falls within the historical partition, it will not be removed from my dataset. Is that correct?

---

Summary of Questions

1️⃣ If the date column used for incremental refresh is modified after a record has already moved to the historical partition, how does Power BI handle this?

• Does it detect and update the record?
• Does it overwrite the previous version or add a new one?
• What happens if any date column used for incremental refresh changes, regardless of whether it's CreationDate or another field?

2️⃣ If a record within the refresh window is deleted from the source database, does Power BI remove it from the dataset?

• If not, does this mean records deleted from the source but outside the refresh window will always remain in Power BI?
  
  

THANK YOU in advance, this is killing me 

[Anonymous]

Hi @Lullabit13,

I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please "Accept  as  Solution" and give a 'Kudos' so other members can easily find it.

Thank you,
Pavan.

[Anonymous]

Hi @Lullabit13,

I hope this information is helpful. Please let me know if you have any further questions or if you'd like to discuss this further. If this answers your question, kindly "Accept  as  Solution" and give it a 'Kudos' so others can find it easily.

Thank you,
Pavan.

[Anonymous]

Hi @Lullabit13,

I wanted to follow up since we haven't heard back from you regarding our last response. We hope your issue has been resolved.
If the community member's answer your query, please mark it as "Accept as Solution" and select "Yes" if it was helpful.
If you need any further assistance, feel free to reach out.

Please continue using Microsoft community forum.

Thank you,
Pavan.

[GilbertQ]

Hi @Lullabit13 

If a record has already moved to historical partition, the updated data will not be updated because historical partitions are not automatically processed. You can put a process in place in order to then reprocess those historical partitions.

You have to use a single date column for incremental refresh, and can't go changing the date columns because potentially you could have overlapping data. What's then means you have duplicate data.

There is an option to use detect data changes, which will allow you then to reprocess partitions where the data has changed. Here are more details. Incremental refresh for semantic models in Power BI - Power BI | Microsoft Learn

[Lullabit13]

Hi @GilbertQ 

I get that switching between different date columns for incremental refresh can cause overlapping data and duplicates. But what if I’m using the same column and a record that was already in the historical partition gets a new date? (e.g from 2024-12-31 to 2025-02-24) Would Power BI ignore it, or would it get pulled back into the dataset?

Also, if a record within the refresh window gets deleted from the source database, does Power BI remove it in the next refresh? I think it does since it drops and reloads the partition. But if I turn on the 'Detect Data Changes' option, my understanding is that the deleted record would stay in Power BI since it only updates partitions where changes are detected. Is that right?

[Anonymous · Accepted solution]

Hi @Lullabit13,

Thank you for reaching out in Microsoft Community Forum.

Thank you @GilbertQ for the helpful response.

1. Historical records do not update automatically, If a record moves to a historical partition and its date changes later, Power BI will not detect it unless you manually refresh historical partitions (via XMLA in Premium).

2. Deleted records within the refresh window are removed automatically, as Power BI drops and reloads those partitions. However, records outside the refresh window remain in Power BI, since historical partitions are never refreshed automatically.

3. 'Detect Data Changes' does not track deletions. It only refreshes partitions where changes are detected, meaning deleted records may remain in Power BI unless the entire partition is refreshed.

Please continue using Microsoft community forum.

If you found this post helpful, please consider marking it as "Accept as Solution" and give it a 'Kudos'. if it was helpful. help other members find it more easily.

Regards,
Pavan.