Forum Discussion

apaceAW's avatar
apaceAW
Regular Visitor
1 year ago
Solved

Power BI Incremental Refresh Behavior Question

Hello. I'm trying to understand the behavior of Power BI's incremental refresh functionality. I'm not a Power BI developer but I have a decent understanding of the basics.   Here's the setup in the...
Need Help
  • lbendlin's avatar
    lbendlin
    1 year ago
    Parameters for range start and range end applied to the last_updated_date in the table

    Let me stop you right there.  last_updated_date is not a suitable column for incremental refresh partition policies. This will result in data duplication across partitions.  Instead use an immutable column like created_date.

     

     

lbendlin's avatar
lbendlin
Super User
1 year ago
Parameters for range start and range end applied to the last_updated_date in the table

Let me stop you right there.  last_updated_date is not a suitable column for incremental refresh partition policies. This will result in data duplication across partitions.  Instead use an immutable column like created_date.

 

 

apaceAW's avatar
apaceAW
Regular Visitor
1 year ago

LOL ok that is what I thought and that is what I tried to tell the BI team. 

This leads me to my next question: if the parameters are applied to the created_date instead and the incremental refresh is still set to pull in 7 days and the detect data changes is unchecked, will power BI pull the data from the source by created_date?

 

What I can't figure out is how does Power BI know which data to replace? There's no key column defined so it can't be performing a merge. 

  • lbendlin's avatar
    lbendlin
    Super User
    1 year ago

    There is no merge in any of this. You have partitions. Their queries are managed by the RangeStart and RangeEnd parameters which are controlled by the Power BI service based on your requirements*. 

    They are then appended.

     

    * I shoud say "guidance.  You may specify 7 days but the service will decide for itself how many partitions to actually refresh, which ones to consolidate etc.

     

    https://learn.microsoft.com/en-us/power-bi/connect-data/incremental-refresh-xmla