Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The ultimate Microsoft Fabric, Power BI, Azure AI & SQL learning event! Join us in Las Vegas from March 26-28, 2024. Use code MSCUST for a $100 discount. Register Now

Reply
Narukkp
Helper V
Helper V

Updated record behavior in data flow incremental refresh

Hi Team,

 

I have a doubt on Power bi Dataflow incremental refresh, If we enable the incremental refresh on dataflow how it will handle the updated records i.e. if suppose I have one employee and his designation has software engineer and after some days he got promoted to senior software engineer, now dataflow will delete the existing record and insert the new record or updated the existing employee record with updated designation ?

 

Why I have got the doubt  here  we do not have any primary key concept and having only dates filed, so that how dataflow will understand the records is either updated record or inserted record ?

1 ACCEPTED SOLUTION

Hi @Narukkp ,

 

For the existing record, the refresh process will update it, not delete it and add a new record at the end of the table.

q6.PNGq7.PNG

q8.PNGq9.PNG

 

Best regards,
Lionel Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

5 REPLIES 5
v-lionel-msft
Community Support
Community Support

Hi @Narukkp ,

 

The refresh mechanism in Power BI just returns the fact data of the data source or the query results to the dataset in the .pbix file. Whether the data is updated or inserted depends on the data table of the data source.

The refresh strategy is just a mover between the data source and the dataset. This process of moving data is similar to the copy + overlay method.

 

Best regards,
Lionel Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

 

Hi,
@v-lionel-msft  thank you for response but here my ask is internally power bi how it will do  i mean for existing record it will delete and insert the new record or just update the existing record ?

Hi @Narukkp ,

 

For the existing record, the refresh process will update it, not delete it and add a new record at the end of the table.

q6.PNGq7.PNG

q8.PNGq9.PNG

 

Best regards,
Lionel Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

But for me its appending one more record at the end instead of updating the existing record.

scenario : I'm using SQL server and set incremental on abc table ( one month History load and 1 week incremental)

                Updated one record in source before refresh run.

                one record is appended to dataset after refresh don.

 

Please make it as solution is it seems to be.

Thank you,

Vara

 

 

 

lbendlin
Super User
Super User

Incremental refresh does not handle late updates well. It is designed for data that is immutable after being written.

 

Until the service supports  full refresh requests of incremental data sources your only remedy at the moment is to initiate a refresh from the desktop which will in turn force a full first refresh of the incremental data source.

Helpful resources

Announcements
Fabric Community Conference

Microsoft Fabric Community Conference

Join us at our first-ever Microsoft Fabric Community Conference, March 26-28, 2024 in Las Vegas with 100+ sessions by community experts and Microsoft engineering.

February 2024 Update Carousel

Power BI Monthly Update - February 2024

Check out the February 2024 Power BI update to learn about new features.

Fabric Career Hub

Microsoft Fabric Career Hub

Explore career paths and learn resources in Fabric.

Fabric Partner Community

Microsoft Fabric Partner Community

Engage with the Fabric engineering team, hear of product updates, business opportunities, and resources in the Fabric Partner Community.

Top Solution Authors
Top Kudoed Authors