Detecting data changes

[amo1999]

Hello 

We are currently new to Power BI in our organization and we are shiffting the existing reports in our system to Power BI using App Owns Data approche.

I faced an situtation that is a little complicated for my current level in BI, I searched in many places but couldn't find the right answer.

We have a table in the database for the inventory, the table has around 2M record and I want to refresh the data on daily basis, the problem I face is that fetching about 700MB of data everyday looks unsuitable especially that the data is increasing very quickly, I wanted to have a historical table in Power BI that contains all the data to date and another one that contains the daily data and using append quries to have them, but I there any way to detect the old changes for the historical table for example an Item in the stock has been deleted or updated is there any mechanism in the Power BI so it updates the records in the historical table or I have to fetch refresh the full table when any change happens in the old data.

Thank you in advance

[amitchandak]

@amo1999 , Have explored incremental load

https://radacad.com/all-you-need-to-know-about-the-incremental-refresh-in-power-bi-load-changes-only
https://thinkaboutit.be/2020/02/how-do-i-implement-an-incremental-refresh-in-power-bi-free-or-pro/
https://www.nabler.com/articles/power-bi-data-refresh-and-scheduling/

Incremental load usinf dax union

https://blog.crossjoin.co.uk/2020/04/13/keep-the-existing-data-in-your-power-bi-dataset-and-add-new-...
https://www.thebiccountant.com/2017/01/11/incremental-load-in-powerbi-using-dax-union/