Forum Discussion

whgenie's avatar
whgenie
Regular Visitor
1 year ago
Solved

Incremental Refresh sourced from SQL server and only add new rows to an already modified table

Hi I tried searching for answers but can't seem to find out that fits what I'm trying to do.   Currently I have a table in SQL server that auto refreshes daily. I use Power Query to load this tabl...
  • lbendlin's avatar
    1 year ago

    It's called "incremental refresh"  for a reason. It expects immutable data and is not what you need (which would be differential refresh).

     

    Incremental Refresh works on the level of partitions, not on rows.  The lowest operation you can do is flush and fill a partition.  Usually this is the latest partition, but with XMLA tools you can also refresh any other partition.

     

    The Power BI service will not care about data duplicates across partitions. Avoiding or eliminating duplicates is entirely your own responsibility.

  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi,

    Thanks for the solution lbendlin  offered, and i want to offer some more information for user to refer to.

    hello whgenie , as lbendlin  mentioned,  there's no way to keep historical data in Power BI, and once your original data is refreshed, Power BI will only keep your most recent data, not your previous data. and Incremental refresh will only retain the data in the interval after the refresh, and cannot retain the complete data. You can refer to the following link.

    Incremental refresh for semantic models in Power BI - Power BI | Microsoft Learn

     If you want to get the new rows from Table A from the last refresh date/time, and to keep all the data in C, it is better that you create a history table in SQL to keep the history data in table c, then import the data to power BI.

     

    Best Regards!

    Yolo Zhu

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.