Forum Discussion
Creating an updating database
Dear community,
I have a data source which overwrites itself each day with latest data. What i want to do is use PowerBI to extract the data, save it into a database, then the next day, collect the "new" overwritten data but only pick up any rows of data which have changed (there is a last modified field i can pull from). I want to then append this onto the bottom of my ever-growing database.
Previously i worked with Qlikview where i would have saved the first file as a database and then loaded from this first, to see if anything had changed but i cant see how to do this in powerBI - help please!
Best regards,
Drew
2 Replies
Hi DrewCarousel ,
The only way I think you can do this with Power BI is by using incremental refresh in the Power BI Service.
You need a Premium Capacity or Premium-per-User licence.
More info:
https://docs.microsoft.com/en-us/power-bi/connect-data/incremental-refresh-overview
Other than this, it needs to be done within your data warehouse.
Pete
Hi BA_Pete
Appreciate your response. I think maybe i need to be more clear on my situation.
I have a blob azure database updating with CSV files each day from our transactional system - each day, the CSV will contain duplicates of the day before's information, but some of the rows of data will have updated info. I have a date modified field so i can use this to tell whether info has updated from the day before, but im not sure how i go about only updating the new rows (and therefore removing the old rows of data) using incremental refresh - all i can see i can do is to load all the information and then filter out the older data. This feels like heavy lifting each day if not much has changed.
Best regards,
Drew
