Reply
1Dan_Crowther
New Member

Using Power Query to keep Refresh data without overwriting existing data?

Hi

I am looking to import a data value from a web page each day on daily scheduled refresh, but i am looking for each data refresh to store each refresh so the the next daily refresh will create a new row of data with the the updated value and the new refresh date, 

However when i tried this by adding adding a custome column of DateTime.LocalNow() function, the fresh still keeps overwring the old refresh with the new updated data refresh... there must be a way with in Power Query to successfully keep refreshed data with it overwriting?

Cheers

Dan

1 ACCEPTED SOLUTION
lbendlin
Super User
Super User

No, there is not. 
Power BI has no memory* . You need to manage your snapshots in the upstream system. * Yes, there is the self referencing option but that has no safety net.

 

View solution in original post

2 REPLIES 2
BA_Pete
Super User
Super User

Hi @1Dan_Crowther ,

 

@lbendlin is correct, Power Query is not a data warehouse and is not designed to retain data beyond its immediate use as standard. However, you could look into using Incremental Refresh, although this comes with its own significant caveats: https://learn.microsoft.com/en-us/power-bi/connect-data/incremental-refresh-overview 

 

Pete



Now accepting Kudos! If my post helped you, why not give it a thumbs-up?

Proud to be a Datanaut!




lbendlin
Super User
Super User

No, there is not. 
Power BI has no memory* . You need to manage your snapshots in the upstream system. * Yes, there is the self referencing option but that has no safety net.

 

avatar user

Helpful resources

Announcements
March PBI video - carousel

Power BI Monthly Update - March 2025

Check out the March 2025 Power BI update to learn about new features.

March2025 Carousel

Fabric Community Update - March 2025

Find out what's new and trending in the Fabric community.

Top Solution Authors (Last Month)
Top Kudoed Authors (Last Month)