Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
clem312
Resolver I
Resolver I

SQL Database needs to be archived

Hi,

 

My SQL database used in Power BI has 3 years of historical data.

It is too much for the application that uses it on a daily bases.

I need to clean at least one year. But I need to keep this year in my Power BI ?

 

How can I do that ?

 

Clement

1 ACCEPTED SOLUTION
edhans
Super User
Super User

Won't work. Power BI will remove data when loading and load fresh. You could use incremental refresh and have it never clear that partition, but it will likley see those changes anyway and clean it up. 

I would consider moving that 1 year of data to a Dataflow, then in Power Query append the dataflow to your SQL data that now only has 2 years, and that will be 3 years.

But, that will still risk losing data if you ever refresh that dataflow, as it will again, purge and reload.

Bottom line is, best practice, DO NOT use Power BI and related technologies for archival. They are not designed for that. Use your database, put it on another system and append in Power Query, etc. Even if you dump that 1 year of data to a CSV file and store that in SharePoint, that is much more resilient than trying to get Power BI to keep it.



Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

DAX is for Analysis. Power Query is for Data Modeling


Proud to be a Super User!

MCSA: BI Reporting

View solution in original post

1 REPLY 1
edhans
Super User
Super User

Won't work. Power BI will remove data when loading and load fresh. You could use incremental refresh and have it never clear that partition, but it will likley see those changes anyway and clean it up. 

I would consider moving that 1 year of data to a Dataflow, then in Power Query append the dataflow to your SQL data that now only has 2 years, and that will be 3 years.

But, that will still risk losing data if you ever refresh that dataflow, as it will again, purge and reload.

Bottom line is, best practice, DO NOT use Power BI and related technologies for archival. They are not designed for that. Use your database, put it on another system and append in Power Query, etc. Even if you dump that 1 year of data to a CSV file and store that in SharePoint, that is much more resilient than trying to get Power BI to keep it.



Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

DAX is for Analysis. Power Query is for Data Modeling


Proud to be a Super User!

MCSA: BI Reporting

Helpful resources

Announcements
Europe Fabric Conference

Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

Power BI Carousel June 2024

Power BI Monthly Update - June 2024

Check out the June 2024 Power BI update to learn about new features.

RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

Top Solution Authors
Top Kudoed Authors