Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The ultimate Microsoft Fabric, Power BI, Azure AI & SQL learning event! Join us in Las Vegas from March 26-28, 2024. Use code MSCUST for a $100 discount. Register Now

Reply
clem312
Resolver I
Resolver I

SQL Database needs to be archived

Hi,

 

My SQL database used in Power BI has 3 years of historical data.

It is too much for the application that uses it on a daily bases.

I need to clean at least one year. But I need to keep this year in my Power BI ?

 

How can I do that ?

 

Clement

1 ACCEPTED SOLUTION
edhans
Super User
Super User

Won't work. Power BI will remove data when loading and load fresh. You could use incremental refresh and have it never clear that partition, but it will likley see those changes anyway and clean it up. 

I would consider moving that 1 year of data to a Dataflow, then in Power Query append the dataflow to your SQL data that now only has 2 years, and that will be 3 years.

But, that will still risk losing data if you ever refresh that dataflow, as it will again, purge and reload.

Bottom line is, best practice, DO NOT use Power BI and related technologies for archival. They are not designed for that. Use your database, put it on another system and append in Power Query, etc. Even if you dump that 1 year of data to a CSV file and store that in SharePoint, that is much more resilient than trying to get Power BI to keep it.



Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

DAX is for Analysis. Power Query is for Data Modeling


Proud to be a Super User!

MCSA: BI Reporting

View solution in original post

1 REPLY 1
edhans
Super User
Super User

Won't work. Power BI will remove data when loading and load fresh. You could use incremental refresh and have it never clear that partition, but it will likley see those changes anyway and clean it up. 

I would consider moving that 1 year of data to a Dataflow, then in Power Query append the dataflow to your SQL data that now only has 2 years, and that will be 3 years.

But, that will still risk losing data if you ever refresh that dataflow, as it will again, purge and reload.

Bottom line is, best practice, DO NOT use Power BI and related technologies for archival. They are not designed for that. Use your database, put it on another system and append in Power Query, etc. Even if you dump that 1 year of data to a CSV file and store that in SharePoint, that is much more resilient than trying to get Power BI to keep it.



Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

DAX is for Analysis. Power Query is for Data Modeling


Proud to be a Super User!

MCSA: BI Reporting

Helpful resources

Announcements
Fabric Community Conference

Microsoft Fabric Community Conference

Join us at our first-ever Microsoft Fabric Community Conference, March 26-28, 2024 in Las Vegas with 100+ sessions by community experts and Microsoft engineering.

February 2024 Update Carousel

Power BI Monthly Update - February 2024

Check out the February 2024 Power BI update to learn about new features.

Fabric Career Hub

Microsoft Fabric Career Hub

Explore career paths and learn resources in Fabric.

Fabric Partner Community

Microsoft Fabric Partner Community

Engage with the Fabric engineering team, hear of product updates, business opportunities, and resources in the Fabric Partner Community.

Top Solution Authors
Top Kudoed Authors