Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
Petanek333
Helper III
Helper III

Data continuity

Hi,

I would like to advise a best practice procedure for maintaining data continuity when working with databases.
I'm getting data from SQL server and the databases there are divided into years.
Now as the year 2022 has ended, it is about to go to a new database for 2023 accounting wise.
I would need the existing 2022 database to be loaded in the model, when refreshing it would no longer update and the 2023 database would be added to it and updated.
So I would like to ask if it is best practice to have the 2022 database without enable load option and append the 2023 database to it, or if this is handled some other way.
Thank you

Translated with www.DeepL.com/Translator (free version)

1 ACCEPTED SOLUTION
BA_Pete
Super User
Super User

Hi @Petanek333 ,

 

Firstly, if you have any power over the database structure, you should be getting rid of the year partitions. It's generally poor database structure and causes exactly these types of problems.

Due to the fact that you have partitioned databases your query isn't going to fold however you cut it, so I think your 2022-Disable load/append to 2023 is as good a solution as any. However, I would probably suggest pushing this build process to a Dataflow so PQ isn't trying do it all each time your report refreshes. This also gives you a pre-made source for any future reports.

 

Pete

 



Now accepting Kudos! If my post helped you, why not give it a thumbs-up?

Proud to be a Datanaut!




View solution in original post

2 REPLIES 2
BA_Pete
Super User
Super User

Hi @Petanek333 ,

 

Firstly, if you have any power over the database structure, you should be getting rid of the year partitions. It's generally poor database structure and causes exactly these types of problems.

Due to the fact that you have partitioned databases your query isn't going to fold however you cut it, so I think your 2022-Disable load/append to 2023 is as good a solution as any. However, I would probably suggest pushing this build process to a Dataflow so PQ isn't trying do it all each time your report refreshes. This also gives you a pre-made source for any future reports.

 

Pete

 



Now accepting Kudos! If my post helped you, why not give it a thumbs-up?

Proud to be a Datanaut!




Thank you @BA_Pete , unfortunately I don't have any power over the database. I am not familiar with dataflow yet, so a great tip for what to learn. 

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors