Advice on where to store data that updates monthly

[DarinSpence]

This question is to those with experience in database managment.  I received about 40GB of XML data.  I painstakingly converted all of the XML files to CSV, which brought the file size down to about 6GB.  There's going to be a new XML file every month, which I will probably convert to a CSV; but I think all of these CSV files should be imported into a database and the new data appended to the database table each month.

Do you have any advice on how I should store this data?  I'd like to keep it simple.  I don't mind running a database on a local desktop, or maybe I should be looking at one of the Azure options?  This data only update once per month.  I'm just hoping to get pointed in the right direction.

Thanks,

-Darin

p.s.  I've setup MSSQL Developer Edition which may be the simplest option; but I wanted to hear how others have solved for a big fact table (well, big-ish).

[v-jingzhang · Accepted solution]

Hi @DarinSpence 

You can post this question at SQL Server forum (sql-server-general - Microsoft Q&A). There are more DB experts there that may help.

Best Regards,
Community Support Team _ Jing

[mahoneypat]

You have several good options.  Azure SQL or Dataverse would likely give you the most flexibility (cloud), but local SQL Server should be good too.  FYI that you likely could automate the conversion of your XML files to CSV with Power Automate.  You could also consider leaving the data as CSVs and refreshing your Power BI model against those, or make a Dataflow with the data.

Pat

[DarinSpence]

Turns out, the answer was Onelake all along! 😉