Big Data Warehouse?

[Anonymous]

Hello Everyone,

I am looking for a tool (platform?) that will allow me to store huge amounts of data. The data will be coming either from ERP systems or from various excel files (same structure of the data, just different values every month).

For instance, something that will be able to handle more than 100k "excel rows" (+40 columns) each month. I was reading about "Hadoop". Is this a good solution? I saw that I can import data to PowerBI from Hadoop, so that would be cool.

If anyone can help to find a good comparison of various "data warehouse" (because I think that this is the correct naming) tools + your personal experience and preference, I would be very grateful.

thanks!

[Jayendran · Accepted solution]

Hi @Anonymous ,

You are in luck , from the recent Microsoft Ignite 2019 , micorosft announced a great tool called Azure Synapse Analytics

Azure Synapse is a limitless analytics service that brings together enterprise data warehousing and Big Data analytics. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources—at scale.

Moreover it has a inbuild capability of PowerBI iteself 😊 

Reference: https://docs.microsoft.com/en-in/azure/sql-data-warehouse/sql-data-warehouse-overview-what-is

[Anonymous]

awesome, I will take a look. thanks a lot! 

[Jayendran]

Cool, you can take a look into this video also

https://www.youtube.com/watch?v=xzxjpQSvDEA

[Anonymous]

@Jayendran  - I would have one more question. As I am still learning Big Data... With this tool, I should be able to load a huge set of data (from excel files, ERP systems, etc.) and then I should be able to access this data with Power Query/Power BI/Tableau/Excel.

Could you confirm if my understanding below is correct? Do you know what are the other similar tools that I could deep dive in?

thank you

[Jayendran]

Hi @Anonymous ,

I'm not exactly sure as I didn't try Azure Synapse Analytics

Here are the some Big Data tools which you can use in Azure.

1. Azure HDInsights
2. Azure Data Lake
3. Azure Data Bricks