topic Re: Big Data Warehouse? in Developer

Big Data Warehouse?

Anonymous — Wed, 06 Nov 2019 08:46:49 GMT

Hello Everyone,

I am looking for a tool (platform?) that will allow me to store huge amounts of data. The data will be coming either from ERP systems or from various excel files (same structure of the data, just different values every month).

For instance, something that will be able to handle more than 100k "excel rows" (+40 columns) each month. I was reading about "Hadoop". Is this a good solution? I saw that I can import data to PowerBI from Hadoop, so that would be cool.

If anyone can help to find a good comparison of various "data warehouse" (because I think that this is the correct naming) tools + your personal experience and preference, I would be very grateful.

thanks!

Re: Big Data Warehouse?

Jayendran — Thu, 07 Nov 2019 13:52:08 GMT

Hi @Anonymous ,

You are in luck , from the recent Microsoft Ignite 2019 , micorosft announced a great tool called Azure Synapse Analytics

Azure Synapse is a limitless analytics service that brings together enterprise data warehousing and Big Data analytics. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources—at scale.

Moreover it has a inbuild capability of PowerBI iteself 😊

Reference: https://docs.microsoft.com/en-in/azure/sql-data-warehouse/sql-data-warehouse-overview-what-is

Re: Big Data Warehouse?

Anonymous — Thu, 07 Nov 2019 13:58:01 GMT

awesome, I will take a look. thanks a lot!

Re: Big Data Warehouse?

Jayendran — Thu, 07 Nov 2019 14:04:39 GMT

Cool, you can take a look into this video also

https://www.youtube.com/watch?v=xzxjpQSvDEA

Re: Big Data Warehouse?

Anonymous — Thu, 07 Nov 2019 16:16:28 GMT

@Jayendran - I would have one more question. As I am still learning Big Data... With this tool, I should be able to load a huge set of data (from excel files, ERP systems, etc.) and then I should be able to access this data with Power Query/Power BI/Tableau/Excel.

Could you confirm if my understanding below is correct? Do you know what are the other similar tools that I could deep dive in?

thank you

Re: Big Data Warehouse?

Jayendran — Fri, 08 Nov 2019 09:38:26 GMT

Hi @Anonymous ,

I'm not exactly sure as I didn't try Azure Synapse Analytics

Here are the some Big Data tools which you can use in Azure.

Azure HDInsights
Azure Data Lake
Azure Data Bricks