Forum Discussion
Handling Large Data Sets
supton How I wish I was just starting my CRM project. You'll find pretty quickly that to build out a solution you will either spend a ton of time pulling in only the data you need, and the processing will still take awhile, or you look for other solutions.
I manually built out a solution using Kingswaysoft to import all the tables from our online instance to a local DB. Built a tabular model on that data set and created my Power BI Reports.
MSFT released Solution Templates 2 days before I launched the reports... I cried :)
Check out the Solution Template for Dynamics 365 here - this basically guides you through building a full scale solution in Azure.
I'm assuming you want to do more than just query a few things, the above is my experiance and recommendations, but other approaches on a smaller scale could work, I just got frustrated with all the processing time, searching for things etc. And we needed a full support solution to report on all aspects of our CRM.
supton How I wish I was just starting my CRM project. You'll find pretty quickly that to build out a solution you will either spend a ton of time pulling in only the data you need, and the processing will still take awhile, or you look for other solutions.
I manually built out a solution using Kingswaysoft to import all the tables from our online instance to a local DB. Built a tabular model on that data set and created my Power BI Reports.
MSFT released Solution Templates 2 days before I launched the reports... I cried :)
Check out the Solution Template for Dynamics 365 here - this basically guides you through building a full scale solution in Azure.
I'm assuming you want to do more than just query a few things, the above is my experiance and recommendations, but other approaches on a smaller scale could work, I just got frustrated with all the processing time, searching for things etc. And we needed a full support solution to report on all aspects of our CRM.
Our system is so insanely customized that there is no way any template would ever be useful for us. :smileymad:
Anonymous Ours too, maybe it was a good thing I didn't invest time into the template to hit a dead end due to the "over" customization in ours. Don't know, but I wanted to dream that there was an easy button I just missed out on :)
We're just coming to the end of launching a whole new Dynamics 365 built from scratch. Most of the new design work is based on me explaining why the old design was bad for reporting. :D
Thank you so much for the reply! Using the Solution Template this way, we would still need an azure subscription though, correct? My initial thought was to setup the Dynamics 365 Data Export Service myself to bring the data into Azure, and have Power BI report live off of that, but considering this solution template uses the data export service and walks you through everything it seems like this is a better option! Is Power BI mostly meant to report off of smaller data sets normally? It seems as though everyone should have run into this issue at some point!
supton Azure sub - yes
No, Power BI can scale, the issue is accessing the CRM data in the methods allowed. It's slow retrieving the information.
So I would say the limitation is in the connection type, not necessarily in Power BI.There are different approaches to solve different problems.
