Forum Discussion
How to deal with huge datasets in DirectQuery mode ?
Hi everyone,
I'm new to Power BI and I try to build some reports using tables containing an average of 1M records for each, so I work in DirectQuery mode.
Well, first, I have deleted all the unnecessary columns from all my tables and then build my model. My model is "good", I can filter data through different tables but I'm facing a problem. Some filters take a very very long time to load and make my SQL Server CPU use 100%.
After thinking about the question, I am wondering if create some views directly on SQL Server and then connect to these views in DirectQuery mode to create my model is a good solution. Even better, use these views to create a "big view" in SQL Server (where I join all my views and select only the data I want to build my reports) and then create a connection to this big view in DirectQuery mode. With this solution, I only have one table to manage in Power BI and build my reports will be easier.
I think these two solutions could work, but as I'm new to Power BI, I don't know if delegate all the work to SQL Server is a good practice... Can anyone shed some light on this?
Best regards,
Clément
Hello clementalba ,
yes, when you create a view and do the transformation in SQL its less pressure on Power BI in processing thus better performance.
in this case, you will be using Power BI as a visualisation tool which is better for performance.
If I answered your question, please mark my post as solution, Appreciate your Kudos 👍
3 Replies
- Idrissshatila
Super User
Hello clementalba ,
yes, when you create a view and do the transformation in SQL its less pressure on Power BI in processing thus better performance.
in this case, you will be using Power BI as a visualisation tool which is better for performance.
If I answered your question, please mark my post as solution, Appreciate your Kudos 👍
Thanks a lot Idrissshatila for your very quick and accurate answer !
Syndicate_AdminAdministrator
I have a view that filters 760 million records connected in Direct Query mode. It won't let me set a timeout in this mode. Is that correct? I've already filtered the period and can see the last 15 days of data, but I can't see a predefined historical period; the connection keeps dropping. What do you recommend?
