Forum Discussion

ZikoPowerBI's avatar
ZikoPowerBI
Helper II
2 years ago
Solved

Power BI Challenges with huge data volume

Hi Team , 

 

I have a general query about performance and capacity of Power BI.

 

I have a scenario where I have to fetch 200 Million records per date . The data source that I am using is Memsql which is on premises database . Premium capacity is P2 . Can someone please help what is the appropriate capacity configuration that we should have with respect to data source as well as Power BI Premium capacity.

 

Tried Direct query , but it is too slow. With Import mode it took almost 7 hours to fetch the whole data. Anyway , we will be implementing IR , but the risk here is that the first load atleast should happen successfully on service wih full volume.

 

Followed all the below steps to fetch the data:

  • Remove unused tables or columns, where possible. 
  • Avoid distinct counts on fields with high cardinality – that is, millions of distinct values.  
  • Take steps to avoid fields with unnecessary precision and high cardinality.
  • Use integers instead of strings, where possible.
  • Be wary of DAX functions, which need to test every row in a table – for example, RANKX – in the worst case, these functions can exponentially increase run-time and memory requirements given linear increases in table size.
  • When connecting to data sources via DirectQuery, consider indexing columns that are commonly filtered or sliced again. Indexing greatly improves report responsiveness.  

 

3 Replies