Forum Discussion

rlansing's avatar
rlansing
Icon for Resolver I rankResolver I
10 years ago

How to Optimize Memory Usage?

Hello, I am running Win 8.1 Pro with Power BI 64 bit (July 2016 update). I have also installed the On-Premises Enterprise Gateway and am having issues when running my scheduled refreshes. I believe ...
Need Help
Tips and Tricks
rlansing's avatar
rlansing
Icon for Resolver I rankResolver I
10 years ago

Thank you. I have used the documents to attempt to optimize the report. I think the issue may require more RAM, so we will increase it to 32GB and see if that can help as well. I will report back if I find a definitive problem in the report that can be fixed.

 

I do think there may be an issue with the gateway though. When I refresh the report in Power BI Desktop, it only uses about 35% RAM while the CPU and disk jump around, but when refreshing using the gateway, it maxes out RAM as well and at times causes this issue.

rlansing's avatar
rlansing
Icon for Resolver I rankResolver I
9 years ago

This issue has returned. Can anyone explain why importing 2-3GB of CSV data would cause 32GB to fill up in less than two minutes? Once the memory is full, Windows starts using the pagefile, which dramatically affects the speed. I have added the issue to the community forum, but I need to know what factors could cause this. Especially when I am not importing anywhere near the amount of data. Switching from Excels to CSVs has improved loading times when the RAM doesn't get too full, but the issue still exists for large datasets.

  • rlansing's avatar
    rlansing
    Icon for Resolver I rankResolver I
    9 years ago

    The issue has been solved! It was a problem with the query steps where two queries were merged together based on text columns. In large datasets I think it required too much memory to perform the table scan necessary to complete the full outer join. I switched it to match based on a whole number and it still takes a little bit of time, but the RAM is much more reasonable and the computer does not become unresponsive.

    • hugoberry's avatar
      hugoberry
      Icon for Responsive Resident rankResponsive Resident
      9 years ago

      Hi rlansing, Another tip that you might follow is to check your data types in the Data Model. I've noticed that a stray number that appears as Text in Data Model, consumes a lot more memory than it should. Moreover, it is worth considering making your "joins" in the data model. Or rather try to migrate some of this work into there.

      Also on the query side. Some of the functions have the ability to "stream" data. I know at least one binary function that does that :) BinaryFormat.Choice