Experiencing Performance Issues in Power BI due to Memory Consumption on "Apply & Close"

[S_Lokesh]

While working on my Power BI report, I am encountering significant performance degradation, particularly during the data transformation phase. I am applying multiple data transformation, merge queries and expanding the row. Expanding queries takes longer time. Upon applying and closing these transformations, the report takes an unexpectedly long time to load, and eventually, it crashes. 

When checking the Task Manager, I noticed that the memory is fully utilized by Power BI. Further investigation revealed multiple Mashup container tasks occupying the memory.

I am seeking guidance on how to address and resolve this memory consumption issue in Power BI.Any insights, best practices, or specific steps to optimize memory usage during data transformation and report loading would be highly appreciated.

[v-shex-msft]

HI @S_Lokesh,

It seems like you are used complex operation(e.g. merge, loop, invoke custom function)in power query. I'd like to suggest you try to use buffer functions to optimize your query performance.

Solved: When to use Table.Buffer - Microsoft Fabric Community

New Options For The Table.Buffer Function In Power Query 

Regards,

Xiaoxin Sheng

[Sahir_Maharaj]

Hello @S_Lokesh,

Here is some best practices to optimize memory usage and improve performance;

1. Optimize your Data Model: Only work with only the necessary amount of data. Remove unused columns and filter out irrelevant rows as early as possible in your queries. Also ensure that columns are using the most efficient data type. (For example, use integers instead of strings for numerical identifiers)
2. Streamline Queries: Be cautious with merging and appending queries as these operations can increase the data volume and complexity. Expanding columns can create a lot of additional data so be selective and only expand the necessary columns.
3. Performance Tuning in Query Editor: Avoid Unnecessary Steps as each step in the query editor adds to the complexity. I've found that applying filters early in the query to reduce the amount of data that needs to be processed. If you're working with large datasets, consider using incremental loading which allows you to load only new or changed data instead of the entire dataset.

Some additional resources that might assist you:

• https://community.fabric.microsoft.com/t5/Desktop/Very-high-memory-consumption-on-refresh/td-p/10134...
• https://community.fabric.microsoft.com/t5/Desktop/Power-BI-high-memory-consumption-when-creating-mea...
• https://zebrabi.com/guide/how-to-solve-the-memory-warning-in-running-a-query-in-power-bi-desktop/

Should you have any questions or further assistance, please do not hesitate to reach out to me.