Memory leak when using PQ advanced editor

[greggy_b]

I can reliably leak gigabytes of memory per minute when editing M code in the advanced editor, leading to performance degradation of UI interactions and eventually process crashes with out-of-memory errors.

The pbix file I am using is available at https://www.antifound.com/assets/aoc2022.pbix.

The data is all from Advent of Code 2022 and is therefore not sensitive.

Reproduction steps:

1. Open the pbix
2. Open the PQ window
3. Select any existing query
4. Open the advanced editor
5. Begin adding query steps and editing the code (specifically code and not comments), making sure to leave enough time to get completion pop-ups
6. Close the advanced editor
7. Close the Power Query window, but do not close Power BI Desktop (applying the new changes does not make a difference)
8. Repeat from step 2

By following these steps, I can reliable get my PBI Desktop process up to 10GB of RAM (out of 16GB on the VM) consumed, at which point it becomes unstable and nonresponsive, either crashing with a webview2 memory error or hanging until I kill it through task manager.

Note, that you must have intellisense enabled (the default), and you must interact with intellisense completions while editing code in the advanced editor. Memory consumed by this process seems never to be freed.

[greggy_b]

If you need some inspiration for steps to write by hand, you can just start re-typing the last few steps repeatedly with new names. Do not copy/paste, but actually type in the new code by hand, so that you trigger Intellisense completion prompts.

[greggy_b]

With more exploration, the rate of growth of RAM seems to be related to the 'Enter Data' sources. To be very clear, I can reproduce this leak in other PBIXes. This comment is not to imply that the issue only appears when 'Enter Data' is used.

The length of the line *OR* the size of the string literal (or possibly both) in these seems to increase the rate at which memory is leaked. I tried replacing these with small literal `#table` definitions that contain less data than in the 'Enter Data' sources in the linked PBIX. When I did so, the rate at which memory leaks seems to be much lower, but again, I can still reproduce the leak.

[v-yetao1-msft]

Hi @greggy_b 

I'm a little confused as to what kind of functionality you need to achieve or what kind of result you want to achieve after all your calculations.

Best Regards,
Community Support Team _ Ailsa Tao

[greggy_b]

@v-yetao1-msft , this is not about any specific computation or result. This is a report of a repeatable behavior that appears to be a memory leak.

If it is necessary to shoehorn this description into a form, then here goes:

When I edit M code in the advanced editor, I need the UI to remain responsive.

When I edit M code in the advanced editor over the course of time, I need the Power BI Desktop  process not to crash with an out of memory error.

This description feels reductive and silly to frame in that way. The M code is working correctly. The mashup container runtime environment is performing as well as it ever has. The process responsible for rendering the GUI exhibits behavior that aligns with a memory leak when I edit M code with Intellisense enabled.

[TaylorClark]

@greggy_b Thanks for posting! We're discussing the source of the memory leak internally and think we have a solution. I would expect a fix in a patch to the December release of Power BI Desktop, or in the first release of 2023

[greggy_b]

Thank you for the update!

I have continued working on this PBIX and I no longer believe in my hypothesis about long lines or size of string literals. I made a version replacing all the 'Enter Data' sources with `#table` expressions, and the performance and leak seem to persist. I now worry that the size of the leak is proportional in some degree to the total size of M code in the PBIX.

[v-yetao1-msft]

Hi @TaylorClark 

Thank you for your reply, if there is any progress on this issue, I hope you can update your reply here, thanks again!

Best Regards,
Community Support Team _ Ailsa Tao

[mattmasson]

We believe we've identified the problem. A fix will be included in the next release of Power BI Desktop (first release of 2023).