Forum Discussion
Power Query suddenly extremely slow when materialising tables (Group / RowCount / Buffer)
hey DataGirl, interesting findings you have there.
A few observations:
- The fact that filtering, column selection, and expansion steps remain fast while materialization steps become the bottleneck could mean that the slowdown is occurring inside the Power Query evaluation engine rather than at the source.
- Table.RowCount timing out on intermediate tables is particularly telling, since it forces full evaluation but performs minimal transformation logic
Because you're seeing the same behavior in both Desktop and Gen1 Dataflows, and across multiple datasets, it seems less likely to be query-specific and more likely related to a recent engine or infrastructure change.
As a troubleshooting step, I'd recommend capturing Power Query Diagnostics in Desktop and comparing timings before and after the materialization step.
https://learn.microsoft.com/en-us/power-query/query-diagnostics
That may help identify whether the engine is spending time evaluating, buffering, or spilling data to disk.
Appreciate if you can "Kudos" and/or "Accept as Solution" if this answered your query.
- DataGirl1 month agoRegular Visitor
I couldn't see anything specific in the diagnostics in desktop - although I did also keep getting this error: Unexpected error: Container exited unexpectedly with code 0x80131623. PID: 13792.
Used features: (none).
I have done some more testing in the service to try and narrow down the issue - summary below (thanks to Co-pilot):Summary of findings
Initially there was a concern that the issue may have been caused by an accidental Cartesian expansion. I have now ruled this out.
The query structure is:
- Date table filtered to reporting period = 1,547 rows
- IRO allocation table = 1,187 rows
- Expanded result = 1,836,289 rows
- Active-date filter reduces this to 453,142 rows
Performance by step
Step Row Count Duration
ExpandIRODates 1,836,289 2.06 seconds FilterIRODates 453,142 1.76 seconds Distinct Keys Test 453,142 A few seconds Table.Group (RowCount only) 453,142 3m 15s Table.Group (alternative test) 453,142 3m 41s Additional testing
To identify where the slowdown occurs, I replaced the original grouping logic with a number of simpler tests.
Test 1 – Distinct Keys
Using only:
Table.Distinct( Table.SelectColumns( FilterIRODates, {"Date","CLAID","IROStartDate"} ) )This completed in a few seconds.
This demonstrates that:
- The row volume is not the issue.
- Generating the grouping keys is not the issue.
- The date expansion is not the issue.
Test 2 – Simplified Group
Using:
Table.Group( FilterIRODates, {"Date","CLAID","IROStartDate"}, {{"Rows", each Table.RowCount(_), Int64.Type}} )This took over 3 minutes.
This removes any complex aggregation logic and simply counts rows within each group.
Conclusion
The evidence now suggests that the bottleneck is specifically within the Table.Group aggregation/materialisation phase.
The following have effectively been ruled out:
- Dataflow retrieval
- Dataset size
- Cartesian expansion
- Date filtering logic
- Distinct key generation
- The specific List.Max(IROEndDate) aggregation used in the original query
The issue appears to occur as soon as Power Query has to create and process grouped partitions, even when the aggregation itself is extremely simple.
At this stage the behaviour seems disproportionate to the workload involved, given that:
- 1.8 million rows can be expanded in around 2 seconds
- 453k rows can be filtered in under 2 seconds
- Distinct keys can be generated in seconds
- Yet grouping the same 453k rows takes over 3 minutes
This suggests the problem is specifically related to how the engine is handling Table.Group rather than the volume or shape of the data itself.