Forum Discussion
kusto error- partial query failure
- 9 months ago
Hi Sania-F,
The error you’re hitting is happening because the query does too much heavy lifting in the join stage, especially across two clusters. When multiple large tables are joined without reducing the data first, Kusto tries to expand everything in memory, which is what leads to the runaway query limit. The safest approach is to shrink each dataset to just the unique keys you actually need before joining, and build your final lookup in a single, optimized step. It also helps to materialize the cross-cluster table once so Kusto doesn’t reload it repeatedly, and to use join strategies like innerunique and broadcast so the engine doesn’t expand row combinations unnecessarily.
Another important point is to delay projecting many columns until the very end projecting too many fields during joins increases memory pressure and can trigger this same failure. If you restructure the query to first summarize on STId or STId Region where needed, then intersect those key sets, and only then join once to your pslll dimension table, you’ll avoid memory spikes and get reliable results even across clusters. This pattern is well-aligned with Kusto best practices for large joins and should resolve the issue without needing further tuning. Let me know if you want help reviewing the final query shape, but this direction will get you past the limit safely.
Best Regards,
Tejaswi.
Community Support
Hi Sania-F,
The error you’re hitting is happening because the query does too much heavy lifting in the join stage, especially across two clusters. When multiple large tables are joined without reducing the data first, Kusto tries to expand everything in memory, which is what leads to the runaway query limit. The safest approach is to shrink each dataset to just the unique keys you actually need before joining, and build your final lookup in a single, optimized step. It also helps to materialize the cross-cluster table once so Kusto doesn’t reload it repeatedly, and to use join strategies like innerunique and broadcast so the engine doesn’t expand row combinations unnecessarily.
Another important point is to delay projecting many columns until the very end projecting too many fields during joins increases memory pressure and can trigger this same failure. If you restructure the query to first summarize on STId or STId Region where needed, then intersect those key sets, and only then join once to your pslll dimension table, you’ll avoid memory spikes and get reliable results even across clusters. This pattern is well-aligned with Kusto best practices for large joins and should resolve the issue without needing further tuning. Let me know if you want help reviewing the final query shape, but this direction will get you past the limit safely.
Best Regards,
Tejaswi.
Community Support