Forum Discussion
Append tables explodes file size
Found it - it was the List.PositionOf lines. I replaced them with standard table merges and suddenly the table size has gone from 9.2GB to 40MB and the refresh is quick again.
The theory behind it is discussed a bit in this post and this is what led me to those lines being the problem:
Solved: List.PositionOf > performance than Merge Queries? - Microsoft Fabric Community
Thanks again Ritaf1983 for getting me thinking in the right directions, even if I never gave you a chance to identify the specific problem before I found it on my own
Hi crispybits77
It’s quite surprising that such small tables could result in an 8GB size, even if performing a Cartesian multiplication (which would involve a merge rather than an append).
Here are some possible reasons for this unexpected behavior:
- Unexpected Cartesian Product: Could one of the merges (joins) or calculations be creating an excessively large number of rows due to incorrect join conditions?
- Unintended Duplication: Adding columns using formulas might unintentionally duplicate data.
- Memory Expansion: The process might temporarily inflate memory usage, especially if heavy calculations are being applied across multiple fields.
Without seeing the file and understanding the exact steps, it’s hard to provide a definitive answer. I’d recommend reviewing each step in the process to identify where this dramatic size increase is happening.
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.