Forum Discussion
undefined
QuestionForGroup Hi All. I have a dataset of 2.9M rows which upon unpivoting and merging with 1 table becomes some 246M rows. After that I have to remove duplicates from the table(based on Col1, Co...
Your dataset blows up (246M rows) and deduplication happens after folding breaks, causing memory errors.
Fix: push unpivot/merge/deduplication to the source (SQL/Dataflow), reduce columns early, and keep query folding alive. If needed, use incremental refresh or DirectQuery.