I have two questions (I already tried to search for the answer but haven't found one):
- Why does sorting operation in Power Query reimports the entire data? This doesn't seem to happen in other operations.
- The refresh (when we click "Close & Apply") is terribly slow after I implement sorting in the Power Query.
Background:
I have imported data from 11 different text files (appending them - turns out about 1.2M rows in all and 23 columns) and after importing, one of the last operations before loading the data into PowerBI is to sort the data by two of the columns. When the operation is executed, I see the entire data being reimported which takes about a minute or two. I do not see any explanation when sorting is explained in tutorials.
Also, since then, the refresh, which used to take just about 4-5 minutes earlier, is taking crazy times (never finished after waiting for about 25-30 minutes).
Initially, I thought it might be a network issue, so I copied the files from network into my C drive and updated the path in the "Source" stage of the data import but no change. I also checked by importing the files from the network into another tool and it worked fine.
After removing the sorting step, the refresh works fine.
