Forum Discussion
Slow CSV import
Hi mwegener and v-easonf-msft,
It turns out that it was a rookie mistake. I was processing my data during the import process which was causing the import to be ridiculously slow. I've since imported the CSVs as is and am applying the calculations to the imported data, which has reduced the import process from 90 minutes to < 10 seconds!
Lesson learned, now just to figure out how to get my formulas to work again (since I now have combined data in one table instead of individual tables).
Thanks for the feedback, though!
Hi mwegener and v-easonf-msft,
It turns out that it was a rookie mistake. I was processing my data during the import process which was causing the import to be ridiculously slow. I've since imported the CSVs as is and am applying the calculations to the imported data, which has reduced the import process from 90 minutes to < 10 seconds!
Lesson learned, now just to figure out how to get my formulas to work again (since I now have combined data in one table instead of individual tables).
Thanks for the feedback, though!
Hi... rookie here too. How can I know if I'm processing the data before or after the import? I'm having blocking issues on refreshing CSV files from a folder (around 20 files, 13-50mb each) and maybe I'm doing the same mistake but I can't see it.