Forum Discussion
Slow data refresh due to merges? (Power Query optimisations)
Merges can get slow, but that sounds ways too long. Can you share your M code for your queries from the Advanced Editor?
You may be interested in this 3-part series by Chris Webb.
Pat
Thanks Pat,
Hard to share all the M code as there are a total of 30 queries in the file, which pull data from 17 tables from 2 sources and then perform different transformations to get to the final tables for reporting. (Unless you know of a way of grabbing all the M-code in one go?).
I appreciate I'm probably using Power Query as more of an ETL tool than it is possibly intended for.
Thanks for the Chris article, very interesting, but not sure I can reduce the number of columns in the underlying tables (and they're not in databases where I could create a view with reduced columns). But useful insight nevertheless!
You can do some serious ETL with M. I didn't realize you were doing many merges. If you suspect it is repeating steps with for each row (seeing the M would have highlighted that potentially), you can incorporate Table.Buffer() with your query to greatly speed things up.
Pat
