Append and Merge Best Practice

[Anonymous]

Hi all, 

I'm hoping to gain some brevity on the most efficient way to append my data.

I have 57 SharePoint sites, each with 5 lists, but every site uses the same structure for the 5 lists.

Appending and Merging in the Query Editor spews out some incredible load times that I'd like to avoid.

Outside of unchecking enable load on the tables and not allowing background data to load; what method can I use to speed things up when I append my data?

I've seen parts of solutions involving M code but nothing I can grasp - where would be the best source material?

thank you kindly,

J

[Jimmy801]

Hello @Anonymous 

have you been able to solve the problem with the replies given?

If so, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too

All the best

Jimmy

[Jimmy801 · Accepted solution]

Hello @Anonymous 

you may have to analyse your code. How your are accessing the data and merging/appending. Depending to how you are accessing it, is for sure useful to place a Table.Buffer somewhere to avoid pulling data multiple times. 

If you use a custom function to pull your data from the list, then this is for sure a good starting point to put a Table.Buffer.

Feel free to post here some M code and how your queries are structure. Then I can look into it.

If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too

Have fun

Jimmy 

[edhans]

You want to uncheck the "Load" box on all of those queries. "Load" means load into the data model. If your final combined query is loaded into the model, then any queries it depends on will process. They are just "Connection Only" queries. In almost every PBIX file I have, I have more queries that are connection only than I do that actually load.