Forum Discussion
Large SharePoint list data
OData gets the filenames, and SP Online List gets the metadata - if you put each separately in a dataflow table, you could merge them together on the unique file identifier in a new query. I mention dataflow only because with the size of the list it will require you to ... what's the word... break it down a bit, then merge it together with something that has a fresh resource pool to work with, if that makes sense. I would do the final merge on the PBI desktop side with the dataflows as the source - the "pipe" to the dataflows is much faster than the pipe to SharePoint. None of these are proper terms, but I hope you get the gist of what I mean. 🙂
Make sure when you use the list connector that you select the columns you want, then right click "remove other columns" - particularly to get rid of the complex field types like person fields. I am pretty sure that the remove columns step "folds" and causes it to go much faster if you do it first in the applied steps.
Use the 2.0 connector, its much faster than 1.0.
I've done exactly this for large libraries, but they were not THAT large... 400,000 is a lot for a SharePoint source, and it'll probably throttle you on top of it being slow to start with, so I can't promise it'll get that many but conceptually that's how I'd approach it.