SharePoint Folder Connector takes too long to refresh

[daniel_siman]

I need to read several files inside a folder in SharePoint. They have the same structure, so I'm usuing the SharePoint Folder connector on PowerBI to do this. The amount of data is not so big, for now I have about 30 files, and each of them with an average of 2000 rows. For the first time, the SharePoint Folder connector runs OK, appending all the files in one table. But every time I need to refresh those queries, it takes forever. Sometimes it takes so long that I experienced a timeout.

I've already searched about this problem and found several posts about the same issue, but none of them with solutions.

Does anyone knows the causes of it and possible solutions?

[christinepayton · Accepted solution]

It's probably SharePoint throttling - it doesn't like the number of calls it gets with a large number of files. You could split up the queries into multiple dataflows and stagger the refresh times a bit as a workaround, or use a script to combine the files into a smaller number of files. 

[DataInsights]

@daniel_siman,

Here's a technique that might help:

https://blog.crossjoin.co.uk/2020/05/14/speed-up-data-refresh-performance-in-power-bi-desktop-using-...