Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
6 years ago
Solved

Avoid throttling when loading from SharePoint folder

I have a report loaded entirely from a SharePoint folder. The report uses around 400 files of small size (2MB). When loading the data in the desktop version, sometimes ( 1 out of 2 times) Sharepoint ...
Need Help
  • Anonymous's avatar
    Anonymous
    6 years ago

    I have an environment with in excess of 18,000 individual files and multiple Dataflows running at the same time and yes it suffers from throttling but Microsoft support pointed me in the direction of buffering each Binary before you expand it and this significantly improved the performance.   

     

    So I land these CSV files into Dataflows and then pull that into the model.

     

    Source = Csv.Document(Binary.Buffer([Content]),[Delimiter=",", ...............

     

    Does not completely get rid of the issue so I also schedule a number of refreshes.

     

    Hope that helps. 

     

    Regards

     

    Andrew

MacJulian's avatar
MacJulian
Advocate I
5 years ago

How do you load all these CSV files?

I used the Power BI SharePoint Folder connector but there I don't see the line:

Source = Csv.Document(Binary.Buffer([Content]),[Delimiter=",", ...............

 

Do you load every single file direct in Power BI?

Anonymous's avatar
Anonymous
Not applicable
5 years ago

 

If you are using the built in function then you will find it the fx part as shown below.

 

Source = Csv.Document(Binary.Buffer(Parameter1),[Delimiter=",", ...............

 

I have not tested this fully but hopefully this works for you

 

 

Andrew