Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
6 years ago
Solved

Avoid throttling when loading from SharePoint folder

I have a report loaded entirely from a SharePoint folder. The report uses around 400 files of small size (2MB). When loading the data in the desktop version, sometimes ( 1 out of 2 times) Sharepoint ...
Need Help
  • Anonymous's avatar
    Anonymous
    6 years ago

    I have an environment with in excess of 18,000 individual files and multiple Dataflows running at the same time and yes it suffers from throttling but Microsoft support pointed me in the direction of buffering each Binary before you expand it and this significantly improved the performance.   

     

    So I land these CSV files into Dataflows and then pull that into the model.

     

    Source = Csv.Document(Binary.Buffer([Content]),[Delimiter=",", ...............

     

    Does not completely get rid of the issue so I also schedule a number of refreshes.

     

    Hope that helps. 

     

    Regards

     

    Andrew

Anonymous's avatar
Anonymous
Not applicable
6 years ago

Hello Anonymous ,

I have looked at the links, and they deal mostly with the model and the report (no mention to SharePoint).
I don´t have performance issues on the model side (it is well streamlined) nor in the load when I do it locally.
If I repoint my load to Sharepoint, I sometimes come up with a throttle issue (both locally and in the premium capacity).
My question is specifically related to SharePoint and what is the best way to design a load from it to avoid throttle.

Thanks a lot,

Jon

    • Anonymous's avatar
      Anonymous
      Not applicable
      6 years ago

      Thanks a lot Anonymous 

      I believe my issue is related to the first post.

      There is no solution to it as of today so I'll follow up in that thread in case somebody was able to solve it.

      Kind regards,

      Jon

      • Anonymous's avatar
        Anonymous
        Not applicable
        6 years ago

        I have an environment with in excess of 18,000 individual files and multiple Dataflows running at the same time and yes it suffers from throttling but Microsoft support pointed me in the direction of buffering each Binary before you expand it and this significantly improved the performance.   

         

        So I land these CSV files into Dataflows and then pull that into the model.

         

        Source = Csv.Document(Binary.Buffer([Content]),[Delimiter=",", ...............

         

        Does not completely get rid of the issue so I also schedule a number of refreshes.

         

        Hope that helps. 

         

        Regards

         

        Andrew