Forum Discussion
Helper IIISlow refresh of SharePoint list data
Yes, pls try that.
Fetch your raw-data in one query and then reference this query. This will create a first step like so:
Source = NameOfTheQueryWithRawData
Change that step into this:
Source = Table.Buffer(NameOfTheQueryWithRawData)
Unfortunately, there's not much informations available around performance tuning at the moment (here are some additional details: https://blog.crossjoin.co.uk/2016/11/20/referenced-queries-and-caching-in-power-bi-and-power-query/)
Helper IIII was thinking about another possibility, but I don't know if it will work. Could a just get my sharepoint list with raw data, then reference that query and then do all of my query steps in that referenced query and not enable this referenced query in the refresh?
Community ChampionYes, pls try that.
Fetch your raw-data in one query and then reference this query. This will create a first step like so:
Source = NameOfTheQueryWithRawData
Change that step into this:
Source = Table.Buffer(NameOfTheQueryWithRawData)
Unfortunately, there's not much informations available around performance tuning at the moment (here are some additional details: https://blog.crossjoin.co.uk/2016/11/20/referenced-queries-and-caching-in-power-bi-and-power-query/)
- hugoberry
Responsive Resident
Follow ImkeF example. If you need more details, post your query and I'll try to point where exactly you can benefit from Table.Buffers.
As for the reason why you need to have a Buffer in place, it is almost like forcing Power Query to firstly download all of the content from SharePoint first (into the Buffer) and then perform all of the other operations.
I assum that in your case, Power Query downloads portions of the list from SharePoint and then performs some pivoting, then fetches some more data from SharePoint again, etc. This way you are wasting a lot of time on waiting, and possibly on duplicate HTTP requests to SharePoint.
Of course this explanation is worth reading if the Table.Buffer works :)
- Matthias93
Thanks for helping guys. I pulled in the raw list data through a query and then created a reference query with the Table.Buffer added where I do the pivots and other operations. The total refresh time went down from 3 minutes to around 18 seconds, which is quite nice.
Regards,
Matt
- Matthias93
Hi,
After some more testing I find that my referenced query is not at all being updated based on the source query. I disabled my referenced query from refreshing too make the refreshing on the report faster. It does not work with or without the Table.Buffer function in front of the referenced query.
Do any of you know what I can do to make this work, I'm getting quite frustrated with it.
Regards,
Matt
I am very new to Power BI and am facing a similar issue and don't know where to try and use the buffer.table step.
I have a sharepoint site that currently contains 562 files with multiple rows of data, each week the number of files grows each week (up to 300 per week).
It can take up to 20min refresh data now, so I can imagine it will only get slower as the number of files increase week over week. This behavior only appears to be an issue with connecting to Sharepoint. I tested with the same dataset on One Drive and there were nowhere near the latency issues - unfortunately I have to use Sharepoint and would love to be able to speed up the refresh time.
I saw other blogs suggesting to uncheck "Auto Date/Time" but that didn't seem to make any impact, so I'm hoping your suggestion is what i need, I just don't know where it goes or how to add it.
Thank you!
My data is taking forever and a day to refresh. How can I make it faster. Data is below.
let
Source = SharePoint.Tables("https://activestirling793.sharepoint.com/AShub/Kit", [ApiVersion = 15]),
#"ada6df12-8ddd-48d5-ada7-56a1d0236a7c" = Source{[Id="ada6df12-8ddd-48d5-ada7-56a1d0236a7c"]}[Items],
#"Expanded Category" = Table.ExpandRecordColumn(#"ada6df12-8ddd-48d5-ada7-56a1d0236a7c", "Category", {"Title"}, {"Category.Title"}),
#"Expanded Product" = Table.ExpandRecordColumn(#"Expanded Category", "Product", {"Title"}, {"Product.Title"}),
#"Renamed Columns" = Table.RenameColumns(#"Expanded Product",{{"Category.Title", "Product"}, {"Product.Title", "Size"}}),
#"Expanded lookup" = Table.ExpandRecordColumn(#"Renamed Columns", "lookup", {"Full Name", "Job Title", "Line Manager", "Status", "Team", "Location", "Request Date", "Approved", "ID"}, {"lookup.Full Name", "lookup.Job Title", "lookup.Line Manager", "lookup.Status", "lookup.Team", "lookup.Location", "lookup.Request Date", "lookup.Approved", "lookup.ID"}),
#"Removed Columns" = Table.RemoveColumns(#"Expanded lookup",{"FileSystemObjectType", "Id", "ServerRedirectedEmbedUri", "ServerRedirectedEmbedUrl", "ContentTypeId", "ComplianceAssetId"}),
#"Changed Type" = Table.TransformColumnTypes(#"Removed Columns",{{"Quantity", Int64.Type}}),
#"Renamed Columns1" = Table.RenameColumns(#"Changed Type",{{"lookup.Full Name", "Full Name"}, {"lookup.Job Title", "Job Title"}, {"lookup.Line Manager", "Line Manager"}, {"lookup.Status", "Status"}, {"lookup.Team", "Team"}, {"lookup.Location", "Location"}, {"lookup.Request Date", "Request Date"}, {"lookup.Approved", "Approved"}}),
#"Changed Type1" = Table.TransformColumnTypes(#"Renamed Columns1",{{"Request Date", type date}})
in
#"Changed Type1"
