Forum Discussion
Unexpected Multiple Requests from Power BI Connector
- 1 year ago
Hey v-kpoloju-msft,
priorities shift at work this week, so I didn't have time yet to look into your suggestions. Hope I will find some time next week to look into Diagnostics.Cache.
Api.Folder doesn't look poosible at this point, but maybe later.
Disabling the preview didn't change anything for me regarding the number of requests. But since I already reduced requests at that point (and I don't have controll over everyone who uses the connector to turn that option off) I don't think that's helping.
Sorry again for the delay and thanks again for all the help and input!
Regards,
Scheißy
Hi Scheißy,
Thank you for reaching out to the Microsoft fabric community forum. Also, thanks to Nasif_Azam, for those inputs on this thread and for the detailed description you are absolutely on the right track, and great job using diagnostics for deeper insight.
The issue you are experiencing, where the API endpoints (especially authentication and file download) are hit multiple times, is a known behaviour in custom Power BI connectors. It often happens because Power BI evaluates the connector logic multiple times for example, during schema discovery, preview, and data load phases.
To reduce redundant calls, especially for authorization and file download, you can implement caching using Function.Cached in your Power Query M code. This ensures that your token and file data are computed once and reused across calls.
Cache the authentication token using Function.Cached. Wrap your file download function (loadCsvFile) in another Function.Cached to prevent redundant fetches. Consider using ResourcePath in Web.Contents to improve connector performance.
Kindly refer to the below mentioned links for more information:
Web.Contents - Power Query M | Microsoft Learn
Power Query M function reference - Power Query M | Microsoft Learn
Connector extensibility in Power BI - Power BI | Microsoft Learn
Hope this helps clarify things and let me know what you find after giving these steps a try happy to help you investigate this further.
Thank you for using the Microsoft Fabric Community Forum.
- Scheißy1 year agoRegular Visitor
Hey v-kpoloju-msft,
thanks for your input. I'll try that stuff and check your links, but I already have a question:
You mention the "ResourcePath" option. I tried that, but I always got an error message that the ResourcePath option isn't a valid Web.Contents option. Google said that this error message should only happen in Visual Studio, but I keep hitting that same error message in Power BI. What am I doing wrong?
- Scheißy1 year agoRegular Visitor
Hey v-kpoloju-msft
I tried to look into your suggestions:
First of all I rechecked the "ResourcePath" option. I switched from "Visual Studio 2019" to "Visual Studio Code" and I did not get the error message that "ResourcePath" is an invalid option. I was hopeful, and since I also updated my Power BI to the newest version I was really hoping that I'm making some progress here. But when I tried the new connector in the updated Power BI version I was hitting the same result: "ResourcePath" is an invalid option.Checking the Web.Contents link you posted above I saw that "ResourcePath" isn't mentioned there as a possible option. In the error response I got in Power BI there are listed possible options for "Web.Contents". Of course no "ResourcePath" was mentioned there, but I found a "Resource" parameter (which is also not listed in the documentation). So I thought maybe the option name just changed. I tried "Resource" instead of "ResourcePath" and my connector worked again in Power BI - but unfortunately the same number of requests are shown in the log 😕
Next thing I tried to wrap the download function. I checked the links again, but couldn't find any examples or help how to do so. But with a little external help I came up with something like this:loadCsvFile = (relativePath as text, token as text) => let cachedLoadCsvFile = () => let _token = if debug <> null then Diagnostics.Trace(TraceLevel.Information, "loadCsvFile " & relativePath & " called with token: " & token, token) else token, response = Web.Contents(MyConnector.StorageUrl, [ RelativePath = relativePath, Headers = [ Authorization = "Bearer " & _token ] ]), csv = Csv.Document(response, [ Delimiter = ",", Encoding = 65001, QuoteStyle = QuoteStyle.Csv ]), table = Table.PromoteHeaders(csv) in table in cachedLoadCsvFile;This actually helped a bit - all the download requests which are made during the navigation table is loaded are gone. This reduces the number of file requests from 8 for every single file to 6. The only thing that unsettles me a bit is that in the navigator the files are now displayed as "functions" and not as "tables" anymore (just to be clear: I'm only talking about the icon here). But I couldn't see any difference in the actual data which was imported, so I hope this is fine.
Of course I would love to reduce the number of requests more than that. So feel free to add more suggestions 🙂Regards,
Scheißy
- v-kpoloju-msft1 year ago
Community Support
Hi Scheißy,
Thank you for your detailed follow-up and for sharing your findings really appreciate the structured troubleshooting you have done here.
You are right in noticing that ResourcePath is not a documented parameter for Web.Contents. It was previously used in some internal/custom connector implementations but is not officially supported in the current Web.Contents - Power Query M | Microsoft Learn
Instead, RelativePath and Query are the recommended and documented options, as outlined here: Web.Contents - PowerQuery M | Microsoft Learn
Using loadCsvFile with a wrapped function is a smart move, and you are correct that this can help with reducing redundant requests during query folding and preview generation, especially in custom connectors.The change in icon from "table" to "function" in the navigator is expected when using a function wrapper, it should not affect data load, as you have rightly observed. As long as the result loads data correctly and consistently, you are on the right track.
Although experimental, you might try the Diagnostics.Cache function to further limit backend requests, especially during navigation. Monitor using diagnostic settings - Azure Cache for Redis | Microsoft Learn
Use Api.Folder (if applicable): If you are calling APIs with many similar file endpoints, you can sometimes bundle them more efficiently using folder-like behaviours. Disable Preview in Navigator (for large APIs): While not a direct solution in M code, instructing users to disable “Allow data preview to download in the background” in Power BI options can reduce load-time requests.
As you have already eliminated 2 requests per file, that is real progress. The function icon behaviour is expected and harmless. If you are still facing request overhead, it may be inherent to how Power Query navigators parse folder/endpoint structures for previews.Hope this helps clarify things and let me know what you find after giving these steps a try happy to help you investigate this further.
Thank you for using the Microsoft Fabric Community Forum.- v-kpoloju-msft1 year ago
Community Support
Hi Scheißy.,
Just checking in to see if the issue has been resolved on your end. If the earlier suggestions helped, that’s great to hear! And if you’re still facing challenges, feel free to share more details happy to assist further.Thank you.