Online refresh not supported - Web.Contents, RelativePath and Query

[sceccolini]

Hello Community,

I need your support to troubleshoot my issue with the online refresh not enabled. I defined the following function 'ExecuteREST' to invoke a REST with pagination by using Web.Contents RelativePath and Query:

let
    GetAllPages = (BaseUrl as text, RelativePathParam as text) =>
        let
            GetPage = (PageNumber as number) =>
                let
                    Response = Json.Document(Web.Contents(
                        BaseUrl,
                        [
                            RelativePath = RelativePathParam,
                            Query = [
                                page = Text.From(PageNumber)
                            ]
                        ]
                    )),
                    Entries = try Response[entries] otherwise null,
                    Updated = try Response[updated] otherwise null
                in
                    [Entries = Entries, Updated = Updated],
            Pages = List.Generate(
                () => [PageNumber = 1, PageData = GetPage(1)],
                each [PageData][Entries] <> null and List.Count([PageData][Entries]) > 0,
                each [PageNumber = [PageNumber] + 1, PageData = GetPage([PageNumber] + 1)],
                each [PageData]
            ),
            CombinedResults = List.Combine(List.Transform(Pages, each [Entries])),
            UpdatedValue = if List.Count(Pages) > 0 then Pages{0}[Updated] else null,
            ResultTable = Table.FromList(CombinedResults, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
            ResultTableWithUpdated = Table.AddColumn(ResultTable, "updated", each UpdatedValue),
            #"Expanded Column1" = Table.ExpandRecordColumn(ResultTableWithUpdated, "Column1", {"content"}, {"content"})
        in
            #"Expanded Column1"
in
    GetAllPages

I'm using this function to consume my REST, as follows:

let
    BaseUrl = "<MY BASE REST URL>",
    RelativePath = "<MY RELATIVE PATH>",
    Source = ExecuteREST(BaseUrl, RelativePath),
    #"Expanded content" = Table.ExpandRecordColumn(Source, "content", {"src"}, {"src"}),
    #"Added Custom" = Table.AddColumn(#"Expanded content", "jsonData", each let
    RelativePathColumn = Text.AfterDelimiter([src], BaseUrl),
    JsonData = Json.Document(Web.Contents(BaseUrl, [RelativePath = RelativePathColumn]))
in
    JsonData),
    #"Expanded jsonData" = Table.ExpandRecordColumn(#"Added Custom", "jsonData", {"properties"}, {"properties"}),
    ...doing some other irrelevant formatting actions on the exported properties.....

Basically, my first REST execution returns several RESTs to be invoked; each returned sub-REST has the same URL as the first one, except for a specific reference for each one. In this specific case, my first REST is <MY BASE REST>/users, and this REST returns one REST endpoint for each user, like <MY BASE REST/users/user1, etc..

The data is returned, but when i publish the report, the online refresh is not supported.

Can you help me on this?

Thank you

Simone

[sceccolini · Accepted solution]

Hi all,

I resolved this by having a unique function rather than a dedicated REST client:

let
    BaseUrl = "MY BASE URL",
    GetPage = (PageNumber as number) =>
        let
            Response = Json.Document(Web.Contents(
                BaseUrl,
                [
                    RelativePath = "MY RELATIVE PATH",
                    Query = [
                        param = "MY PARAM",
                        page = Text.From(PageNumber)
                    ],
                    Timeout = #duration(0, 0, 10, 0) // Imposta il timeout a 10 minuti
                ]
            )),
            Results = try Response[entries] otherwise null
        in
            Results,
    GetAllPages = List.Generate(
        () => [PageNumber = 1, Results = GetPage(1)],
        each [Results] <> null and List.Count([Results]) > 0,
        each [PageNumber = [PageNumber] + 1, Results = GetPage([PageNumber] + 1)],
        each [Results]
    ),
    CombinedResults = List.Combine(List.Transform(GetAllPages, each _)),
    ResultTable = Table.FromList(CombinedResults, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Expanded Column1" = Table.ExpandRecordColumn(ResultTable, "Column1", {"content"}, {"content"}),
    #"Expanded content" = Table.ExpandRecordColumn(#"Expanded Column1", "content", {"properties"}, {"properties"}),
....

[Anonymous]

Hi @sceccolini 

My suggestions are as follows for your reference:

• please try not to use dynamically constructed URLs in Web.Contents, and consider using relative paths and query parameters.
• Set the data source privacy level.

The following document is for your reference:

Solved: Web.Contents with dynamic file URL - Microsoft Fabric Community

Hope this helps.

Best Regards,
Yulia Xu

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.