can't save dataflow one or more entities references a dynamic data source site

[Anonymous]

Hi, 

The code I use is the following:

let

// Source = Json.Document (Web.Contents ("https://test.test.com/api/v2/tickets?order_by=created_at&per_page=100&include=stats")),

BaseURL = "https://test.test.com/api/v2/tickets?order_by=created_at&include=stats&per_page=100",

EntitiesPerPage = 100,

GetPage = (Index) =>

la Url = BaseURL & "& page =" & Text.From (Index),

Json = Json.Document (Web.Contents (URL)),

Data = Record.Field (Json, "data")

in Json,

// EntityCount = GetEntityCount (),

// PageCount = Number.RoundUp (EntityCount / EntitiesPerPage),

PageCount = 201,

PageIndices = {1 .. PageCount-1},

Pages = List.Transform (PageIndices, each GetPage (_))

in

sides

Based on troubleshooting, it looks like this line is failing:

Pages = List.Transform (PageIndices, each GetPage (_))

Everything works fine until I try "Save and Close" and then I get the message that the data “can't save dataflow one or more entities references a dynamic data source site"

Please let me know how to solve this issue/error?


thanx in advance.

[tonmcg]

Dataflows requires the first parameter of `Web.Contents` to be a base url that contains only the domain name (host) or scheme and domain name. In your case, the host is `test.test.com` and the scheme is `https`. It would look like so:

let
    Source = Web.Contents("https://test.test.com")
in
    Source

Because you're requesting data from an API endpoint that requires pagination, you've set up your requests in a way that dynamically changes the url of each each request. Again, Dataflows requires the base url of the `Web.Contents` function to be static.

In other words, you'll need to set explicitly both the base and relative path of the url and programatically update its query parameters for each request.

Gvien that `test.test` is not a real API, there's no way for me provide testable code. But this should get you on your way:

let
    PageCount = 201,
    PageIndices = {1 .. PageCount-1},
    EntitiesPerPage = 100,
    GetPage = (Index) =>
        let
            request = getRequest(Index),
            jsonResponse = Json.Document(request),
            data = jsonResponse[data]
        in
            data,
    getRequest = (pageNumber as any) => Web.Contents(
        "https://test.test.com",
        [
            Headers = [
            ],
            Query = [
                order_by = "created_at",
                include = "stats",
                per_page = "100",
                page = Text.From(pageNumber)
            ],
            RelativePath = "/api/v2/tickets"
        ]
    ),
    Pages = List.Transform(PageIndices, each GetPage(_))
in
    Pages