Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
6 years ago

can't save dataflow one or more entities references a dynamic data source site

Hi, 

The code I use is the following:

 

let

// Source = Json.Document (Web.Contents ("https://test.test.com/api/v2/tickets?order_by=created_at&per_page=100&include=stats")),

BaseURL = "https://test.test.com/api/v2/tickets?order_by=created_at&include=stats&per_page=100",

EntitiesPerPage = 100,

GetPage = (Index) =>

la Url = BaseURL & "& page =" & Text.From (Index),

Json = Json.Document (Web.Contents (URL)),

Data = Record.Field (Json, "data")

in Json,

// EntityCount = GetEntityCount (),

// PageCount = Number.RoundUp (EntityCount / EntitiesPerPage),

PageCount = 201,

PageIndices = {1 .. PageCount-1},

Pages = List.Transform (PageIndices, each GetPage (_))

in

sides

 

Based on troubleshooting, it looks like this line is failing:

Pages = List.Transform (PageIndices, each GetPage (_))

 

Everything works fine until I try "Save and Close" and then I get the message that the data “can't save dataflow one or more entities references a dynamic data source site"

Please let me know how to solve this issue/error?


thanx in advance.

1 Reply

  • Dataflows requires the first parameter of `Web.Contents` to be a base url that contains only the domain name (host) or scheme and domain name. In your case, the host is `test.test.com` and the scheme is `https`. It would look like so:

     

    let
        Source = Web.Contents("https://test.test.com")
    in
        Source

     

    Because you're requesting data from an API endpoint that requires pagination, you've set up your requests in a way that dynamically changes the url of each each request. Again, Dataflows requires the base url of the `Web.Contents` function to be static.

     

    In other words, you'll need to set explicitly both the base and relative path of the url and programatically update its query parameters for each request.

     

    Gvien that `test.test` is not a real API, there's no way for me provide testable code. But this should get you on your way:

    let
        PageCount = 201,
        PageIndices = {1 .. PageCount-1},
        EntitiesPerPage = 100,
        GetPage = (Index) =>
            let
                request = getRequest(Index),
                jsonResponse = Json.Document(request),
                data = jsonResponse[data]
            in
                data,
        getRequest = (pageNumber as any) => Web.Contents(
            "https://test.test.com",
            [
                Headers = [
                ],
                Query = [
                    order_by = "created_at",
                    include = "stats",
                    per_page = "100",
                    page = Text.From(pageNumber)
                ],
                RelativePath = "/api/v2/tickets"
            ]
        ),
        Pages = List.Transform(PageIndices, each GetPage(_))
    in
        Pages