Forum Discussion
can't save dataflow one or more entities references a dynamic data source site
Hi,
The code I use is the following:
let
// Source = Json.Document (Web.Contents ("https://test.test.com/api/v2/tickets?order_by=created_at&per_page=100&include=stats")),
BaseURL = "https://test.test.com/api/v2/tickets?order_by=created_at&include=stats&per_page=100",
EntitiesPerPage = 100,
GetPage = (Index) =>
la Url = BaseURL & "& page =" & Text.From (Index),
Json = Json.Document (Web.Contents (URL)),
Data = Record.Field (Json, "data")
in Json,
// EntityCount = GetEntityCount (),
// PageCount = Number.RoundUp (EntityCount / EntitiesPerPage),
PageCount = 201,
PageIndices = {1 .. PageCount-1},
Pages = List.Transform (PageIndices, each GetPage (_))
in
sides
Based on troubleshooting, it looks like this line is failing:
Pages = List.Transform (PageIndices, each GetPage (_))
Everything works fine until I try "Save and Close" and then I get the message that the data “can't save dataflow one or more entities references a dynamic data source site"
Please let me know how to solve this issue/error?
thanx in advance.
1 Reply
- tonmcg
Resolver II
Dataflows requires the first parameter of `Web.Contents` to be a base url that contains only the domain name (host) or scheme and domain name. In your case, the host is `test.test.com` and the scheme is `https`. It would look like so:
let Source = Web.Contents("https://test.test.com") in SourceBecause you're requesting data from an API endpoint that requires pagination, you've set up your requests in a way that dynamically changes the url of each each request. Again, Dataflows requires the base url of the `Web.Contents` function to be static.
In other words, you'll need to set explicitly both the base and relative path of the url and programatically update its query parameters for each request.
Gvien that `test.test` is not a real API, there's no way for me provide testable code. But this should get you on your way:
let PageCount = 201, PageIndices = {1 .. PageCount-1}, EntitiesPerPage = 100, GetPage = (Index) => let request = getRequest(Index), jsonResponse = Json.Document(request), data = jsonResponse[data] in data, getRequest = (pageNumber as any) => Web.Contents( "https://test.test.com", [ Headers = [ ], Query = [ order_by = "created_at", include = "stats", per_page = "100", page = Text.From(pageNumber) ], RelativePath = "/api/v2/tickets" ] ), Pages = List.Transform(PageIndices, each GetPage(_)) in Pages