Forum Discussion
How create API with paging ?
There's a couple of things that immediately spring to mind:
1) Have you set up your paged call like this, using RelativePath?
Using this RelativePath structure for the Web.Contents is a key aspect of avoiding Dynamic Data Source errors.
You may need to adjust which parts of the URL sit within the core/RelativePath sections.
You may also need to use the same structure for your original ping call, but I've never had issues with this initial call being the basic Web.Contents structure.
2) Have you deleted all unused queries in the Dataflow that don't use the correct Web.Contents structure?
I've found that, even if incorrectly-structured calls are disabled from loading, they still cause the entire DF to fail on initial query validation. Not sure why, just something I've experienced.
Pete
BA_Pete many thanks - your breakdown in steps is very helpful! By following your guidance I found out the total number of records is more than 25 million... way too much! I first need to figure out how I can make sure the endpoint only returns records up until a specific date, which doesn't seem to be a parameter I can choose from. The dates only become visible after expanding some records and lists. Any ideas how to tackle this in an efficient manner?
There will be a way to filter on fields that are nested within the records, but you'll need to read the API documentation for your specific source.
For example, on one of the sources I use, I can filter nested fields using a parameters like this:
&_ff[]=picklist_id
&_ft[]=eq
&_fc[]=190Where:
_ff[] is the field to filter
_ft[] is the filter condition
_fc[] is the comparator value
So, this group of parameters filters the records to only those where the [picklist_id] field equals 190.
As above, you'll need to read your own API docs to find out how your endpoint needs the citeria presented, but I'm 99.9% sure it's doable.
Pete
Hi BA_Pete I managed to follow your instructions sucessfully, which is great!
However, when I want to save and close the dataflow, I get this error: One or more tables references a dynamic data source... is there any way to solve this?
There's a couple of things that immediately spring to mind:
1) Have you set up your paged call like this, using RelativePath?
Using this RelativePath structure for the Web.Contents is a key aspect of avoiding Dynamic Data Source errors.
You may need to adjust which parts of the URL sit within the core/RelativePath sections.
You may also need to use the same structure for your original ping call, but I've never had issues with this initial call being the basic Web.Contents structure.
2) Have you deleted all unused queries in the Dataflow that don't use the correct Web.Contents structure?
I've found that, even if incorrectly-structured calls are disabled from loading, they still cause the entire DF to fail on initial query validation. Not sure why, just something I've experienced.
Pete
Hi BA_Pete - your 1)-suggestion was indeed the issue. The original ping call needed a RelativePath, and that solved the problem. Thanks so much, you're a star! 😊 🌟