Forum Discussion
Paginated REST API - Reduce number of requests and handling updated data
Incremental refresh won't help you with that as the partition ranges are managed automatically by the service. You can only choose between day, month, quarter or year.
I would instead use an external storage.
Hi lbendlin,
Thank you for your answer. That is very useful !
I am indeed only interested in the latest status.
Based on the join kind schema from Power Query :
If we consider my update table as the left one (Query name : Day1-Update) and the right one as the initial table (query name : Day1-Initial).
Should I do as follow ?
- A Right anti join as you said to retreive all the visitors without updates from "Day1-Initial"
- Merge "Day1-Update" with "Day1-Initial" (query name : Day1-Result)
I am missing something, How am i supposed to keep the data for the next day ?
On day 2, "Day2-Initial" must be equal to "Day1-Result" otherwise, the result of "Day1-update" will be lost and erased by "Day2-Update". Does that make sens ?
Thank you for your time !
I am missing something, How am i supposed to keep the data for the next day ?You could take a risk and self reference your semantic model for that. Personally I would rather store my snapshots in a safe place.
Hi lbendlin
Here is the model that I try to implement thanks to your insights :
Can I apply an incremental refresh to this case in order to have the following result
The parameters RangeStart and RangeEnd (parameters requested for the Incremental refresh in PowerBI) would be :
RangeStart = Last modified Date
RangeEnd = Timestamp of refresh
Is it tricky with a paginated REST API ?
Thank you
Incremental refresh won't help you with that as the partition ranges are managed automatically by the service. You can only choose between day, month, quarter or year.
I would instead use an external storage.