Forum Discussion
using a column max value to create an incremental refresh
Not directly possible. Power BI Incremental Refresh is exclusively based on datetime partition boundaries. You can create your own partitions with different boundary rules but that would require you to re-invent all the partition management processes.
Does your source table have a suitable date or datetime column that you can use instead?
If your data is immutable then you can also consider running your own extract process based on your described logic, and place the results in a sharepoint folder so Power BI can pick the extracts up from there.
1.5M rows is not big, and does not normally warrant incremental refresh. Your mentioned timeout is a valid reason, however.
Thanks for the reply, I do have a datetime column, but what I'm struggling with is inserting that into the web API url to modify the actual query sent to the source. All the incremental query seems to do is filter the results of the source query, and since I can't query all of the rows in one go without it timing out, I'm not sure if that will work for us. Whereas it's a lot easier to just say "skip this many rows and pull the next 100k". As you say timeout is the only reason I need it to be incremental, although we are trying to be frugal with compute usage.