topic Re: Incremental refresh for Custom data connector. Store custom data. in Developer

Incremental refresh for Custom data connector. Store custom data.

RandomPBIUser — Mon, 09 May 2022 14:53:14 GMT

We have developed a custom data connector which connects to the Rest API of some sort.

Rest API supplies us with sort of "last committed transaction number", which we can use in our queries in order not to fetch all data from the server.

For example we receive LastTransaction=1000 and we can supply it as a query parameter FromTransaction=1000.

Is it possible to store the LastTransaction somewhere and use it in the next data refresh?

Re: Incremental refresh for Custom data connector. Store custom data.

Anonymous — Thu, 12 May 2022 08:03:22 GMT

Hi @RandomPBIUser ,

I don't quite understand your needs. Do you want to have a field inside the Power BI to store the maximum transaction number of the last transfer and then use that for the next refresh of the filter criteria? Please review the following links using Query parameter and check they can help you resolve the problem...

Create Dynamic Query Parameters in Power BI Desktop

Creating Dynamic Parameters in Power Query

Unable to create dynamic function parameters in Power BI Custom Connector

Best Regards

Re: Incremental refresh for Custom data connector. Store custom data.

RandomPBIUser — Thu, 12 May 2022 08:23:37 GMT

I need to store last committed transaction number somewhere and use that in the next refresh. This would avoid fetching old data I already have.

Re: Incremental refresh for Custom data connector. Store custom data.

Anonymous — Fri, 13 May 2022 09:52:03 GMT

Hi @RandomPBIUser ,

This may require a manual way to get the maximum transaction number in time after the last refresh and store it somewhere (like an excel file) and then connect to that excel file in Power BI Desktop also for your report. Create the parameters and take the values from the query associated with the excel file. You can refer to the video below to do this.

Store max transaction number in excel file

Populate a Power BI parameter list using a query

Finally, try to do some configuration in the custom connector: add a filter (like: transaction number>[parameter value]) ....

Power BI Data Connector manual

Best Regards