Forum Discussion
How could I append rows to the table created through custom connector?
Hi,
I am new to Power Bi and currently, I am trying to implement a solution using custom connectors.
In this custom connector, I bring the data through rest APIs. So, I think it won't have any query folding support. The source from which I bring the data has many rows (100s of thousands daily), and I want to bring that data to Power BI. Now, is there any way that we can only fetch the latest data only?
For example, if I have collected the data from 1st Jan 2020 to 31st March 2020. Then, is there any way that when I do the next refresh, I only bring the data from 31st March 2020 to today's date, let's say 7th April 2020, and append that data to the data that I have previously fetched?
While going through the Github connector, I found that the data is fetched again every time and is re-written to the table. So, is there a way to preserve the data which is already fetched and only add the new data pulled from the API?
Not generally, incremental refresh is only available for a certain few types of data sources
2 Replies
Not generally, incremental refresh is only available for a certain few types of data sources
In this case, it would be great if you help me determine what would be a good way to implement this particular use case? I tried with the API of powerbi, but it has limit of maximum number of rows (5 million).
Again, thanks for the quick reply.
