Incremental data load using Power Query, append new records to existing table

[steph_io]

Is there a work around for the Power BI Pro users to do incremental refresh of data?

Would there be M code or R scripting that would read existing tables and only append or update records, rather than reload the entire output?

We are moving to the CDS and using Power Query within the Dataflows and it does not provide further functionality to help this limitation. 

[ImkeF]

Hi @steph_io 

if you're using dataflows, you're in a good position: http://excel-inside.pro/blog/2019/03/01/incremental-refresh-for-pro-accounts-with-power-bi-service-d...

[steph_io]

How do you connect the dataflow to an existing entity?  When you use 'Get data' there is no source for the CDS like in the PowerBI desktop tool. 

In a prior post I had asked a similar question on this subject. It seems the Dataflow always need to go back to the original source. 

https://community.powerbi.com/t5/Power-Query/Power-query-combine-tables-not-working-in-PowerApps/m-p...

Is there a way to reference an existing entity from the dataflow?

thanks

[ImkeF]

Hi @steph_io 

sorry, that was the wrong link. Just create the code in PBI Desktop and copy it to dataflows like described here: http://excel-inside.pro/blog/2019/03/21/classical-incremental-refresh-for-cloud-data-sources-in-powe... 

[v-juanli-msft]

Hi

Thanks to ImkeF for providing such useful solutions.

Although it has some limitation, it may be helpful for many cases and provides many useful techniques.

Best Regards
Maggie