Append query to dataset and remove outdated data

[Anonymous]

I want to have a dataset in my Power BI file with 2 years worth of transactional data. This creates over 600M rows.

Instead of refreshing the entire dataset each time I run the report, is there a way to only run the query for missing data (connected to a Teradata DB) and append it to an existing data set with the majority of the data? Then I would like to remove transactional data that has exceeded 2 years.

Refreshing over 600M rows takes a few hours and I'd prefer to find a more efficient way to keep this data current.

[v-eachen-msft]

Hi @Anonymous ,

If you have a Premium license, you can use incremental refresh in the service. Here is the document for your reference.
https://docs.microsoft.com/en-us/power-bi/service-dataflows-incremental-refresh

As for the desktop, I don't think that there is anyway to actually do an incremental load on a Desktop model.

[Anonymous]

If you are using PBI Service, a data flow could take care of this.  I think its a little glitchy sometimes, but it should do data diffs so that you do not have to refresh the entire report.

Jared

[Anonymous]

Thanks for the reply. We don't have Power BI service, so I am using the desktop app. Is there any sort of solution for the desktop version?

[Anonymous]

Do you have a Pro license? Power BI Service is just the online capaiblities of Power BI.  You should have that access.

I am not aware of a solution for PBI Desktop.  I am betting there is one, just not sure. Sorry!