Data Refreshes (gateway or desktop)

[caantjes]

I am using the Gateway to daily refresh data.

The tables however contain a lot of history and it looks as if the refresh will replace all old data with new.

What is a recommended approach to just append new data ?

Can the refresh process be configured to just look for the delta rather than all ?

The current refresh took over 90 minutes which I know if not necessary.

Thanks in advance,

Corne

[ankitpatira · Accepted solution]

@caantjes Unfortunately data refresh in power bi currently is only full refresh and not delta. Below is one way you can limit data refresh.

If using sql server or other database such as Oracle you have option to enter sql queries and using that you can limit amount of rows that you bring which will also apply to data refresh such as only rows where datetime stamp is within past 30 days.

[caantjes]

Hi, thanks,

I agree through Advanced Query I can filter the dataset in PowerBI to be limited to all dates beyond a certain date.  This limits the dataset in PowerBI Desktop and subsequent publishing in Cloud. 

This does however not really accomplish my goal. I would like the old data to remain present but untouched, and append to it. Now I loose the abiltiy to present the dashboard for months prior to the selected date. 

This sounds like a common request. Can you help me ?

Corne

[Cubist]

Is this still an issue that delta refresh is not possible?

Edit:  Looks like this is still an issue.

One possible work around:

http://www.thebiccountant.com/2017/01/11/incremental-load-in-powerbi-using-dax-union/

Vote for Suggestion:

https://ideas.powerbi.com/forums/265200-power-bi-ideas/suggestions/7288623-incremental-data-loads

Thank you -