Updating Datasource without loading from start

[Macwin]

Im looking for a solution to update my large database without reloading everything. just import new records.

My Data source is coming from postgres sql view (getting data every 24 hours in a 1 hour split (time column) ).

How it should be:

My BI has data from 12-11-2019 till 12-18-2019.  Im refreshing/updating my datasource for new data of 12-19.2019. 

Instead of starting on 12-11-2019, it should start on 12-19-2019. So it  just append the new imported data to the already existing data.

My Idea 

using Advance Settings -> SQL Command (On the datasource import settings)

And somthing like this

where source.timecolumn > Last(bi-datatable.timecolumn)

tl;dr 

My Import should not  begin from 0. 😛

[Macwin]

Im using Premium service and I already found the page about incremental refresh (But I dont know how it should work) and I cant use query mode because of the big dataset im using. (lifetime aggregations)

Also I dont even need power query.

I cant get further till step 

https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh#query-folding

its always disabled. 

How I should I set up the incremental refresh on my my example above ?

[v-xicai · Accepted solution]

Hi @Macwin ,

If the PBIX file was authored in a version of Power BI Desktop prior to the February release, the warning will be displayed regardless until a data refresh is performed. 

Reference:

https://powerbi.microsoft.com/en-us/blog/incremental-refresh-query-folding/

Best Regards,

Amy

Community Support Team _ Amy

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[JarroVGIT]

Unfortunately, Incremental Refreshes (that is what you are after) is only available in PowerBI Premium:

See here: https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh

What you could do is implement a DirectQuery on some sort of Datawarehouse solution, in which you perform the necessery transformations that you are currently doing in PowerQuery. That way, your report is always up to date but you will move your problem back in your data pipeline. This would entail a re-architecturing effort of your data pipeline.

Kind regards

Djerro123

-------------------------------

If this answered your question, please mark it as the Solution. This also helps others to find what they are looking for.

Keep those thumbs up coming! 🙂

[Mariusz]

Hi @Macwin 

You can use incremental Refresh 
https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh

Or Direct query mode.

Best Regards,
Mariusz

If this post helps, then please consider Accepting it as the solution.

[parry2k]

You can only import incremental data using power BI premium otherwise it is always going to be full refresh.