Forum Discussion
Updating Datasource without loading from start
Im looking for a solution to update my large database without reloading everything. just import new records.
My Data source is coming from postgres sql view (getting data every 24 hours in a 1 hour split (time column) ).
How it should be:
My BI has data from 12-11-2019 till 12-18-2019. Im refreshing/updating my datasource for new data of 12-19.2019.
Instead of starting on 12-11-2019, it should start on 12-19-2019. So it just append the new imported data to the already existing data.
My Idea
using Advance Settings -> SQL Command (On the datasource import settings)
And somthing like this
where source.timecolumn > Last(bi-datatable.timecolumn)
tl;dr
My Import should not begin from 0. 😛
Hi Macwin ,
If the PBIX file was authored in a version of Power BI Desktop prior to the February release, the warning will be displayed regardless until a data refresh is performed.
Reference:
https://powerbi.microsoft.com/en-us/blog/incremental-refresh-query-folding/
Best Regards,
Amy
Community Support Team _ Amy
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
5 Replies
- You can only import incremental data using power BI premium otherwise it is always going to be full refresh.
Hi Macwin
You can use incremental Refresh
https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refreshOr Direct query mode.
Best Regards,
Mariusz
If this post helps, then please consider Accepting it as the solution.Unfortunately, Incremental Refreshes (that is what you are after) is only available in PowerBI Premium:
See here: https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh
What you could do is implement a DirectQuery on some sort of Datawarehouse solution, in which you perform the necessery transformations that you are currently doing in PowerQuery. That way, your report is always up to date but you will move your problem back in your data pipeline. This would entail a re-architecturing effort of your data pipeline.
Kind regards
Djerro123
-------------------------------
If this answered your question, please mark it as the Solution. This also helps others to find what they are looking for.
Keep those thumbs up coming! 🙂
Im using Premium service and I already found the page about incremental refresh (But I dont know how it should work) and I cant use query mode because of the big dataset im using. (lifetime aggregations)
Also I dont even need power query.
I cant get further till step
https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh#query-folding
its always disabled.
How I should I set up the incremental refresh on my my example above ?
Hi Macwin ,
If the PBIX file was authored in a version of Power BI Desktop prior to the February release, the warning will be displayed regardless until a data refresh is performed.
Reference:
https://powerbi.microsoft.com/en-us/blog/incremental-refresh-query-folding/
Best Regards,
Amy
Community Support Team _ Amy
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
