Incremental Refresh is slower than a normal refresh...

[Anonymous]

I set up incremental refresh on a dataset with about 3million records.  I set the incremental refresh to work on 1 of the 18 months in the dataset.  Everything works, but it actually takes longer to refresh now, which is odd.  The refresh of the whole dataset (before) was about 3 minutes, now after it's all setup, after the initial refresh, it's around 17 minutes.  I can't wrap my head around how this can be.  Maybe incremetnal refreshes don't help for all datasets, only very, very large.  Odd that I can't benefit from from this long awaited feature. 

Any Advice on this?

[Anonymous]

Hi Team,

I use Power BI premiuim and and after implementing incrimental refresh the refresh takes more time than normal refresh.

 i need to store 2 month data ( current + last month) and refresh only current month data.

daily insert will be approax : 6 lakhs records. can you suggest me what needs to be done to get refresh faster

[GilbertQ]

Hi there

What you need to ensure is how many steps you have in the Power Query Editor which could consume time.

Also make sure that your model is using a star schema which make everything efficient.

Check to make sure your internet connection is fast to move the data if from On-Premise?

And finally make sure that there is enough CPU and memory for the incremental refresh to complete when refreshing.

[GilbertQ]

Hi there

If you are getting the data from a database source, could you possibly run Query Profiler and see what query is being sent to the source?

Also if my entire dataset was refreshing in 3 minutes and the requirement was to only refresh daily then I would leave it as is. No point trying to fix something that is working pretty quick.

[Anonymous]

I hear you on the 3 minute refresh.  I was eager to use this to save some time.  It's a big feature that I've been waiting a long time for.  There's always a more efficient solution.  I've got 100's of users using datasets, so I hate tying up the back end of the Vertipaq engine with reloading millions of stale data, potentially impacting performance.  It seems inefficient on both a system resource and time perspective.

Whatever the driver, it's extremely odd that incremental is slower than the full refresh.  Seems like total non-sense to me.  hopefully it's a bug that they'll address before making it GA in Premium.  I personally find it Disapointing.  I'm glad it helps out people running massive datasets though.

[GilbertQ]

Hi there

The reason that it possibly takes longer is due to the fact that in order to complete an incremental refresh there are other things happening in order to make this happen, such as moving current partitions, processing previous partitions if they need to be merged, creating new partitions and then processing new partitions.

All of this takes time, and incremental refresh since its inception in SSAS Multi-Dimensional has always been intended for larger datasets. And in my experience if something is taking less than 5 minutes to do a full process the time and effort involved to make this change is not worth it.