Forum Discussion
Incremental Refresh didn't save time
I have a very large dataset of over 25 million rows and I set up a 90-day incremental refresh for it. But the refresh took nearly as long as the usual refresh. Does anybody know how to reduce the inc...
- Hi there
It would then appear that both of those data sources do not support query folding.
Which means that it has to load all the data first, and then determine which data to load incrementally. In doing it in this manner it will not be any quicker using incremental refreshing.
It would appear to be from the following documentation
https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh#limitations
If you read further up it says it should display the warning.
Are both the snowflake and the dataflow used in the same PBIX?
https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh#limitations
If you read further up it says it should display the warning.
Are both the snowflake and the dataflow used in the same PBIX?
No. I tried the same pbix file with two data sources and they both didn't work.
Hi Anonymous
I am fairly certain that if you are combining the 2 data source that the incremental refreshing process has to load all the data before it can incrementally load the data.
And that is why you are not seeing it saving any more time.
What happens if you try them individually?
Hi GilbertQ ,
Thank you for your advice. I didn't make it clearly. I use dataflow and snowflake individually for incremental refresh and they both didn't work.
Thanks,
- Hi there
It would then appear that both of those data sources do not support query folding.
Which means that it has to load all the data first, and then determine which data to load incrementally. In doing it in this manner it will not be any quicker using incremental refreshing. - No worries, glad to help
Incremental refreshing can be a bit of a challenge at times that is for sure. But when it works, it works really well!