Forum Discussion
Incremental Refresh didn't save time
I have a very large dataset of over 25 million rows and I set up a 90-day incremental refresh for it. But the refresh took nearly as long as the usual refresh. Does anybody know how to reduce the inc...
- Hi there
It would then appear that both of those data sources do not support query folding.
Which means that it has to load all the data first, and then determine which data to load incrementally. In doing it in this manner it will not be any quicker using incremental refreshing.
Hi,
I tried to use Dataflow and Snowflake. But they were both slow. I saw the warning of query folding when I set it up. Is there any other way I can speed up the refresh without changing the data source?
Thanks,
What is your data source?
Snowflake and Dataflow
- Thanks for that.
As far as I am aware you currently cannot do incremental refreshes on a dataflow.
Have you enabled the Enhanced Dataflow ability in your Premium settings? That might help with the dataflows?
And I think the same happens for Snowflake in that it cannot create a query to just get the latest data, so it has to read all the data first to only load the incremental part.Thanks for the information. I applied incremental refresh for dataflow. But it shows the same warning as Snowflake: unable to confirm if the m query can be folded it is not recommended to use incremental refresh with non-foldable queries
Do you know what kind of data source support query folding?
Thanks,