Forum Discussion
Solved
Dataflow with incremental refresh doing a full table load x16 times
I'm working on adding incremental refresh to a fact table in my dataflow, unfortunately when this dataflow does it's next refresh I see 16 processes on my mysql instance that are all performing the s...
With the information given, the first place I'd be checking is to ensure query folding is occurring.
I assume query folding works with MySQL the same as MSSQL?
My understanding is, without query folding, the advantages of incremental refresh are redundant.
After messing around with it I think you're right, but I'm confident everything was folding eariler (before I put a keep-rows filter for testing). I'll see if it runs correctly now though
Keep an eye on the query folding icons.
I had issues this morning with a merge (Inner Join) not folding but Left Join does (at least that's what the icon suggested).
Also, another issue to watch out for, if using Comparer.OrdinalIgnoreCase, it will break the query folding. As well as many other things. 😆