Forum Discussion

wintergolem's avatar
wintergolem
Frequent Visitor
4 years ago
Solved

Dataflow with incremental refresh doing a full table load x16 times

I'm working on adding incremental refresh to a fact table in my dataflow, unfortunately when this dataflow does it's next refresh I see 16 processes on my mysql instance that are all performing the s...
Need Help
  • KNP's avatar
    KNP
    4 years ago

    With the information given, the first place I'd be checking is to ensure query folding is occurring.

    I assume query folding works with MySQL the same as MSSQL?

    My understanding is, without query folding, the advantages of incremental refresh are redundant.

     

KNP's avatar
KNP
Super User
4 years ago

With the information given, the first place I'd be checking is to ensure query folding is occurring.

I assume query folding works with MySQL the same as MSSQL?

My understanding is, without query folding, the advantages of incremental refresh are redundant.

 

wintergolem's avatar
wintergolem
Frequent Visitor
4 years ago

After messing around with it I think you're right, but I'm confident everything was folding eariler (before I put a keep-rows filter for testing). I'll see if it runs correctly now though

  • KNP's avatar
    KNP
    Super User
    4 years ago

    Keep an eye on the query folding icons.

    I had issues this morning with a merge (Inner Join) not folding but Left Join does (at least that's what the icon suggested).

    Also, another issue to watch out for, if using Comparer.OrdinalIgnoreCase, it will break the query folding. As well as many other things. ğŸ˜†