Forum Discussion

ketan5050's avatar
ketan5050
Icon for Microsoft Employee rankMicrosoft Employee
1 year ago
Solved

Filter activity not working properly

Hello community,   I have a json file which contains copyActivity.translator mappings for different tables. I am using filter activity to filter translator from this json file for a particular tabl...
  • justinjbird's avatar
    1 year ago

    The scope of a variable is at the pipeline level, not in the iteration of a foreach loop.

     

    Thinking out loud (not tested this), I assume that the absolute simplest fix would be to set the foreach to sequential, but in most circumstances that's not desired outcome.

     

    But to properly achieve this in paralleld, how I tend to approach this is produce a payload for the foreach that includes the translations rather that retrieving them separately through lookups withi the foreach. The foreach will naturally filter the set leaving you with the translations for the individual tables. You also have less contact with the database that way (i.e. you don't have lots of individual lookups happening concurrently).

     

    Without seeing the wider solution, I'm making somes assumptions but if at some point you have a lookup that is retrieving tables to process, extending that with the mappings would solve this.

  • ketan5050's avatar
    1 year ago

    Thanks, justinjbird . That sequential run solution is correct, i have already tested it but since there are very large tables to pull, it is not ideal solution. when we look at the time when set variable activities are evaluated it is 11:08:26. But for filter it is 11:08:35. it means whatever value last set variable activity set for the variable, filter will only evaluate for that variable and will always give same output for every for loop iteration (unless again set variable activity run after filter and change variable name). I had unified mapping file for all tables. i divided that into seperate files for each table and now instead of searching for specific table mapping in a large file, i am pulling table mapping directly from its mapping file using tablename and its working fine.