Forum Discussion
Error handling pattern for ForEach with partial failures: continue and report, or fail fast?
- 6 days ago
Thank you v-saisrao-msft. Here is the summary of the pattern from this thread.
Short answer: let the loop continue past failures, record each failure, then raise one consolidated failure after the ForEach so the run status still shows red and alerts fire.
Inside ForEach -> Copy activity; its Upon Failure path logs table name + error to a control table Continue -> connect both Upon Failure and Upon Skip from the logging activity to the next activity After loop -> Lookup: failure count for pipeline().RunId in the control table Decision -> If Condition: count > 0 -> Fail activity with a clear summary message; else succeed Parallel runs -> write independent failure rows instead of incrementing a shared variable; make sure the logging target handles concurrent writesThe final Fail activity is what keeps the pipeline run status accurate, so monitoring and alerts still work even though individual tables did not stop the loop.
AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.
Thank you v-saisrao-msft. Here is the summary of the pattern from this thread.
Short answer: let the loop continue past failures, record each failure, then raise one consolidated failure after the ForEach so the run status still shows red and alerts fire.
Inside ForEach -> Copy activity; its Upon Failure path logs table name + error to a control table
Continue -> connect both Upon Failure and Upon Skip from the logging activity to the next activity
After loop -> Lookup: failure count for pipeline().RunId in the control table
Decision -> If Condition: count > 0 -> Fail activity with a clear summary message; else succeed
Parallel runs -> write independent failure rows instead of incrementing a shared variable;
make sure the logging target handles concurrent writesThe final Fail activity is what keeps the pipeline run status accurate, so monitoring and alerts still work even though individual tables did not stop the loop.
AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.