ADF already has a feature for retrying pipeline activities on error.
The problem that we face here is that a retry only makes sense if there are transient (mostly network) problems.
But if there is an error with the data, we can't simply retry the activity since we need to fix the data first.
That is why I think it would be very helpful if we could define a condition for a retry.
Practically I am thinking about defining an expression that returns a Boolean (https://learn.microsoft.com/en-us/azure/data-factory/control-flow-expression-language-functions).
This way we could analyze the error message and look for error codes or specific keywords "timeout", for deciding if a retry makes sense.
frithjof_v
on:
Job scheduler API: List failed runs
anshulsharma
on:
Automated export for Eventhouse data to Lakehouse ...
on:
REST API for removing all tags