Execution Of Child Data Pipeline based on Master Pipeline Output

[AmruthaVarshini]

Hi Team,

I am facing an issue when trying to Execute my 'child pipeline' based on output of 'Master Pipeline'.

Here is the scenario:

1. Master Pipeline has a 'ForEach loop' activity which does export of data to SQL Server.

2. Copy data activity is used inside the ForEach loop to do the above.

3. Also Exception handling is done for Copy data activity(i.e. in case of failure of any export of data, it sends out an email to users)

4. Once loop is successfully executed, it should execute the next pipeline-child.

5. Above is done based on 'Execute Pipeline' Activity.

Now the conern in the above is, whenever an export of Data(i.e. CopyData) activity is failing it is sending out an mail which is expected, but it is returning the status of this pipeline as Success.

But my child pipeline is dependent on the success of CopyData Activity but not the success of pipeline. To sum up, child pipeline should run only when export of data is completed successfully for all the iterations. Even if one of the iteration fails, child pipeline should not be executed.

I tried using 'Fail' Activity and also 'Append Variable' activity to capture the errors, but it is not working.

Could you please let me know how to achieve this Scenario in Fabric? Also Please provide any insights or references you thought would be helpful to achieve this.

Thank you.

[NandanHegde]

Can you please share the image of your flow within the foreach activity?

In case if the copy activity has only 1 activity attached to it with on failure dependency, even though the copy activity fails, your pipeline status would be a success.

Please refer the end of below blog for detailed understanding :

https://datasharkx.wordpress.com/2021/08/19/error-logging-and-the-art-of-avoiding-redundant-activiti...

[AmruthaVarshini]

Hi @NandanHegde ,

Attaching the image of flow here. Please let me know your inputs.

[NandanHegde · Accepted solution]

The pipeline is a sucess because of the below combination of copy activity :

if you want the pipeline to fail replace it with below combination

[AmruthaVarshini]

Hi @NandanHegde ,

Thanks a lot!! It worked out, saved my day.

[Anonymous]

Hi @AmruthaVarshini ,

Glad to know that your query got resolved. Please continue using Fabric Community on your further queries.