Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply

Execution Of Child Data Pipeline based on Master Pipeline Output

Hi Team,

 

I am facing an issue when trying to Execute my 'child pipeline' based on output of 'Master Pipeline'.

Here is the scenario:

 

1. Master Pipeline has a 'ForEach loop' activity which does export of data to SQL Server.

2. Copy data activity is used inside the ForEach loop to do the above.

3. Also Exception handling is done for Copy data activity(i.e. in case of failure of any export of data, it sends out an email to users)

4. Once loop is successfully executed, it should execute the next pipeline-child.

5. Above is done based on 'Execute Pipeline' Activity.

 

Now the conern in the above is, whenever an export of Data(i.e. CopyData) activity is failing it is sending out an mail which is expected, but it is returning the status of this pipeline as Success.

 

But my child pipeline is dependent on the success of CopyData Activity but not the success of pipeline. To sum up, child pipeline should run only when export of data is completed successfully for all the iterations. Even if one of the iteration fails, child pipeline should not be executed.

 

I tried using 'Fail' Activity and also 'Append Variable' activity to capture the errors, but it is not working.

 

Could you please let me know how to achieve this Scenario in Fabric? Also Please provide any insights or references you thought would be helpful to achieve this.

 

Thank you.

 

1 ACCEPTED SOLUTION

The pipeline is a sucess because of the below combination of copy activity :

NandanHegde_0-1718883228491.png

if you want the pipeline to fail replace it with below combination

NandanHegde_1-1718883293582.png

 

View solution in original post

5 REPLIES 5
NandanHegde
Super User
Super User

Can you please share the image of your flow within the foreach activity?

In case if the copy activity has only 1 activity attached to it with on failure dependency, even though the copy activity fails, your pipeline status would be a success.

 

Please refer the end of below blog for detailed understanding :

https://datasharkx.wordpress.com/2021/08/19/error-logging-and-the-art-of-avoiding-redundant-activiti...

 

Hi @NandanHegde ,

 

Attaching the image of flow here. Please let me know your inputs.

 

MicrosoftTeams-image (1).png

The pipeline is a sucess because of the below combination of copy activity :

NandanHegde_0-1718883228491.png

if you want the pipeline to fail replace it with below combination

NandanHegde_1-1718883293582.png

 

Hi @NandanHegde ,

 

Thanks a lot!! It worked out, saved my day.

Hi @AmruthaVarshini ,

Glad to know that your query got resolved. Please continue using Fabric Community on your further queries.

Helpful resources

Announcements
FabricCarousel_June2024

Fabric Monthly Update - June 2024

Check out the June 2024 Fabric update to learn about new features.

July Newsletter

Fabric Community Update - July 2024

Find out what's new and trending in the Fabric Community.