Forum Discussion
Helper IIPipeline Activity with multiple conditions
Hi folks,
i have a real basic question that I am not finding the answer to:
Will the "TMM Mapping " Notebook in the example above wait for both the "SOP Bucket Creator" and "Load TMM DB" before it starts to run ?
Thans for your help!
Joachim
Hi JoachimSA
Thanks for using Fabric Community.
I have created a repro of your scenario:
1) Inititally the Dataflow1 and Dataflow2 will start running.
2) After this the Notebook1 starts running.
3) If both Notebook1 and Dataflow2 run successfully , then the Notebook2 starts running.
Hope this helps. Please let me know if you have any further questions.
7 Replies
- NandanHegde
Super User
yes, it would wait for both the previous activities to be completed and that too succesfully completed before it starts its execution.
The dependencies in data pipe lines acts as an AND expression.
Hi JoachimSA
Thanks for using Fabric Community.
I have created a repro of your scenario:
1) Inititally the Dataflow1 and Dataflow2 will start running.
2) After this the Notebook1 starts running.
3) If both Notebook1 and Dataflow2 run successfully , then the Notebook2 starts running.
Hope this helps. Please let me know if you have any further questions.- JoachimSA
Perfect thank you!
Just to confirm: all activities without a condition will be triggered on pipeline start ?
I have a scenario where i would want to switch off certain parts of the pipeline depending on which input data was changed in the background so I do not need to run through everything for every small update of some side info.
are there any ressources on how one would switch off certain branches of the tree of activities ?
If an activity is switched off but is still connected to the following node it will be taken out of the condition for the following node to run right ?
I just wasn able to find very good documentation on those very basic questions...do you have a hint for me ?
Thanks and best regards,
Joachim
- NandanHegde
Can you plz provide a normal PPT/ visual representaion of what you want to achieve?
To ans to your query, yes
all activities without any dependency would run paralley on pipeline start.
And you can switch / turn off activities based on multiple approaches like leveraging IF or Switch activity or via dependencies.A visual representation of your ask would help us help you better
- JoachimSA
Hi Nandan,
so depending on which inputs changed i would like to easily exclude each of the 3 Highlighted Blocks individually from updating.
As you can See, at the moment i just deactivated some of them. Would that be the way to go ? Longer branches get increasingly cumbersome to switch on and of then.
Also: Each activity write a deltatable at the end which is the used by the next one. Is that the optimal way when I work in a Lakehouse ?
