Forum Discussion
Making do without a container (and without excessive duplication)
Hello everyone,
How do you develop your data pipelines without containers (as available on SSIS for example) and without duplicating things ?
For example, here I have a flow that calls two child pipelines and I want it to report an error if one of them is in error (so that the others can continue to load or run).
Here I have two child pipelines, so two failed components. But if I have to do this for 20 tables, I'll have to make 20 failed components...
Whereas with a container, you put the 20 components inside and if one is in error, the error will be reported but the other tables will continue to load anyway.
Thank in advance for your return,
Have a nice day,
Vivien
plz refer the below blog :
wherein you can create a generic pipeline framework to capture the logs via 1 activity only rather than having it duplicated across every activity
4 Replies
plz refer the below blog :
wherein you can create a generic pipeline framework to capture the logs via 1 activity only rather than having it duplicated across every activity
Thanks for your feedback.
It does work, but I don't think it's very clean.
Is a container planned in the roadmap?
ADF has been in picture since last couple of years and Fabric data pipelines is hosted on top of it.
So I do not foresee the above request on any immediate basis , but you can still create a feedback for the product team to work on if need be.
