Forum Discussion
Split complex dataflow into smaller dataflows
Thanks for your reply AnkitKukreja
Let me rephase my question,
when you deal with a complex workflow with lots of step to extract, merge based off some conditions; the condition is the check point for new subset of data. Since dataflow is chained together, it seems that you can only do single output from upstream to be input from downstream within the same workspace, you basically cannot pass multiple subset data playing as conditional checkpoint to downstream dataflow to continue define your workflow and this make the workflow become huge and complex. How to you split such workflow in your practice when there are multiple inputs needed from the upstream dataflow? Not by the best practice links that you sent and I have read multiple times as it's very basic conceptual suggestion.
Best regards,
R.