Forum Discussion
skip activity
Hi Anonymous,
I suggest you can build a small framework that reads json config files (or you can put it in some table in WH as well) and runs the activities based on json config. For example, let's say you have 15 notebooks to run and some of the notebooks have some sequential activity that is based on if condition.
You could put all the 15 notebooks and their corresponding sequences in a json config file/WH Table. Have a lookup to read that metadata and then pass it to for-each. Inside for-each you can choose to have the notebook run and then the if-else condition.
This way your pipeline becomes scalable and easier to maintain.
Hi Anonymous ,
Thank you govindarajan_d for the helpful response!
I just wanted to kindly follow up to see if you had a chance to review the previous response provided by Govindarajan. I hope it was helpful. If yes, please Accept the answer so that it will be helpful to others to find it quickly.
Thank you.