skip activity

Hi Anonymous,

I suggest you can build a small framework that reads json config files (or you can put it in some table in WH as well) and runs the activities based on json config. For example, let's say you have 15 notebooks to run and some of the notebooks have some sequential activity that is based on if condition. 

You could put all the 15 notebooks and their corresponding sequences in a json config file/WH Table. Have a lookup to read that metadata and then pass it to for-each. Inside for-each you can choose to have the notebook run and then the if-else condition. 

This way your pipeline becomes scalable and easier to maintain.