pipeline trigger using metadata table

Hi DiKi-I, 

For execution order, I am using the dependencies column in my registry table. It contains a dataset like so: {bronze:1, silver:3}
this would mean the job depends on bronze job 1 and silver job 3. 
Then I am building a directed acyclic graph to determine the order of jobs to run, and I am executing the jobs in that order, waiting for them to finish, then executing the next later. 

I also have my notebook set up to be able to run jobs concurrently, so if there's 15 jobs with no dependencies, they will all start at the same time. If you're doing something similar, keep in mind your capcaity usage. I am using F64 capacities, if you're using a smaller capacity or your pipelines are very complex, you might run into CU limits. 

If you found this helpful, consider giving some kudos. If I answered your quesiton or solved your problem, mark this post as the solution so future community members can find it easily.