frithjof_v's avatar
frithjof_v
Icon for Community Champion rankCommunity Champion
5 months ago
Status:
New

Data Pipelines - Run only selected activities

For debugging and testing pipeline activities during development, allow us to select one or multiple activities and run only the selected pipeline activities.

 

For example, I'm working on editing a Notebook, and now I want to run the pipeline with that Notebook, but I don't want to run all the activities in the pipeline because that is time consuming.

 

I'm aware of the Deactivate activity option that already exists, but that requires Editing and Saving the pipeline as well. Instead, I just want to select which activities to run interactively by highlighting those activities and click Run, without needing to Edit/Save the pipeline.

 

An option to 'Run from selected activities' would also be great - select one or more activities, and run the selected activities and all subsequent activities.

3 Comments

  • This would be useful. I end up enabling / disabling steps in a pipeline to test, and it does get confusing if you're jumping on multiple things at work.
  • ChrisClegg's avatar
    ChrisClegg
    Frequent Visitor

    Yes, the same as abhidotnet, I end up enabling / disabling activities to test.  The ability to run selected, especially without saving the pipeline, would be really useful.  +Vote from me.

Recent ideas