Run spark job definitions with arguments

[MBroeks]

Hello,

My idea is a small feature request:

Within spark job definitions it's possible to pass command line arguments to the spark job.

I would like the possible to call my spark job definition from a pipeline activity and be able to pass command line arguments.

This arguments field in the spark job definition activity should be able to contain dynamic content.

This makes spark jobs more flexible and creates the possibility to make them metadata driven from a pipeline.

Kind regards,

Martijn

[nishalit]

Hello,

My idea is a small feature request:

Within spark job definitions it's possible to pass command line arguments to the spark job. 

I would like the possible to call my spark job definition from a pipeline activity and be able to pass command line arguments.

This arguments field in the spark job definition activity should be able to contain dynamic content.

This makes spark jobs more flexible and creates the possibility to make them metadata driven from a pipeline.

Kind regards,

Martijn

[nishalit]

Currently a work in progress