Currently, after a pipeline execution, you can view both the input and output JSON for each activity. However, when using this information during execution (for example, as dynamic content in downstream activities or scripts), only the output JSON is accessible.
The problem is that the output JSON does not contain the ID of the notebook or dataflow being executed, while this information is available in the input JSON. As a result, it is not possible to dynamically retrieve important metadata such as the notebook or dataflow ID during pipeline execution.
I propose one of the following solutions:
- Include the activity input as an additional node within the output JSON, making the input metadata available through dynamic content.
- Alternatively, allow the activity input JSON to be directly accessible as dynamic content from subsequent activities.
Either approach would solve the issue and enable downstream activities to access important metadata, such as notebook and dataflow IDs, without requiring hardcoded values or manual maintenance.
