topic How to get metadata from a pipeline output in Pipelines

How to get metadata from a pipeline output

Anonymous — Mon, 29 Jul 2024 17:26:16 GMT

Hello,

I'm looking for a solution to get the matedata from the output of a pipeline.

The goal is simple, i want to get the link and the name af the activity and take thoses in a table.

If you have any ideas about how can i do this ?

Ben

Re: How to get metadata from a pipeline output

Anonymous — Tue, 30 Jul 2024 01:42:09 GMT

Hi Anonymous ,

To use a Get Metadata activity in a pipeline, complete the following steps:

1. Select the new Get Metadata activity on the canvas if it is not already selected, and its Settings tab, to edit its details.

2. Choose a dataset, or create a new one with the New button. Then you can specify filter options and add columns from the available metadata for the dataset.

3. Use the output of the activity as an input to another activity, like a Switch activity in this example. You can reference the output of the Metadata Activity anywhere dynamic content is supported in the other activity.

4. In the dynamic content editor, select the Get Metadata activity output to reference it in the other activity.

If you want to learn more about this, you can look at this document: Get Metadata activity - Azure Data Factory & Azure Synapse | Microsoft Learn

Best Regards

Yilong Zhou

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Re: How to get metadata from a pipeline output

Anonymous — Tue, 30 Jul 2024 08:05:52 GMT

Hi Anonymous

Your idea is great but this is not a solution for my case.

In the pipeline's execution list we can take a csv file back. Do you know if i can take this back in a notebook with spark ?