Forum Discussion
Get Fabric Pipeline activities output into log table
Hi everyone,
In a Fabric pipeline, there is an option to export the output of it:
The CSV file looks like this:
I need to log this exact information into an audit table - so after each pipeline run i want to log all the details of the activities - name, run start, run end, duration. I don't seem to find a way to do it, so your help is very much appreciated!
Thank you!
Hello Eleanordum
The Export to CSV function lets you quickly download the activities or runs you see on your screen, which can be useful for a fast review or to share by email. It only exports what you currently see in the browser, and isn’t connected to a service you can call directly.
You can turn on Workspace Monitoring in Fabric, but right now it only shows information at the pipeline level (L1). Activity-level details (L2) aren’t yet available in the Eventhouse/KQL logs.
The current workaround is to use Custom activity logging. For each important activity, you can record details like:
- The pipeline run ID
- Activity name and type
- Where the data is going (lakehouse, file, batch ID)
- Status and any error messages
- Start and end times, plus metrics
After that, you can use ItemJobEventLogs to check overall health and alerts for runs, and your custom table to look into specific activity details. This is the standard method used in organizations today.
4 Replies
- deborshi_nag
Super User
Hello Eleanordum
The Export to CSV function lets you quickly download the activities or runs you see on your screen, which can be useful for a fast review or to share by email. It only exports what you currently see in the browser, and isn’t connected to a service you can call directly.
You can turn on Workspace Monitoring in Fabric, but right now it only shows information at the pipeline level (L1). Activity-level details (L2) aren’t yet available in the Eventhouse/KQL logs.
The current workaround is to use Custom activity logging. For each important activity, you can record details like:
- The pipeline run ID
- Activity name and type
- Where the data is going (lakehouse, file, batch ID)
- Status and any error messages
- Start and end times, plus metrics
After that, you can use ItemJobEventLogs to check overall health and alerts for runs, and your custom table to look into specific activity details. This is the standard method used in organizations today.
AhmedMamdohAdvocate IV
Okay
- V-yubandi-msft
Community Support
Hi Eleanordum ,
deborshi_nag, has correctly pointed out the issue and shared a few steps to follow. Have you tried those steps, and did they help? If you are still facing any issues or need additional information, please let us know.
Thanks for your valuable response deborshi_nag .
Regards,
Yugandhar.
- V-yubandi-msft
Hi Eleanordum ,
We haven’t received a response from your end yet. Please let us know whether the issue has been resolved or if you’re still facing any difficulties. Feel free to reach out if you need further assistance.
Thank you.