Forum Discussion

Post Partisan

11 months ago

Solved

Best practice for error logging

Can someone share the best practices around pipeline creation like erro handling. logging etc in fabric.

neetesh91
11 months ago
Hi DiKi-I : The following best practices can be applied for effective error logging in Fabric pipelines.

1. Capture Errors with Try–Catch (On Failure Path)

Use the pipeline activity dependency (OnFailure, OnCompletion) to branch into an error-handling activity.
Example: If a Copy Data activity fails → trigger a Stored Procedure, Notebook, or Dataflow to log the error.

2. Create a Centralized Error Logging Table in Lakehouse or Warehouse, below is one the example of schema for Error logging table.

ActivityName
RunId
ErrorCode
ErrorMessage
ActivityType
StartTime,
EndTime
FailureTimestamp
InputParameters / SourceTarget (optional, but may be useful in debugging)

3.Similar to ADF, Fabric pipelines also expose system variables that can be used to capture runtime values in the error log table. A few examples are listed below.

@pipeline().RunId
@pipeline().DataFactory
@activity().Activity
@activity().Error.Message
@utcNow()

4. Build a proc/notebook/Dataflow to write error logs and trigger it via OnFailure or OnCompletion .

Thanks.

Community Support

11 months ago

I hope the information provided is helpful.I wanted to check whether you were able to resolve the issue with the provided solutions.Please let us know if you need any further assistance.

Thank you.