Forum Discussion
DiKi-I
Post Partisan
Post PartisanBest practice for error logging
Can someone share the best practices around pipeline creation like erro handling. logging etc in fabric.
Hi DiKi-I : The following best practices can be applied for effective error logging in Fabric pipelines.
1. Capture Errors with Try–Catch (On Failure Path)- Use the pipeline activity dependency (OnFailure, OnCompletion) to branch into an error-handling activity.
- Example: If a Copy Data activity fails → trigger a Stored Procedure, Notebook, or Dataflow to log the error.
2. Create a Centralized Error Logging Table in Lakehouse or Warehouse, below is one the example of schema for Error logging table.- ActivityName
- RunId
- ErrorCode
- ErrorMessage
- ActivityType
- StartTime,
- EndTime
- FailureTimestamp
- InputParameters / SourceTarget (optional, but may be useful in debugging)
3.Similar to ADF, Fabric pipelines also expose system variables that can be used to capture runtime values in the error log table. A few examples are listed below.- @pipeline().RunId
- @pipeline().DataFactory
- @activity().Activity
- @activity().Error.Message
- @utcNow()
4. Build a proc/notebook/Dataflow to write error logs and trigger it via OnFailure or OnCompletion .Thanks.
DiKi-IPost Partisan
Post PartisanDo you have any documentation around it, if you could share?