Forum Discussion
Solved
Best practice for error logging
Can someone share the best practices around pipeline creation like erro handling. logging etc in fabric.
Hi DiKi-I : The following best practices can be applied for effective error logging in Fabric pipelines.
1. Capture Errors with Try–Catch (On Failure Path)- Use the pipeline activity dependency (OnFailure, OnCompletion) to branch into an error-handling activity.
- Example: If a Copy Data activity fails → trigger a Stored Procedure, Notebook, or Dataflow to log the error.
2. Create a Centralized Error Logging Table in Lakehouse or Warehouse, below is one the example of schema for Error logging table.- ActivityName
- RunId
- ErrorCode
- ErrorMessage
- ActivityType
- StartTime,
- EndTime
- FailureTimestamp
- InputParameters / SourceTarget (optional, but may be useful in debugging)
3.Similar to ADF, Fabric pipelines also expose system variables that can be used to capture runtime values in the error log table. A few examples are listed below.- @pipeline().RunId
- @pipeline().DataFactory
- @activity().Activity
- @activity().Error.Message
- @utcNow()
4. Build a proc/notebook/Dataflow to write error logs and trigger it via OnFailure or OnCompletion .Thanks.
Can you state as to which components within MSFT Fabric are you using? As the logging / error framework can differ based on your architectural flow and the offerings ( data pipelines, Dataflows ,notebooks etc) being used.
I would need it for notbooks and Pipelines in fabric
