Data pipeline failed. How do you troubleshoot?

Hi rajan889262,

Are you stuck on a specific case or just a normal step by step? 

If the second option, when a pipeline fails in Fabric, check the run details first, then dig into the specific activity that failed. Here's the usual step by step you can follow:

1. Open the workspace and go to Monitor (or the pipeline item itself), find the failed run and open it.
2. Look at the pipeline canvas view for that run, the activity with the red X is the one that failed.
3. Click on that activity and check the Output/Error tab, it shows the error code and message.
4. If it's a Copy activity, check the Copy activity details for source/sink error, rows read/written, and duration, this narrows down if it's a connectivity, permission or data issue.
5. If it's a Notebook or Dataflow Gen2 activity, open the linked notebook/dataflow run directly, the pipeline error message is usually generic and the real stack trace is inside the notebook/dataflow run history.
6. Check activity run consumption/duration too, sometimes it's a timeout or capacity throttling (look at Capacity Metrics app if you suspect CU exhaustion).
7. If using parameters or expressions, verify the resolved values in the Input tab, a lot of failures come from a bad parameter passed at runtime.
8. Retry the single failed activity if the pipeline supports it, instead of rerunning the whole pipeline, to save cost and time while testing your fix.

real example: 

Hope this can clarify you 🙂

Best regards,

Carlos