Forum Discussion
Data pipeline failed. How do you troubleshoot?
Data pipeline failed. How do you troubleshoot?
Hi rajan889262,
Are you stuck on a specific case or just a normal step by step?
If the second option, when a pipeline fails in Fabric, check the run details first, then dig into the specific activity that failed. Here's the usual step by step you can follow:
- Open the workspace and go to Monitor (or the pipeline item itself), find the failed run and open it.
- Look at the pipeline canvas view for that run, the activity with the red X is the one that failed.
- Click on that activity and check the Output/Error tab, it shows the error code and message.
- If it's a Copy activity, check the Copy activity details for source/sink error, rows read/written, and duration, this narrows down if it's a connectivity, permission or data issue.
- If it's a Notebook or Dataflow Gen2 activity, open the linked notebook/dataflow run directly, the pipeline error message is usually generic and the real stack trace is inside the notebook/dataflow run history.
- Check activity run consumption/duration too, sometimes it's a timeout or capacity throttling (look at Capacity Metrics app if you suspect CU exhaustion).
- If using parameters or expressions, verify the resolved values in the Input tab, a lot of failures come from a bad parameter passed at runtime.
- Retry the single failed activity if the pipeline supports it, instead of rerunning the whole pipeline, to save cost and time while testing your fix.
real example:
Hope this can clarify you 🙂
Best regards,
Carlos
4 Replies
Hi rajan889262,
Are you stuck on a specific case or just a normal step by step?
If the second option, when a pipeline fails in Fabric, check the run details first, then dig into the specific activity that failed. Here's the usual step by step you can follow:
- Open the workspace and go to Monitor (or the pipeline item itself), find the failed run and open it.
- Look at the pipeline canvas view for that run, the activity with the red X is the one that failed.
- Click on that activity and check the Output/Error tab, it shows the error code and message.
- If it's a Copy activity, check the Copy activity details for source/sink error, rows read/written, and duration, this narrows down if it's a connectivity, permission or data issue.
- If it's a Notebook or Dataflow Gen2 activity, open the linked notebook/dataflow run directly, the pipeline error message is usually generic and the real stack trace is inside the notebook/dataflow run history.
- Check activity run consumption/duration too, sometimes it's a timeout or capacity throttling (look at Capacity Metrics app if you suspect CU exhaustion).
- If using parameters or expressions, verify the resolved values in the Input tab, a lot of failures come from a bad parameter passed at runtime.
- Retry the single failed activity if the pipeline supports it, instead of rerunning the whole pipeline, to save cost and time while testing your fix.
real example:
Hope this can clarify you 🙂
Best regards,
Carlos
Hi User,
When a Fabric Data Pipeline fails, I usually start by identifying where the failure occurred and then work through the dependencies.
A typical troubleshooting approach is:
Check the Pipeline run history to identify the activity that failed and review the detailed error message.
Open the failed activity logs to determine whether the issue is related to authentication, connectivity, permissions, data quality, or a timeout.
Verify that all linked services, connections, and credentials are valid and haven't expired.
Confirm that the source and destination systems are available and accessible.
If the pipeline runs notebooks or Spark jobs, review the notebook execution logs for detailed exceptions.
Validate input parameters, file paths, and schema changes that may have occurred since the last successful run.
If the issue is intermittent, check the Fabric Capacity Metrics and service health for resource constraints or ongoing service incidents.
After making changes, rerun the failed activity (or use retry if supported) to confirm the issue has been resolved.
For more information:
Monitor pipeline runs: https://learn.microsoft.com/fabric/data-factory/monitor-pipeline-runs
Microsoft Fabric Data Factory documentation: https://learn.microsoft.com/fabric/data-factory/
Microsoft Fabric known issues: https://learn.microsoft.com/fabric/get-started/known-issues/
Could you share the exact error message or activity that failed? That will help narrow down the root cause and provide more targeted guidance.
💡Helpful? Give a Kudos 👍 — keep the community growing.
✅Solved your issue? Mark this as the Accepted Solution ✔️
Best regards,
Prince Singh | Data Science & Microsoft Fabric EnthusiastHi rajan889262
Have you had a chance to look through the responses shared earlier? If anything is still unclear, we’ll be happy to provide additional support.Hi rajan889262
Following up to confirm if the earlier responses addressed your query. If not, please share your questions and we’ll assist further.
