Forum Discussion

suriaprakash's avatar
suriaprakash
Frequent Visitor
2 years ago
Solved

Notebook Run Is Failing Continuously

Hi,   The notebook in my pipeline is continuously failing despite multiple retries. Occasionally, it runs fine. This issue started on 07/18 at midnight PT and still persists as of 07/19 at midnight...
Dataflow
Need Help
  • suriaprakash's avatar
    suriaprakash
    2 years ago

    The problem got sorted out automatically. New spark pool with unlimited executors didn't solve the issue. Looks like a Fabric issue which happened coninously for a day or so.

suriaprakash's avatar
suriaprakash
Frequent Visitor
2 years ago

The notebook works fine when ran individually, but fails when executed as an activity in the pipeline. The error is also not self explanatory.

Below is what I have in the first cell to configure the session,

%%configure
{
    "defaultLakehouse": {
        "name": {
            "parameterName": "lakeHouseName",
            "defaultValue": "Overall_Lakehouse"
        },
        "workspaceId": {
            "parameterName": "lakeHouseWSId",
            "defaultValue": "74cbe53b-YYYY-XXXX-yyyy-XXXXXXXXXX"
        }
    }
}