Forum Discussion
Container exited with a non-zero exit code 137
Were you able to get this issue fixed? I am experiencing the exact same issue. A lot of executors failing with error code 137. I am migrating jobs currently running fine (daily, never once crashing) from Azure Synapse into Fabric. I am using identical pool sizes, but even so - the fabric jobs are crashing left and right.
Even when doubling the spark pool size (going from 3x small nodes to 3x medium nodes) I am seeing similar executor failures. Sometimes the jobs manage to finish, sometimes they pull the livy session down with them and the entire application fails.
Monitoring the spark application memory usage while executing, the executors are only satuated to around 50% memory usage when they die. They do however almost always die when fully utilized on CPU...
I have tried everything; disabled persisting of dataframes, increased overhead memory on executors, ... But no change; Fabric just cant keep my simple jobs alive. And they are simple; reading from delta, saving to delta, working with 100MB-4GB delta tables. This can be run on a potato, but apparently not in Fabric..
Note: I am not using Native Execution engine, because that **bleep** brings it own case of issues to the party. Gluten exploding in my face at every turn.. So this should be as close to 1:1 to Azure Synapse as it gets, I would think..