Forum Discussion

fieldsl's avatar
fieldsl
Frequent Visitor
1 year ago
Solved

ErrorCode: LakehouseOperationFailed

We have a Master Pipeline that runs overnight, and we’re encountering the LakehouseOperationFailed error several times a week. The failure doesn’t consistently occur on the same Copy Data activity, b...
Data Engineering
Data Pipeline
Lakehouse
  • v-saisrao-msft's avatar
    v-saisrao-msft
    1 year ago

    Hi fieldsl,

    The issue may be due to several pipelines writing .parquet files to the Lakehouse at the same time. Consider staging the Copy Data activities with dependencies to prevent parallel writes, and review the Monitoring Hub for more information about pipeline activity and resource usage.

     

    Thank you.

  • BalajiL's avatar
    BalajiL
    1 year ago

    fieldsl  As I said earlier, there are many concurrent threads running in parallel. Try to optimize the concurrency at the pipeline level, that would resolve the issue. 

     

     

v-saisrao-msft's avatar
v-saisrao-msft
Community Support
1 year ago

Hi fieldsl,

The issue may be due to several pipelines writing .parquet files to the Lakehouse at the same time. Consider staging the Copy Data activities with dependencies to prevent parallel writes, and review the Monitoring Hub for more information about pipeline activity and resource usage.

 

Thank you.