concurrency lock condition error when writing multiple records to an audit log table simultaneously.

• Issue: Facing lock condition errors when writing multiple records to an audit log table simultaneously.
• Setup:
  • One master pipeline invoking multiple child pipelines.
  • Audit log notebook placed in both master and child pipelines.
  • During a child pipeline failure, the notebook activity updates the audit log table in parallel for both child and master pipeline rows.
• Problem: Despite updates being made as separate rows, lock conditions occur, causing pipeline errors.
• Attempts to Resolve:
  • Applied partition by.
  • Used update statements.
  • These methods did not resolve the issue in Fabric.

Framed Question:

What are some effective strategies or best practices to avoid lock condition errors when writing multiple records to an audit log table simultaneously in a pipeline setup, especially in a Fabric environment?