Dataflow Gen2 : System Out of Memory

[Morse-AMHG]

Hello, Since yesterday we've been seeing a common issue with muliplte Gen2 Dataflows failing kicking out the failure message: Error: Exception of type 'System.OutOfMemoryException' was thrown. 

We are using an F64 Capcity and I've checked the known issues log and couldn't see anything that related. 

[Morse-AMHG]

Thanks - We spoke with our IT Team turned out the Server was lagging and a Server restart solved this issue, but much opportunity to work to some of these guidelines. Thank you. 

[Morse-AMHG]

Hi, Its happening on the most simple ones also like brining in a small amount of data 5k rows from Excel stored in a Sharepoint. 

[BhavinVyas3003]

Try to convert the Excel to CSV, store it in OneLake or Azure Blob Storage, and update your Dataflow Gen2 to use the CSV file.

[BhavinVyas3003 · Accepted solution]

This error means your dataflow might be using more memory than available.

• Optimize queries: Remove unnecessary columns/rows early.

• Split large dataflows: Break into smaller linked dataflows.
• Use staging: Load raw data in one dataflow, transform in another.
• Check capacity load: Use Fabric Capacity Metrics app to monitor usage.