Forum Discussion
Dataflow Gen2 : System Out of Memory
Hello, Since yesterday we've been seeing a common issue with muliplte Gen2 Dataflows failing kicking out the failure message: Error: Exception of type 'System.OutOfMemoryException' was thrown.
We are using an F64 Capcity and I've checked the known issues log and couldn't see anything that related.
This error means your dataflow might be using more memory than available.
- Optimize queries: Remove unnecessary columns/rows early.
- Split large dataflows: Break into smaller linked dataflows.
- Use staging: Load raw data in one dataflow, transform in another.
- Check capacity load: Use Fabric Capacity Metrics app to monitor usage.
4 Replies
Thanks - We spoke with our IT Team turned out the Server was lagging and a Server restart solved this issue, but much opportunity to work to some of these guidelines. Thank you.
This error means your dataflow might be using more memory than available.
- Optimize queries: Remove unnecessary columns/rows early.
- Split large dataflows: Break into smaller linked dataflows.
- Use staging: Load raw data in one dataflow, transform in another.
- Check capacity load: Use Fabric Capacity Metrics app to monitor usage.
Hi, Its happening on the most simple ones also like brining in a small amount of data 5k rows from Excel stored in a Sharepoint.
Try to convert the Excel to CSV, store it in OneLake or Azure Blob Storage, and update your Dataflow Gen2 to use the CSV file.
