Redshift Dataflow refresh issue - out of memory

That would be more of a question for your Redshift admin. Show them the query and ask them for advice how to refactor it,

Hi gp10 ,

Based on the error message, consider reducing the size or complexity of the dataset, or consider refactoring a large dataset into multiple smaller datasets. If you refresh only the last part, you can use incremental refresh of the dataflow. Only the latest data that has changed needs to be refreshed.
It can also be one of the steps in the data flow that takes up too much memory. Check the recently added steps and try to simplify it.

There is video from Guyinacube on memory usage https://www.youtube.com/watch?v=lopTsclrpHI

You can refer to the following documents that may be helpful to you:

Understanding and optimizing dataflows refresh - Power BI | Microsoft Learn

 refresh failed due to "out of memory for query res... - Microsoft Power BI Community

Best Regards,

Neeko Tang

If this post  helps, then please consider Accept it as the solution  to help the other members find it more quickly.