Forum Discussion
Problems with importing data from AWS Redshift
Hi everyone,
I have a Power BI Report called stored in a Power BI Workspace within my work organsiation. It is set to refresh weekly on Monday mornings.
The refresh fails and the reason for the failure given is that the report size is too large and takes up too much memory:
Data source error: Resource Governance: This operation was canceled because there wasn't enough memory to finish running it. Either reduce the memory footprint of your dataset by doing things such as limiting the amount of imported data, or if using Power BI Premium, increase the memory of the Premium capacity where this dataset is hosted. More details: consumed memory 25518 MB, memory limit 25514 MB, database size before command execution 85 MB. See Troubleshoot XMLA endpoint connectivity in Power BI - Microsoft Fabric to learn more.
However a copy of this report does refresh from another workspace at the same time. Both workspaces have the same license configuration and capacity.
Further, there is a near identical report that brings in a larger dataset that continues to refresh without any issues. This report sources its data from a data warehouse internal to the organisation.
Also there is another report in the same workspace, which is a larger size and that brings in a larger dataset without any refresh issues from AWS Redshift.
For reference the report that fails is listed as (495mb) in size although the actual file size is 265,100 kb.
The 2nd report in the same workspace that gets data internally is (748mb)
The 3rd report in the same workspace that gets data from Redshift without issue is (501mb)
12 Replies
Unfortunately changing the workspace setting to a large semantic model, didn't fix the issue, and the report continues to fail when trying to refresh in service with the error message siting not enough memory.
I will try another round of to reducing the report in size.Hello Kevin8,
Thank you for the update.
Reducing the report size is a good approach, and hopefully that helps resolve your issue.If you continue to experience the issue after making those changes, please let us know, we’ll be happy to assist further. Your update will be valuable to the community and may assist others with similar concerns.
Thank you for being part of the Microsoft Fabric Community.
Hi Tayloramy,
The capacity SKU is F64. The workspace also says:Semantic model storage format Small semantic model storage format.
Do you have Calculated Columns or Power Query transformation steps that disables native query option? If yes, even the refreshed datasets size is lower then the max limit, the size of the model coult exceed the max limit during the refresh process. Consider making all those transformations on the source level which is AWS in your case.
Hi Cengizhanarslan,
I have shrunk the report as much as possible, to fold the data to the source so that the data transformations are done in the sql query and the power query steps are reduced as much as possible.
However, before I did this the error message was the same, the report was a fraction too big, and needed to be made slightly smaller to work.
After massively reducing the report size, the same error message occured. This leads me to conclude it is not a Power BI memory issue. The same error message is given regardless of the report size.
Any ideas?
Hi Kevin8,
It's not the filesize of the pbix file that is the problem here, it's how much the data expands during refresh.
For an F64 capacity, the max memory available is 25GB, which is the limit you're hitting.
Can you describe more about what data sources you're using and what sort of transformations are being done?
Hi Tayloramy,
Its a Fabric64 workspace. Are there any other specs that you need?Hi Kevin8,
Thank you for posting your query in the Microsoft Fabric Community Forum, and thanks to tayloramy & cengizhanarslan for sharing valuable insights.
Could you please confirm if your query has been resolved by the provided solutions? This would be helpful for other members who may encounter similar issues.
Thank you for being part of the Microsoft Fabric Community.
Hello Kevin8,
We hope you're doing well. Could you please confirm whether your issue has been resolved or if you're still facing challenges? Your update will be valuable to the community and may assist others with similar concerns.
Thank you.
