Forum Discussion
Power BI Desktop - rsQueryMemoryLimitExceeded while importing from Amazon Athena (50M rows, 18M rows
- 1 month ago
I’ve seen similar behavior with large Athena imports, and in your case the error looks more like a Power BI limitation than an Athena one.
The rsQueryMemoryLimitExceeded message is typically coming from the Power BI Desktop side (the local Analysis Services / VertiPaq engine used during refresh). Athena may execute the SQL fine, but Power BI still needs to pull, materialize, and compress the dataset into memory, and that’s where it’s hitting the ~1 GB per-query limit.
Regarding increasing the memory limit in Desktop — as far as I know, there’s no straightforward supported setting to raise that specific per-query cap. More RAM on the machine can help overall performance, but it usually doesn’t bypass this limit.
For incremental refresh, since your source only has a Unix timestamp, I’d recommend filtering directly on the raw BIGINT column instead of doing from_unixtime(eventtime/1000) in the WHERE clause. Wrapping the column in a function often hurts pushdown/query performance. Converting RangeStart / RangeEnd into Unix epoch values and comparing directly against eventtime is usually much more efficient.
About Value.NativeQuery(): it’s useful when you need custom SQL (especially with Athena), but the downside is that it can interfere with query folding, which matters for incremental refresh. If incremental refresh is the goal, I’d test carefully whether folding is still happening. If not, that may become a bottleneck.
Also, even after reducing to 4 columns, the row count is still huge. If you’re pulling ~2 days of data and ingesting ~18M rows/day, that’s potentially ~36M rows in one refresh. Even with fewer columns, fields like conversationid or attributevalue can have high cardinality and compress poorly.
If I were designing this pipeline, I’d look at:
Partitioning the Athena table by date (if not already)
Using incremental refresh
Pre-aggregating or reducing data in Athena (views/CTAS)
Avoiding large full imports into Desktop whenever possible
At 50M+ rows with 18M new rows daily, I’d strongly lean toward a partitioned + incremental architecture rather than relying on full import refreshes in Desktop.
Helpful? Give a Kudos 👍Solved? Mark as Solution ✔️— Azad Singh Thakur | Power BI Developer
Hi manoj_0911,
Thank you for reaching out to Microsoft Fabric Community.
Thank you Azadsingh for the prompt response.
As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided by the user for the issue worked? or let us know if you need any further assistance.
Thanks and regards,
Anjan Kumar Chippa