Power BI Desktop - rsQueryMemoryLimitExceeded while importing from Amazon Athena (50M rows, 18M rows

[manoj_0911]

Power BI Desktop - rsQueryMemoryLimitExceeded while importing from Amazon Athena (50M rows, 18M rows/day)

Question

Hi everyone,

I'm importing data from an Amazon Athena table into Power BI Desktop (Import mode).

Environment

• Source: Amazon Athena

• Connection: Amazon Athena connector using Value.NativeQuery()

• Total rows: ~50 million

• New rows per day: ~18 million

• eventtime is stored as a Unix timestamp (Whole Number/BIGINT)

• No native DateTime column exists.

Initially I was using:

SELECT *
FROM table
WHERE from_unixtime(eventtime/1000) >= current_timestamp - interval '2' day

To reduce memory usage, I changed it to:

SELECT
    conversationid,
    eventtime,
    attributename,
    attributevalue
FROM table
WHERE from_unixtime(eventtime/1000) >= current_timestamp - interval '2' day

Even after selecting only 4 columns, Power BI Desktop still fails with:

Underlying Error:
rsQueryMemoryLimitExceeded

Consumed memory: 1055 MB
Memory limit: 1024 MB

My questions are:

1. Is this limitation coming from Power BI Desktop/Analysis Services or from the Athena connector?

2. Is there any way to increase the per-query memory limit in Power BI Desktop?

3. Since the source only has a Unix timestamp (BIGINT), what is the recommended approach for implementing Incremental Refresh?

4. Is using Value.NativeQuery() with Athena recommended for large datasets, or should I avoid native queries to preserve query folding?

5. Has anyone successfully imported similar data volumes from Athena into Power BI?

Any recommendations or best practices would be appreciated.

Thank you!

[v-achippa]

Hi @manoj_0911,

We wanted to kindly follow up to check if the solution provided by the user for the issue worked? or let us know if you need any further assistance.

Thanks and regards,

Anjan Kumar Chippa

[krishnakanth240]

Hi @manoj_0911 

1. Source of the limitation

1 GB per query memory limit comes from Power BI Desktop’s internal Analysis Services engine and not Athena. It’s a fixed cap where you cannot increase it in Desktop

2. Increasing the limit

There is no way to raise 1024 MB query memory limit in Power BI Desktop. Workaround is to reduce data volume per query or move to Power BI Premium capacity which has higher limits

3. Incremental Refresh with Unix timestamp

Since source only has a BIGINT Unix timestamp, you should convert it to a DateTime column in Athena and expose that column. Using DateTime column as partition column for Incremental Refresh

4. Value.NativeQuery vs query folding

Value.NativeQuery is fine for Athena if you need precise SQL control. However, it breaks query folding in many cases. For Incremental Refresh, folding is critical and avoid NativeQuery. Prefer the standard Athena connector with filters

5. Large dataset imports from Athena

Importing 50M+ rows directly into Desktop is not practical. You can use DirectQuery for very large datasets or pre aggregate data in Athena before loading. Implement Incremental Refresh with a proper DateTime partition.Validate in Desktop with smaller samples then publish to Service with Premium capacity for full refresh

[v-achippa]

Hi @manoj_0911,

Thank you for reaching out to Microsoft Fabric Community.

Thank you @Azadsingh for the prompt response.

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided by the user for the issue worked? or let us know if you need any further assistance.

Thanks and regards,

Anjan Kumar Chippa

[Azadsingh · Accepted solution]

I’ve seen similar behavior with large Athena imports, and in your case the error looks more like a Power BI limitation than an Athena one.

The rsQueryMemoryLimitExceeded message is typically coming from the Power BI Desktop side (the local Analysis Services / VertiPaq engine used during refresh). Athena may execute the SQL fine, but Power BI still needs to pull, materialize, and compress the dataset into memory, and that’s where it’s hitting the ~1 GB per-query limit.

Regarding increasing the memory limit in Desktop — as far as I know, there’s no straightforward supported setting to raise that specific per-query cap. More RAM on the machine can help overall performance, but it usually doesn’t bypass this limit.

For incremental refresh, since your source only has a Unix timestamp, I’d recommend filtering directly on the raw BIGINT column instead of doing from_unixtime(eventtime/1000) in the WHERE clause. Wrapping the column in a function often hurts pushdown/query performance. Converting RangeStart / RangeEnd into Unix epoch values and comparing directly against eventtime is usually much more efficient.

About Value.NativeQuery(): it’s useful when you need custom SQL (especially with Athena), but the downside is that it can interfere with query folding, which matters for incremental refresh. If incremental refresh is the goal, I’d test carefully whether folding is still happening. If not, that may become a bottleneck.

Also, even after reducing to 4 columns, the row count is still huge. If you’re pulling ~2 days of data and ingesting ~18M rows/day, that’s potentially ~36M rows in one refresh. Even with fewer columns, fields like conversationid or attributevalue can have high cardinality and compress poorly.

If I were designing this pipeline, I’d look at:

• Partitioning the Athena table by date (if not already)

• Using incremental refresh

• Pre-aggregating or reducing data in Athena (views/CTAS)

• Avoiding large full imports into Desktop whenever possible

At 50M+ rows with 18M new rows daily, I’d strongly lean toward a partitioned + incremental architecture rather than relying on full import refreshes in Desktop.

Helpful? Give a Kudos 👍

Solved? Mark as Solution ✔️

— Azad Singh Thakur | Power BI Developer

https://www.linkedin.com/in/thakurazad/