Forum Discussion
Refresh time too long
- 5 months ago
HANA ignores your PQ filters and scans full 20M rows because query folding breaks. Check View Native Query - if grayed out, that's your issue.
Fixes:
-
Push ALL filters to HANA SQL view/calc view at source (not PQ)
-
Enable Incremental Refresh (Premium) on date partition - refreshes ~1/30th data
-
Your AWS idea works: Export filtered 20K to AWS table daily via Dataflow/Logic App, connect PBI to AWS instead
Incremental refresh is fastest if Premium available.
-
- 5 months ago
In Power Query, right-click the step where you apply the filter and check whether View Native Query is available. If it is not, then the filtering is probably not being pushed to SAP HANA.
1) Keep filtering at the source
Apply the filter as early as possible and avoid steps that break folding, such as:-
adding custom columns too early
-
complex merges
-
certain text transformations
-
row-by-row logic
Try to keep the query simple until after the source filter is applied.
2) Create a source-side view
Idea of an intermediate table/view would be a good option.In practice, this is usually done as:
-
a database view in SAP HANA or AWS
-
or a staged table refreshed on a schedule
Then Power BI connects to that smaller prepared object instead of the large transactional table.
This is often the most reliable option for performance.
3) Consider incremental refresh
This only helps if you are loading historical data repeatedly and the model supports a date-based partition strategy. Also keep it mind that incremental refresh works in Power BI Service not in Desktop. -
Hi kkanda
Can you clearify where you apply the SQL Filters? Are you doing any filtering in PowerQuery or only filtering in the advanced options SQL statement of the connector. Normally PBI shouldn't load all 20 million records when you are already filtering in your SQL statement.