Forum Discussion

kkanda's avatar
kkanda
Resolver II
5 months ago
Solved

Refresh time too long

Hi All, I believe that this subject is not new. I have seen the earlier posts on this topic.  Let me state my problem. I have a data source in SAP HANA which is about 20 million records. I applied ...
  • Kedar_Pande's avatar
    5 months ago

      

    HANA ignores your PQ filters and scans full 20M rows because query folding breaks. Check View Native Query - if grayed out, that's your issue.

    Fixes:

    1. Push ALL filters to HANA SQL view/calc view at source (not PQ)

    2. Enable Incremental Refresh (Premium) on date partition - refreshes ~1/30th data

    3. Your AWS idea works: Export filtered 20K to AWS table daily via Dataflow/Logic App, connect PBI to AWS instead

      Incremental refresh is fastest if Premium available.

    kkanda

  • cengizhanarslan's avatar
    5 months ago

    In Power Query, right-click the step where you apply the filter and check whether View Native Query is available. If it is not, then the filtering is probably not being pushed to SAP HANA.

     

    1) Keep filtering at the source
    Apply the filter as early as possible and avoid steps that break folding, such as:

    • adding custom columns too early

    • complex merges

    • certain text transformations

    • row-by-row logic

    Try to keep the query simple until after the source filter is applied.

     

    2) Create a source-side view
    Idea of an intermediate table/view would be a good option.

    In practice, this is usually done as:

    • a database view in SAP HANA or AWS

    • or a staged table refreshed on a schedule

    Then Power BI connects to that smaller prepared object instead of the large transactional table.

    This is often the most reliable option for performance.

     

    3) Consider incremental refresh
    This only helps if you are loading historical data repeatedly and the model supports a date-based partition strategy. Also keep it mind that incremental refresh works in Power BI Service not in Desktop.