Forum Discussion

JasonBurdetts's avatar
JasonBurdetts
Frequent Visitor
6 months ago
Solved

Bulk Import and Incremental Refresh

Hi everyone,   I have a table with the results coming from a web service API call. I am looking to set this table up using increment refresh as I am able to send the RangeStart and RangeEnd as para...
Need Help
  • Natarajan_M's avatar
    Natarajan_M
    6 months ago

    Hi JasonBurdetts  , 

    Based on the inputs, this is how this it will look  


    Partitions :

    From August to January, data will be organized by month. In February, it will be organized by day. Once we reach March 8, the February daily data will be consolidated into monthly data, and March will be arranged by day.



    Documentation : https://learn.microsoft.com/en-us/power-bi/connect-data/incremental-refresh-overview

    Partition                                Date Range

    2025-08                                01-Aug-2025 → 31-Aug-2025
    2025-09                                01-Sep-2025 → 30-Sep-2025
    2025-10                                01-Oct-2025 → 31-Oct-2025
    2025-11                                01-Nov-2025 → 30-Nov-2025
    2025-12                                01-Dec-2025 → 31-Dec-2025
    2026-01                                01-Jan-2026 → 31-Jan-2026
    2026-02-01 (Day wise)         01-Feb-2026
    ---- 02 Feb to 23 Feb  ---      Day wise partitions
    2026-02-25 (Day wise)         25-Feb-2026




    Thanks 

    If this response was helpful in any way, I’d gladly accept a kudo.
    Please mark it as the correct solution. It helps other community members find their way faster

Natarajan_M's avatar
Natarajan_M
Super User
6 months ago

Hi JasonBurdetts , instead of loading the historical data as a single partition (Say for the entire year) you can break it by month by adjusting the incremental refresh policies and load the data by months

Thanks 

JasonBurdetts's avatar
JasonBurdetts
Frequent Visitor
6 months ago

Thank you again for the response. I am new to the incremental refreshes, so how they work is not entirely clear. Let's say I set the archive data to 6 months, and then I set the refresh to be 7 days. Will the initial data loading:

  • Start at 6 months ago and ask the API for data between 6 months and 6 months + 7 days
  • Then do the next 7 days with another API call
  • Keep doing 7 day increments until it has "caught up"?

If this is the case, will it keep retrying a period if the API becomes rate limited?

Jason