Forum Discussion

smpa01's avatar
smpa01
Community Champion
5 years ago

dataflow Incremental Refersh Q

I have a datasource which is as following   | Year | Period | firstDay | Finperiod | tranDateTime | doc# | value | |------|--------|----------|-----------|-------------------|--------|-----...
Need Help
  • smpa01's avatar
    smpa01
    Community Champion
    5 years ago

    Thanks Fowmy for looking into this and helping me decipher inceremental refresh.

    I set up "Refresh rows from the Past 1 year" with the hope

    that

         Incremental will bring all the rows but update only the rows from the past 1 year

    and

         I thought that based on the DATETIME column (in this case tranDateTime)

         Power BI will figure out what the lastest Year is (2022 in this case)

                and

                      Refresh rows from past 1 year (Refresh rows from 1 year past of 2022)

     

    But that is clearly not the case and How should I interpret what Power BI interprets as - Refresh Rows from the past* ____ Years

     

    It can't be current calendar date as my query returns everything from the df upto trandDateTime 12/31/2021 12:01:00 AM and today is 06/30/2021. If Incremental refresh can go past today, why it stops at 12/31/2021 12:01:00 AM?

     

     

    • Fowmy's avatar
      Fowmy
      Super User
      5 years ago

      smpa01 

      I need to do an experiment and get back to you, meanwhile, others will definitely help you out.


      • smpa01's avatar
        smpa01
        Community Champion
        5 years ago

        I have done several experiments with incremental refresh on df today and I can confirm that if my dataset contains rows with future dateTime value [2022-01-01 12:00:00 AM which is > last day of the current year (2021-12-31 11:59:59 PM)], the refresh will not return those rows after incremental refresh when set up with Refresh rows from past 1 year. 

        I dont know whether it is a bug or meant to be like that, would be nice if someone from MS can confirm that.

        I am trying to determine if I should set up incremental refresh on a df which contains transaction dataset that runs on a custom financial calendar where the period runs from 16th-15th (e.g. 2021period1 runs from 12-16-2020 12:00:00 AM to 01-15-2021 11:59;59 PM). I dont want to run into a situation on 16th dec on a given calendar year with this current incremental refresh setting where it does not bring data from the newP1 financial year. If that is going to be the case, I dont want to rely on incremental.