Forum Discussion

AnandGNR's avatar
AnandGNR
Icon for Helper I rankHelper I
4 years ago

Incremental Refresh takes almost same time as Full refresh in PBI Service

Hi, I have set up incremental refresh on 2 fact tables that are part of my dataset. I have given an active range of 12 months with an archive of 2 years and use LastModified to detect changes. I kno...
Need Help
Prachi0202's avatar
Prachi0202
Frequent Visitor
2 years ago

lbendlin  Thank you for the quick revert.
Below are the parameters I am using for it,
= Table.SelectRows(#"Changed Type", each [TransactionDate] >= RangeStart and [TransactionDate] <= RangeEnd)

TransactionDate is in DateTime Format and I am using just date part from this column (named as TransactionsDate) to connect to the Calendar dimension table. 
My queries:

1. Does the mentioned part above makes any difference in the data.
2. Detect data change I am seting on the UpdateDate - this the date (in our datawarehouse table) reflecting when the source data is actually updated. As TransactionDate is the combination 2-3 date columns combination from the source. So we are maintaining InsertDate (data loaded date) and UpdateDate (to reflect the updation in source data).
So, still you have the same concern about the data detect change using UpdateDate ?

 


Kindly suggest a solution for Import data.
Thanks and Regards,
Prachi

lbendlin's avatar
lbendlin
Icon for Super User rankSuper User
2 years ago

= Table.SelectRows(#"Changed Type", each [TransactionDate] >= RangeStart and [TransactionDate] < RangeEnd)

  • Prachi0202's avatar
    Prachi0202
    Frequent Visitor
    2 years ago

    Hey thanks, it worked. Can you help me with the Detect Data change option, because the refresh is still taking 40 mins. And I need to use UpdateDate for detecting the data chnges, its the requirement.

    Thanks and Regards
    Prachi

    • lbendlin's avatar
      lbendlin
      Icon for Super User rankSuper User
      2 years ago

      Disable that, it is useless.

      • Prachi0202's avatar
        Prachi0202
        Frequent Visitor
        2 years ago

        But then how can we keep a control of the data which is updated, as the UpdateDate is keeping track of such records at our DataWarehouse end.
        As the transactions which are updated at a later date for whatever reason, UpdateDate is keeping track of such records.

        For eg. My transaction is on date 10th Feb 2024 and this data is loaded in table. Then later this transaction data is updted on 14th Feb 2024, so I need to make the changes in my data in the warehouse. So in such scenario, is UpdateDate will be needed to keep track of updated records. So its needs to be in the Detect Data Change option?