Hi All, I'm having issues in applying incremental refresh for my data set. I have source table imported from data flow, let's say it is table A (disabled load). Then I create manipulation using reference and resulting to result table called table B (enabled load). The question is to benefit the faster refresh, where should I put the incremental refresh between 2 tables? Im afraid if I just put it in table B, the import process still takes all period and just the result gets filtered. Appreciate your help in this matter.

"Load Disabled" is just eye candy. The Power Query engine will decide if the lineage requires the table to be processed anyway.

Incremental refresh on Source Table or Result Table

10 Replies

lbendlin
Super User
5 years ago
That's not how incremental refresh works. In your case it would have to happen at the dataflow refresh level.

How big is your dataflow and how long does it take to ingest?
- arridhonabil
  Frequent Visitor
  5 years ago
  Hi lbendlin ,
  
  Thanks for the feedback. I believe data set and data flow have incremental refresh feature as it is under same PQ environment/interface. What I'd like to know since to apply the feature in desktop I need to set up the filter by parameter RangeStart and RangeEnd on the table. The question is if I have those 3 table types mentioned above, can I apply the incremental refresh? If yes, at which tables?
Anonymous
Not applicable
5 years ago
HI arridhonabil,

As lbendlin said, these operations do not mean incremental refresh. Please take a look at the following link to know more about incremental refresh.

Incremental refresh for datasets in Power BI - Power BI | Microsoft Docs

If these not suitable for your requirement, you can also consider adding an advanced t-sql query to your connector if it supports custom query.

Power Query SQL Server connector | Microsoft Docs

Regards,

Xiaoxin Sheng
lbendlin
Super User
5 years ago
Incremental refresh most of the time only makes sense for fact tables.

While you can run incremental refresh on both a dataflow and its "child" dataset this seems to be rather excessive. Dataflows are designed to be ingested very fast, regardless of size. So having incremental refresh on the dataflow should be sufficient. Note: The setup process is slightly different. You do not need to create the RangeStart and RangeEnd parameters in a dataflow, the Power BI Service does that for you. On the dataset you need to create the parameters.
- arridhonabil
  Frequent Visitor
  5 years ago
  Hi lbendlin ,
  
  Thanks for the feedback. I understand for dataflow, we dont need to set that up. But for my condition I'd like to put the incremental refresh on Result Table Level or the data set in desktop. Could you share your view for the 3 tables if you've experienced this?
  
  Thanks
lbendlin
Super User
5 years ago
Needs more context - which of your tables is a fact table with data being added to it frequently/rapidly?
- arridhonabil
  Frequent Visitor
  5 years ago
  All of the tables are fact tables actually. Let me give example.
  
  This is table A
  
  This is table B which the next process after table A
  
  This is table C which only references previous table B
  
  So all tables actually a sequence, not stand alone.
lbendlin
Super User
5 years ago
Only Table A is a true fact table. The other tables are not really required - you can use visuals for that.
- arridhonabil
  Frequent Visitor
  5 years ago
  I see. So should I put the filter by parameter on Table A eventhough it is currently on load disabled? Or will it only works for enabled load table?
lbendlin
Super User
5 years ago
"Load Disabled" is just eye candy. The Power Query engine will decide if the lineage requires the table to be processed anyway.

Forum Discussion

Incremental refresh on Source Table or Result Table

10 Replies

Related content

Incremental Refresh on Dataflow sourced from SharePoint

Increment Refresh

Incremental refresh

Incremental Refresh

Incremental refresh