Incremental refresh on Source Table or Result Table

[arridhonabil]

Hi All,

I'm having issues in applying incremental refresh for my data set. 

I have source table imported from data flow, let's say it is table A (disabled load). Then I create manipulation using reference and resulting to result table called table B (enabled load). The question is to benefit the faster refresh, where should I put the incremental refresh between 2 tables? Im afraid if I just put it in table B, the import process still takes all period and just the result gets filtered.

Appreciate your help in this matter.

[lbendlin · Accepted solution]

"Load Disabled"  is just eye candy. The Power Query engine will decide if the lineage requires the table to be processed anyway.

[lbendlin]

Only Table A is a true fact table.  The other tables are not really required - you can use visuals for that.

[arridhonabil]

I see. So should I put the filter by parameter on Table A eventhough it is currently on load disabled? Or will it only works for enabled load table?

[lbendlin]

Needs more context - which of your tables is a fact table with data being added to it frequently/rapidly?

[arridhonabil]

All of the tables are fact tables actually. Let me give example.

This is table A

This is table B which the next process after table A

This is table C which only references previous table B

So all tables actually a sequence, not stand alone.

[lbendlin]

Incremental refresh most of the time only makes sense for fact tables. 

While you can run incremental refresh on both a dataflow and its "child" dataset this seems to be rather excessive.  Dataflows are designed to be ingested very fast, regardless of size. So having incremental refresh on the dataflow should be sufficient. Note:  The setup process is slightly different.  You do not need to create the RangeStart and RangeEnd parameters in a dataflow, the Power BI Service does that for you.  On the dataset you need to create the parameters.

[arridhonabil]

Hi @lbendlin ,

Thanks for the feedback. I understand for dataflow, we dont need to set that up. But for my condition I'd like to put the incremental refresh on Result Table Level or the data set in desktop. Could you share your view for the 3 tables if you've experienced this?

Thanks

[Anonymous]

HI @arridhonabil,

As lbendlin said, these operations do not mean incremental refresh. Please take a look at the following link to know more about incremental refresh.

Incremental refresh for datasets in Power BI - Power BI | Microsoft Docs

If these not suitable for your requirement, you can also consider adding an advanced t-sql query to your connector if it supports custom query.

Power Query SQL Server connector | Microsoft Docs

Regards,

Xiaoxin Sheng

[lbendlin]

That's not how incremental refresh works. In your case it would have to happen at the dataflow refresh level.

How big is your dataflow and how long does it take to ingest?

[arridhonabil]

Hi @lbendlin ,

Thanks for the feedback. I believe data set and data flow have incremental refresh feature as it is under same PQ environment/interface. What I'd like to know since to apply the feature in desktop I need to set up the filter by parameter RangeStart and RangeEnd on the table. The question is if I have those 3 table types mentioned above, can I apply the incremental refresh? If yes, at which tables?