Incremental Refresh does not pull all the data for the defined time duration

[visuresh]

I have defined the parameters right and configured settings correctly. Query folding works. The data refreshes only for the value specified in the RangeStart and RangeEnd parameters and not for the time duration specified in the incremental refresh settings.

[Anonymous]

Hi @visuresh ,

Not very clear about the situation. Let say you have filtered data from July to October and set store last 3 months' data and refresh last one month's data, which part of the data was refreshed in your case?

Best Regards,

Jay

[deanismael]

As mentioned, the start / end date parameters appear to have been configured correctly. Are you observing the same behaviour in Desktop and app.powerbi.com?

Per your original comment...

---

@visuresh wrote:

I have defined the parameters right and configured settings correctly. Query folding works. The data refreshes only for the value specified in the RangeStart and RangeEnd parameters and not for the time duration specified in the incremental refresh settings.

---

Can you confirm if this is the case as earlier in the thread you confirmed the partitions were created and backfilled correctly which indicates it is not an incremental refresh issue.

[deanismael]

You're refreshing in the Service (not Desktop) which is where you're observing these persistent filters? If you're running a Premium capacity, can you validate if the data has been partitioned correctly.

[visuresh · Accepted solution]

I cofirmed that the partitions are correct and data is populated. If I remove the reference query, the incremental refresh works fine. 

[visuresh]

Yes, I am observing this in Service. To verify tha partitions, should I open the model in SSMS?

[deanismael]

That's correct. If you workspace is backed by a Premium capacity, grab the connection string from the dataset settings and connect via an Analysis Services connection in SSMS. The partitions will have been created after the initial refresh and can be viewed by right clicking on the table with the incremental policy applied and selecting the 'Partitions' option.

[visuresh]

I checked and I did not see "applyRefreshPolicy" in the script. But the partitions are created correctly. I am not sure why "applyRefreshPolicy" is not present though Incremental Refresh is defined for the table.

[deanismael]

Is data present in some / all of those partitions?

[visuresh]

Data is present in all the partitions

[deanismael]

Worth double checking if you have any query, page or visual filters enforcing limits on the data, but as I understand it, your original question re. incremental refresh is not the issue here.

[visuresh]

I have setup incremental refresh on one of the Power query. Later I have referenced this query  and created other queries with operations that are not allowed for Query folding. And the data from the later queries is used in the visual. I am expecting that incremental refresh to work on the original first query. Is this approach ok?

[deanismael]

So you've duplicated the query, added steps to break folding and the result is that one visual is filtered based on the rangeStart / rangeEnd parameter settings and the other (which is built from the second query) is complete when viewing via powerbi.com?

[visuresh]

I did not duplicate but referenced only