Incremental Refresh is not working in Production workspace

[Anonymous]

I have a report with incremental refresh and published it to test and then deployed in Prod workspace using deployment pipeline. I have scheduled the reports weekly once in test and prod. Full load is completed in both the environments. Scheduled incremental refresh also done in both the environments. But I can see the latest data in test and could see the number of records fetched usig SSMS. But in Prod even though refresh history shows it as completed, it didn't load any of the records. May I know the reason behind this?

I did manual refresh for last 1 week using SSMS then it worked. But why the scheduled one is not working in Prod.

@Anonymous  

 

[pallavi_r]

Hi @Anonymous ,

As @lbendlin has mentioned, your partitions are very small. Set it to Monthly.

Couple of other checks:

1. Is query folding happening? you can check this by right clicking on the dataset. view native query will be enabled.

2.The data parameter for range start and range end should not be same as detect data changes date.

3.If all these are correct, then do a full refresh with less data. You can utilize view with date range for the same 

4.Once full refresh is done, do partition wise refresh using process data.

5.Precess Recalc 

6.When you achieve success till step 5, these are step by step process. then try incremental refresh it should work.

Thanks,

Pallavi

[Anonymous]

Hi Pallavi,

Thanks for your reply.

1. I am using Azure Databricks, so query folding is not applied.

2. Range start I gave Nov1st 2024, and range end I gave as Dec1st 2024. So we have enough gap.
3. I publsihed my report after setting incremental refresh. I gave 7days for refresh period.
4. Then I did full refresh manually in the prod after deploying it from test using deployment pipeline.
5. Scheduled refresh is happend on 17th Dec, it got completed without any error but didn't pickup records.
then I tried refreshing the last 7days data using SSMS. Issue got fixed here. but we don't know the root cause and not sure if it pick up the records in next scheduled refresh or not. So my manager is expecting a root cause for this. So I have raised a case with Microsoft. Case 2412181420002119

[lbendlin]

When you set the refresh policy to "automatic"  then Power BI Service will decide by itself if it wants to refresh a particular partition or not. If there was no data change (or the service didn't catch it) then "automatic"  may not do anything.  Switch to "full"  to force a refresh.

[Anonymous]

It has the changes, refresh also done but couldn't pick the records in Prod. The same happend in test and able to pickup the records. 

On demand : this is full load refresh

Scheduled : Incremental refresh
 You can see the refresh as completed without any error. Then I did refresh from SSMS as it didn't pick any records. 

[lbendlin]

Can you confirm in SSMS that you actually see more than one partition?

[Anonymous]

When I verified the records after schedule refresh I could see zero records for 2024Q41208. Then after refresh from SSMS I could see the number of records  got updated here post completion of refresh

[lbendlin]

Your partitions are rather small. I would go with monthly base partitions rather than daily.

If you have a Pro license you can open a Pro ticket at https://admin.powerplatform.microsoft.com/newsupportticket/powerbi
Otherwise you can raise an issue at https://community.fabric.microsoft.com/t5/Issues/idb-p/Issues .

[Anonymous]

We are using Premium licence. where can I set partitions? My backend tables would get loaded based on weekly basic, so I scheduled the refresh weekly.

[lbendlin]

when you set up incremental refresh you can decide which partitions to use. Something like 

"keep last 5 years and refresh last 1 months"

[Anonymous]

There I have given archive period as 2 years and refresh for last 7 days.