Scheduled Refresh Not Pulling New Data

[jasperj101]

Good Afternoon All, 

I have a scheduled refresh of a 2 daily dataflows followed by a scheduled refresh of a semantic model that pulls data from these flows. 

I noticed that the data in the dataset was 6 days old, even though data coming into the flows was up to today's date. All connections were correctly referenced from data flow to source & semantic model to flow. The refresh history logs for the 12 refreshes that occured over the last 6 days (bi-daily refreshes) had no errors/failures for the flow or the semantic model. 

The proble, fixed itself by triggering a manual refresh in the dataflows followed by datamodel. However, this is quite concerning as the scheduled refreshed did not pick up any new data but did not fail. 

Has anybody else come across something similar? 

Thanks, 

J

[v-fenling-msft]

Thanks for Tutu_in_YYC's concern about this issue.

Hi, @jasperj101 

Do you currently have this problem?
May I ask how you determined that the data was not refreshed?
Have you tried clearing your cache or changing your browser?

Also you can try to recreate a semantic model with the same data stream and configure Scheduled refresh in a different workspace to check if the problem still exists.

I hope my suggestions give you good ideas, if you have any more questions, please clarify in a follow-up reply.
Best Regards,
Fen Ling,
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[jasperj101]

Hi, 

I am still having this issue, in fact it has gotten worse. On Friday I manually refreshed the flow and model which pulled in the recent data up to 08/11. A scheduled refresh then occured this morning (11/11) on the flow and then the model as per the below timings. The data has REVERTED back to the old data from 02/11! 

The model is stored and processed on an F128 capacity premium license. 

The data was found out as outdated by looking in the model to find most recent date in the source table that this dataflow is creating. Other dataflows connected to this model with schedule refreshed are working fine, it is just this one. All of our dataflows connect to Dataflow Gen1 (source data in this location is definitely up to date) 

Thanks, 

J

[jasperj101 · Accepted solution]

Hi All, 

Just to confirm this is now fixed, by changing the timings of the scheduled refresh for both the dataflow and dataset (I shifted them both forward by 30 minutes) this fixed the issue. 

However, I have no idea why this issue occured and the scheduled refreshes were pulling in old data from a fixed point in time without erroring... Any explanatgions would be muich appreciated. 

Thanks, 

J

[Tutu_in_YYC]

Can you let us know the refresh schedule of the dataflow and semantic model? And are you using the option in power bi service or API to refresh them?

[jasperj101]

Hi, 

Dataflows both refresh at 05:30 UTC and take around 5 minutes. 

Model then refreshes at 07:00 and 12:00 UTC, takes around 40 minutes. 

Scheduled refreshes are configured in the PowerBI sevice. 

[Tutu_in_YYC]

I was concerned that there could be overlap between refresh of Dataflow and Semantic model, but you do have enough buffer time between them. Thats good.

If i understand correctly, scheduled refreshes are all succcessful for both dataflow and semantic model. But when you open the report, the data is 6 days old. 

Can you tell me how did you check if its 6 days old? is it based on a date that you calculated using a function in DAX/Power Query or did you look into the date in the data itself?

One thing you can do to confirm if the tables are really refreshed, is by using SSMS to see when table partition was last processed.

These are possible issues:
1. Web browser cache

2. Transformation issues

3. Dataflow reference in semantic model

4. Bug in Power BI service