Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Dataflow is caching data between linked entities and not updating.

Situation:

I have a dataflow set up like this in the service:

tempsnip.png

 

At step 1, I have incremental refresh enabled - pointing at an insert and LAST_UPDATE date/time column.   Enhanced Compute Engine is set to ON.  This is a table with ~2m rows.

 

At step 2, Enhanced Compute Engine is set to "Optimized."  For testing purposes I'm only linking the table from step 1 and not performing any transformations at all on that table.

 

Result:

I am seeing a mismatch in data between linked entities in step 1 and 2:

 

Here is what a particular value looks like in the Extract step, with the filter applied at the very last step (after the incremental refresh step):

tempsnip1.png

 

 

Here is the same table, with a filter applied as the very first step against the linked table:

tempsnip12.png

 

What is going on here?  I've had this issue crop up periodically now and I can resolve it by dropping the cached table in the extract or sometimes by toggling ECU on and off...but nothing seems terribly clear about what's causing the issue

Status: Investigating

Hi @Nerble ,

 

Just to determine if your situation is similar to other users we've encountered recently, have you updated your gateway to the latest version.
If not, does the issue still occur after you update to the latest version?

 

Best regards.
Community Support Team_ Caitlyn

 

Comments
v-xiaoyan-msft
Community Support
Status changed to: Investigating

Hi @Nerble ,

 

Just to determine if your situation is similar to other users we've encountered recently, have you updated your gateway to the latest version.
If not, does the issue still occur after you update to the latest version?

 

Best regards.
Community Support Team_ Caitlyn

 

Nerble
Frequent Visitor

We're running version 3000.154.3.  We typically wait a month or so between updates.  I've run into this issue prior to February though too

rdnguyen
Helper V

I felt relieved when seeing your case is very similar to mine because it would be odd if me alone in such situation 

Dataflow refresh didn't load newest data - Microsoft Power BI Community