Forum Discussion

dbeavon3's avatar
dbeavon3
Icon for Memorable Member rankMemorable Member
1 year ago
Solved

Dataflow GEN2 not showing proper duration

There is a GEN2 dataflow that claims to be completing in only about 30 mins:       ... however when I download the logs, it seems clear that it took a lot longer (1:20 hours)      ...
Need Help
  • dbeavon3's avatar
    dbeavon3
    1 year ago

    Hi GilbertQ 

    Thanks for the reply.

    I actually got to the bottom of this with the PG.

    The following image (bracket) is one single refresh operation, but the presentation of it is split in two.  The failure of this refresh operation never actually happened. ... it is only half of an operation, and the "Succeeded" half is the part of the presentation that is the real.  You need to add the durations of both of them together, to get the true duration.


    The UX here is not ideal, to say the least.  The SR is TrackingID#2503060040010671 and hopefully there will be an ICM in about 5 days, and a bug fix in less than 30.

     

GilbertQ's avatar
GilbertQ
Icon for Super User rankSuper User
1 year ago

Hi dbeavon3 

 

What I would recommend doing is going to look at the fabric metrics app to see what the duration was. Because at the end of the day, that is what you are going to be measured against and from your screenshot that looks like a different time point for the refresh.

dbeavon3's avatar
dbeavon3
Icon for Memorable Member rankMemorable Member
1 year ago

IMO, the Fabric metrics app tends to be buggier than other parts of Fabric.  (Eg. I reported a bug where all spark notebooks are listed as having been cancelled in the metrics app, even though the "monitor" blade shows they succeeded.  I worked on that support case for a few weeks before I realized PG wasn't that interested.)

Also the metrics app is pretty far off the beaten path, but this "show history" is just one click away.  Seems like someone would want to hear about the bug, right?