Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
Joshrodgers123
Advocate IV
Advocate IV

Dataflow Activity in Pipeline

I've been working on a pipeline that needs to run dataflows. I am noticing that the duration of the dataflow activity often far exceeds that actual runtime of the dataflow. For example, the below dataflow activity shows it took 4m 3s. When I look at the actual refresh history of the dataflow it shows it took 34s. 

 

This adds a huge amount of time to the pipeline where it is essentially doing nothing. Anyone else notice this?

 

Joshrodgers123_0-1701795833002.png

 

Joshrodgers123_1-1701795874703.png

 

5 REPLIES 5
Joshrodgers123
Advocate IV
Advocate IV

Hello @v-gchenna-msft 

 

Yes, I am still seeing this issue. 

 

As another test, I created a dataflow that loaded 1 single record, with 1 column using the enter data feature. So there was no data source and no data destination.

Joshrodgers123_1-1702060497018.png

 

I ran this in a pipeline 5 times consecutively. You can see the timings of the dataflow and the corresponding pipeline activity below. Overall, it took almost 20 minutes to run.. 

 

Joshrodgers123_0-1702060382373.png

 

I can send the dataflow, pipelines, and activity ids if needed, but something is definetly wrong as this should have compeleted in seconds.

Hi @Joshrodgers123 ,

Apologies for the issue that you are facing.

This might require a deeper investigation from our engineering team about your workspace and the logic behind it to properly understand what might be happening. 

Please go ahead and raise a support ticket to reach our support team:

https://support.fabric.microsoft.com/support
Please provide the ticket number here as we can keep an eye on it.

Hello @Joshrodgers123 ,

We haven’t heard from you on the last response and was just checking back to see if you got a chance to create a support ticket. If yes, please provide the ticket number here as we can keep an eye on it.

v-gchenna-msft
Community Support
Community Support

Hi @Joshrodgers123 ,

Thanks for using Fabric Community.

Apologies for the issue you are facing. I cannot find any huge difference when I tried to reproduce the scenario.

Data Pipeline:

vgchennamsft_0-1701851381055.png

Data Flow:

vgchennamsft_1-1701851437625.png


It might be an intermittent issue. Do let us know if you still observe execution time difference.



Hello @Joshrodgers123 ,

We haven’t heard from you on the last response and was just checking back to see if your query was answered. Otherwise, will respond back with the more details and we will try to help .

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

March 2024 FBC Gallery Image

Fabric Monthly Update - March 2024

Check out the March 2024 Fabric update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.