dataflow refresh failed

[ManasiL]

Hi,

I have one dataflow that has 15-17 queries inside that load data in respective tables in data warehouse. When the query count was less the dataflow refresh works fine. When i add more queries then it started failing. We are on F2 compute capacity. 

So, is it because the compute is less so it is failing? Or is there any limit on the number of queries we can have inside a dataflow.

[Anonymous]

Hi @ManasiL 

Can you tell me if your problem is solved? If yes, please accept it as solution.

Regards,

Nono Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[Anonymous · Accepted solution]

Hi @ManasiL 

Thank you very much charlyS for your prompt reply.

Since the errors occur randomly for different tables, it might indicate that specific queries are more resource-intensive or that there are intermittent issues with the data source or network.

And according to the error message, the system reached the maximum number of retries when trying to create the table, but no progress was made. This may be due to insufficient resources or excessive system load.

This can happen if your F2 computing power is unable to meet the current request.

You can monitor the capacity to see if it is the cause of insufficient capacity. In Fabric, monitoring capacity helps you see current resource usage, including compute power and storage capacity.

Announcing Microsoft Fabric capacities are available for purchase | Microsoft Fabric Blog | Microsof...

What is the Microsoft Fabric Capacity Metrics app? - Microsoft Fabric | Microsoft Learn

Evaluate and optimize your Microsoft Fabric capacity - Microsoft Fabric | Microsoft Learn

Regards,

Nono Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[ManasiL]

Hello @charlyS ,

Below is the error screenshot. Initially this dataflow had 10 queries inside when it was running fine. Later I added 5 more queries so total 15 queries. Also during every dataflow refresh it will fail with the same error for 2 random tables. So i deleted 2 queries and so total 13 queries in the dataflow and it was successful.

Also the tables are just descriptive tables. Not so heavy tables.

[charlyS]

Hello, 

Can you share the error message by any chance ? 

If you want to know if your dataflow is failing due to capacity limits you can use the Fabric Capacity Metric App (https://learn.microsoft.com/en-us/fabric/enterprise/metrics-app) and check on the throthling pane if you have background rejection (sample here https://learn.microsoft.com/en-us/fabric/enterprise/media/fabric-drill-through.gif#lightbox).