Forum Discussion

remconicolai-'s avatar
remconicolai-
Frequent Visitor
2 years ago
Solved

Increasing CU capacity usage running dataflows

I've scheduled just a few dataflows in Fabric to transfer data from parquet files to a Fabric lakehouse. While running this schedule every hour, the CU background usage exceeds and finaly reaches the...
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi, SaiTejaTalasila 

    thanks for your concern about this issue. 

    Your method is also a solution. 

    And I would like to share some additional solutions below.

     

    Hi,remconicolai- 

    I am glad to help you.  

    According to your description, you want to know what causes this exceeding CU growth for just a few simple dataflows? 

     

    If I understand you correctly, then you can refer to my solution. 

     

    Check the complexity of the data flow: Even if the data flow seems simple, make sure there is no hidden complexity or inefficiency. Some transformation operations may be more resource-intensive than expected, so you can check for transformations and optimize some of them appropriately. 

      

    Throttling Policies: Understand and configure throttling policies to manage and balance performance and reliability. 

      

    Monitor CU Usage: Use the  Microsoft Fabric Capacity Metrics app to monitor CU usage. This helps determine if there are specific times or operations that cause spikes. 

      

    Adjust Scheduling: Instead of running streams every hour, try staggering times or running streams during off-peak hours to distribute the load more evenly. 

      

    Increase Capacity: If your current capacity is consistently reaching its limits, consider increasing Fabric capacity to better handle the load. 

     

    Here's the official documentation, hope it helps: 

    Understand your Fabric capacity throttling - Microsoft Fabric | Microsoft Learn 

     

    I hope my suggestions give you good ideas, if you have any more questions, please clarify in a follow-up reply.


    Best Regards,
    Fen Ling,


    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.