Forum Discussion

DamianMcc's avatar
DamianMcc
Frequent Visitor
1 year ago
Solved

Help me understand Fabric Capacity

I don't understand how the Fabric capacity works. Specifically around when capacity utilisation 'resets'. 

 

I have set a daily pipeline where data is ingested from a remote API into a data lake using a data pipeline. This data is then processed into the lake house using data flow gen 2. It takes about 10 minutes of processing for the entire run, on a very small dataset with 36 tables generated in the lake house. 

A single Power BI report then refreshes its data after this. 

 

My Fabric SKU is F4. 

 

Looking at the Fabric Capacity Metrics, I'm consuming a lot of resources using data flow gen 2. I'm mainly expanding JSON arrays and hiding columns. 

 

By my calculations, F4 = 4 CU(s)

That would give me 345,600 CUs per 24-hour period. 

My data flow Gen 2 utilises approximately 138k - 193k CU(s) daily. 

 

Why am I running out of capacity?

 

 

 

 

Just over 2 minutes on the data pipeline

 

 

7-minute run on the data flow gen 2

 

 

capacities
  • v-aatheeque's avatar
    v-aatheeque
    1 year ago

    Hi DamianMcc ,
    nilendraFabric  Thanks for your prompt response.

    Thanks for reaching out to the Microsoft Fabric Community forum.

    Thank you for your detailed explanation of the situation. we understand that you're seeing a consumption of 193K CUs for a specific group of transactions and are expecting to have 345K CUs available for the next day.

    However, your capacity might not reset all at once every 24 hours. Instead, it could be refilled gradually over time. For example, if your capacity is refilled gradually over time, you won't see an immediate full reset at the end of 24 hours.

    Additionally, if there are other background processes or transactions that consume capacity, even if they are not visible to you, they could be affecting your available CUs.

    To better estimate and manage CU usage, consider the following:

    • Keep track of your CU usage every hour instead of just looking at the daily total.
    • Watch out for sudden spikes—if any transaction uses more than 14,400 CUs in an hour, it might be using capacity from future hours.
    • Adjust your CU usage by either increasing your capacity or spreading out transactions more evenly throughout the day.

    Additonally you can refer the below Docs : 

    Understand the metrics app compute page - Microsoft Fabric | Microsoft Learn
    Metrics app calculations - Microsoft Fabric | Microsoft Learn

     

    If our response or a community member’s response addressed your query, please mark it as 'Accept Answer' and click 'Yes' if you found it helpful.

    If you have any further questions, feel free to reach out.

9 Replies

  • nilendraFabric's avatar
    nilendraFabric
    Icon for Super User rankSuper User
    1 year ago

    Hello DamianMcc 


    Each Fabric SKU, such as F4, provides a fixed number of compute resources. For F4, you have 4 CUs available at any given time to handle workloads like dataflows, pipelines, and Power BI report refreshes.

     

    Fabric uses a smoothing mechanism to distribute CU usage over time. For interactive operations (e.g., user-triggered queries), smoothing occurs over 5-minute windows. For background jobs (e.g., scheduled pipelines or dataflows), smoothing spans 24 hours

     

    If your workload temporarily exceeds the allocated CUs, Fabric borrows resources from future time periods (up to 24 hours). This allows workloads to complete faster but reduces available capacity later in the day

     

    Your daily workload includes:

    • A pipeline that consumes approximately 0.306 CUs.
    • A Dataflow Gen2 operation that consumes approximately 2.67 CUs.
    • A Power BI refresh (assuming 0.5 CUS).
    • Total direct CU usage from your pipeline and Dataflow Gen2: ~3.5 CUs.
    • with other BC activites , it will be around 4 CU

    If there’s any additional spike or inefficiency (e.g., query folding issues in Dataflow Gen2), you could temporarily exceed 4 CUs, triggering borrowing and showing as 100% utilization for the day

     

    Hope this helps

    Please accept the answer if this is helpful

     

     

     

    • DamianMcc's avatar
      DamianMcc
      Frequent Visitor
      1 year ago

      Thanks. I understand that a little better. 

       

      I'm still troubled by what I think should be available to me as capacity per day, what Fabric tells me I'm consuming and why, given I should be in an 'in credit' position, I'm running out of capacity. 

       

      Per the table below. 4 CU(s) is 345k of CU(s) per day. 

      I'm informed by Fabric I'm consuming 193K of CU(s)

       

      It looks to be like I am not consuming my daily 'limit'? But yet I am. 

       

       

       

       

      • nilendraFabric's avatar
        nilendraFabric
        Icon for Super User rankSuper User
        1 year ago

        hello DamianMcc 

        Let me clarify why it seems like you aren’t consuming your daily "limit" of 345,600 CUs (for an F4 SKU) but are still running out of capacity.

        •  consumption of 193K CUs reflects the total smoothed usage of your workloads over the day. This number doesn’t necessarily mean you’ve hit your theoretical daily maximum of 345,600 CUs.
        • However, if your workloads frequently exceed the instantaneous limit of 4 CUs/second, Fabric starts borrowing from future capacity. This borrowing can lead to running out of capacity even before reaching the daily limit.

          The theoretical daily limit assumes that usage is evenly distributed across all seconds in a day. In practice:
          • If your workloads create spikes (e.g., using more than 4 CUs/second during peak times), you’ll quickly deplete available capacity.
          • Borrowing from future intervals leaves less capacity for subsequent operations, even if your total usage is below the daily allowance.