Forum Discussion
Resolver IWhy is there CU usage while no activity running?
I'm checking the capacity report to try to understand the usage.
but I found that I have a constant 12CU(s) consumed all the time.
in the middle of the night, when I have nothing running, CU(s) are consumed and the capacity report is not able to identify what is consuming this.
I'm using F2 for now and 12CU(s) = 21%!!!
today there was near no activity (3 at 10AM (for a total of 4min) and 1 at 3PM (for 5 seconds)) and the throttling counter is increasing the day long for no reason.
this causes capacity errors everywhere.
so did the CU(s) calculation can be trusted?
How can we validate what Microsoft is billing us?
because for now I think it's completely broken.
can we have a real time counter of the current CU used and which resource use it?
20 Replies
Hi Jerome22,
I'm having the same issue. I'm trialling Fabric (on an F8) and the week before last I saw that I seemed to be using a lot of CUs for background tasks when I had nothing running. Last week I decided not to use Fabric at all to eliminate completely anything that I was doing, I had no scheduled jobs running and no queries being made to my Lakehouse’s, Warehouses or Dataflows, no refreshes of Dataflows or data models and I didn’t view any reports that would be using my Fabric data either. The only thing I had was 5 mirrored dbs (all MS documentation that I've read states that these should not use CUs to keep in sync, should only be using CUs when you query them, unless I’ve misinterpreted anything) and I have shortcuts setup in my Lakehouse’s (again these should not be using CUs as no data is being copied, they are shortcuts). As I said, none of my dataflows or pipelines that take the data from the Lakehouse, carry out transformationss and land the data in my Warehouse are running or scheduled to run, yet all last week every day I seemed to be using between 15% and 20% of my CUs on background tasks. Last Wednesday I had supposedly used 96,441 CUs, on Thursday 140,000 and on Friday 110,000 all when I'm doing nothing and have nothing running.
I'm really liking Fabric and all the new capabilities that are available but I'm really not liking that 15 to 20% of my daily CUs on an F8 are being used when I'm not doing anything. I'm already quite concerned that when I have been running jobs that they are using a lot of CUs and this is only a small amount of what I would like to have running on Fabric.
I hope that you got some answers or its resolved for you. I will also be raising an MS support ticket.
Jerome22MS is still unable to solve the problem for me.
and we are suffering billing issues, we are paying more than our capacity!
so check if it's the case for you too.
at least for us we are in F2 mode and some day we are paying 23$ isntead of 13$, not a so big issue, but still a concern.
I am having the same issue. I thought it was normal for there to always be CU activity because I have never had anything but. Today I realized it didn't make sense that when there is nothing running there should be no or minimal CU activity. I run the dataflows twice a day and this is my CU usage... flatline.... ~20% usage on a 32.
I opened a support ticket as well with screen shots, my first interaction were links to two documents I've already read. I need an answer to - if no processes are running - what is the expected CU?
Anyone make any progress on this?
2408220040006359- Jerome22
Hi,
first... you are "lucky", your blue line is going down... mine is going up , never down! 🙂
and I have to pause and resume my instance to solve the problem (the big spikes are when I pause and resume)
I discussed with MS for my case, which is the same as yours.
but I don't understand clearly how the background activities are calculated.
it's far from a clear answer.
they agree that there is a problem and some activities should not be there in the CU usage.
but they were not able to provide any ETA to solve the issue.
did you get some info on your side?
- Jerome22
for info.
I increased my capacity to see how fast the system can recover.
there was a drop just after this change in capacity, but not enough to make the environment useable again.
there is still a plenty of CU to burn down for no reason.
I finally pause my Fabric Capacity.
I got a spike of 112 000 CUs (or about 101 000% of CU usage)
Then I restart my capacity and everything was back to the normal. no more CU consumed for no reason.
and finally I run a simple pipeline which took 5min to copy some data.
and boom, again I have a constant CU usage of 31% !!!! (1h after I ran the pipeline and everything is IDLE)
the capacity metric report is definitly broken or the CU evaluation is broken.
I no longer have processes consuming my capacity for no reason.
v-cboorla-msftMicrosoft Employee
Hi Jerome22
Thanks for using Microsoft Fabric Community.
Apologize for the issue you are facing. The best course of action is to open a support ticket and have our support team take a closer look at it.
Please reach out to our support team so they can do a more thorough investigation on why this it is happening: Microsoft Fabric Support and Status | Microsoft Fabric
After creating a Support ticket please provide the ticket number as it would help us to track for more information.
Meanwhile you can try the below steps that might help you.
Temporary Glitch: Clearing cookies and caches can sometimes resolve temporary glitches within the application that might be causing the issue.
Corrupted Data: In rare cases, corrupted data stored in the browser's cache related to Microsoft Fabric might be causing the issue. Clearing the cache removes this potentially problematic data.
Hard Refresh: A hard refresh bypasses the cached version of the webpage and forces the browser to download the latest version from the server. Press Ctrl+Shift+R (Windows) or Cmd+Shift+R (Mac).
I hope this information helps.
Thank you.
- Jerome22
Hi,
For info, the dev teams is still not able to solve the issue after 3 weeks.
we also found that the price we are paying is more than the capacity we setup!
for now we are in F2 (plan to upgrade later with an agreement when the problem will be solved) but paying up to 50% more some days! (23 CAD$ instead of 13$)
I start to be very worried about the capacity of your teams to support Fabric. if the devs are not able to find the root cause for something like this, this means the product has a lot (too much) issues to solve everywhere else.
frithjof_vCommunity Champion
Please watch this YouTube video about the Fabric Capacity Metrics App. Hopefully you will be able to find the answer to the issue after watching it:
Tips on Using the Fabric Capacity Metrics App
https://youtu.be/EuBA5iK1BiA?si=8hm207pOqcrn6pt7
I think what confuses many users, is the fact that some visuals in the Metrics app show the smoothed consumption (while other visuals show the raw consumption). Please watch the video, it explains it quite well.
Also, please note that when you pause a capacity, the remaining cumulative overages and smoothed operations on your capacity are summed, and added to your Azure bill. https://learn.microsoft.com/en-us/fabric/enterprise/pause-resume
- Jerome22
yes, I know the remaining CUs are billed.
but for now there is a bug, the system still counting terminated activities and background activities still appeared without any reason.
so my background CU is going higher and higher and never down like we can see anywhere else.
the extra CU I pay are just few minutes of CUs, so normally these CU should be recovered during IDLE time without any issue, but it's not the case.
for example, during 1am and 5am there is no activity in fabric, this mean that any terminated background activity can use this available capacity to "reimburse" Microsoft. but I'm seeing an increase in usage...
in the video you sent, we can clearly see that the blue chart (background) is going up than down and than back to 0 most of the time, even after there was big front activities.
also in the video the number of background activities is pretty low, few hundreds.
in my case this value jump to hundred of thousands !!! yes more than 200 000 background activities, during IDLE time. really crazy.
and I'm not doing something complicated or resource intensive. (compared to the video)
well... there are issues and MS has to solve this. or improve the way background activities are "billed"
- frithjof_v
When we use the Multi Metric Ribbon Chart to drill down to a specific day and a specific hour (for example 1 am), and then click on the specific hour on the x-axis of the Multi Metric Ribbon Chart, this crossfilters the Items (14 days) visual so that it changes name to Items (1 day).
However, when we select a specific day and hour combination in the Multi Metric Ribbon Chart, we are actually able to see the list of which artifacts are consuming CU (s) in that specific hour, by looking at the Items (1 day) visual.
Which artifacts in your capacity are using CU (s) at that time? What type of items do you have in the Items (1 day) visual when you use the Multi Metrics Ribbon Chart to cross-filter down to a specific day and hour? For example, 1 am on Oct 13. This is very useful to understand what is going on.
The two visuals mentioned above show the actual, raw consumption. So these are the visuals which are easiest to interpret.
The CU (%) Over Time visual shows the smoothed consumption, so it is a bit more difficult to interpret. However, if the CU (%) Over Time visual is rising in the specified hour, there must be some artifacts in the Item (1 day) visual that tells which artifacts are the cause of the rise.
The drill-through page (Timepoint) also shows smoothed consumption mainly, so that page is also more difficult to interpret. Also note that a single refresh of a semantic model will trigger several background operations. So it's better to use the main page's Multi Metric Ribbon Chart to cross-filter the Items (14 days) visual in order to identify which artifacts are the cause of the CU (s) consumption. And use the tooltip in the Items (14 days) visual to see which kind of operations are consuming CU (s) for your items.
Just finding out that Fabric runs a load of background processes (One Lake Indexing etc) even when the CUs aren't being actively used. Apparently, the admin has no control over this and the only way to ascertain what's going on is by poring over the inaccurate and misleading data in the frankly crappy metrics app.
I've just experienced an hour of downtime on my production reports because of this even though I'm well within my SKU limits for the actual operations I'm performing.
Looks like we've been missold again with Fabric. Sounded so good too:
As per copilot's comments, you as an admin have to proactively come up with strategies to get around this whilst fumbling about in the dark because Microsoft actively hide this sort of detail from their customers.
