Forum Discussion
Lakehouse high CU usage, unknown source
Hi
Can anyone please help me understand what is causing this high usage. The ratio between seconds and CU's consumed seems very skewed, and I do not know what is causing it. Here are some screenshots from the monitoring app. How can 85 seconds cause 1.2 million CU's to be consumed??
Here is the detail:
Support finally got back to me with this reply. They admitted to it being a backend Microsoft issue, but no word on how and if they would be fixing it or if I would be reinbursed the lost CU's. The abbreviated 'DMS' is for Data Movement Services:
The increased CU usage was attributed to a disabled file system (FS) on the DMS side, which resulted in a significant number of “GetBlobProperties” API calls from DMS to the One Lake Client (OLC). This led to the observed spike in CU usage. After the FS was re-enabled on July 29th, CU usage decreased accordingly.
The spike occurred on 07/18 and subsided on 07/29, aligning with the period when the FS was disabled and subsequently re-enabled on the DMS side, which reduced the number of calls from DMS to OLC.
17 Replies
Support finally got back to me with this reply. They admitted to it being a backend Microsoft issue, but no word on how and if they would be fixing it or if I would be reinbursed the lost CU's. The abbreviated 'DMS' is for Data Movement Services:
The increased CU usage was attributed to a disabled file system (FS) on the DMS side, which resulted in a significant number of “GetBlobProperties” API calls from DMS to the One Lake Client (OLC). This led to the observed spike in CU usage. After the FS was re-enabled on July 29th, CU usage decreased accordingly.
The spike occurred on 07/18 and subsided on 07/29, aligning with the period when the FS was disabled and subsequently re-enabled on the DMS side, which reduced the number of calls from DMS to OLC.
They won't reimburse unless you push them for it. Make sure you have the receipts.
I have done nothing to or with (i.e. no pipeline, notebooks queries etc reading or writing) these lakehouses today, yet the comsumed CU's is exceptionally high, I would have expected them to be zero.
Since you have a Pro license you should open a Pro ticket at https://admin.powerplatform.microsoft.com/newsupportticket/powerbi
Thanks, I was trying in the wrong place for support ticket! I did successfully log a ticket and will update this post with Microsoft's findings.
Very interesting.
Tomorrow I'll take a look at my lakehouses to see if we also have such high Cu utilization rates.
I would have noticed that, though.I'll get back to you tomorrow.
Best regards
Just an update: Stopping and restarting the Capacity seemed to have stopped whatever these rogue background processes are that were consuming the CU's. The support ticket is still open/under investigation so no word on what the root cause is/was.
Hello schneiw ,
Here you can find a Microsoft article on what all the processes are.
https://learn.microsoft.com/en-us/fabric/onelake/onelake-consumption
Another article with an example, I think you should now be able to figure out what generates these CUs.
https://learn.microsoft.com/en-us/fabric/onelake/onelake-capacity-consumption
And then here is the general description of the metric APP.
https://learn.microsoft.com/en-us/fabric/enterprise/metrics-app
I hope the information helps you.
Best regards
Thank you for your links to help me understand. Its still not that clear what is causing this. From the documentation its based on "operations", which are suppoed to be 104 CUs per 10000:
Using the FAUM app, I can see then how many operations are being perfomred on a given day per Lakehouse. For example most of these lakehouses are averaging around 2900 operations for 3 seconds, yet the total CU's are vastly different for some . Something isnt adding up here.....
To take it one step further, Iterative Read via Proxy is 4798 Cu's per 10000 vs the 104 per 10000 for Other Operations via Redirect (see below).
Yet take one day example of ours, where we have 1.4 million operations of Iterative Read via Proxy vs. 692K Other Operations via Redirect so about half. Yet the total CU's is 607 thousand vs 4.2 million
If I do the math, 692 326 operations / 10 000 * 104 = 7200 CUs. Not sure where they are getting 4 217 837 from ? That would mean we are being charged 692 326 / 10000 * x = 4 217 837 ....therefore x = 60 918 CUs per 10000???
