Forum Discussion
API / programmatically gather storage/capacity/CU metrics for warehouse tables in Fabric (Notebook)
- 10 months ago
Hi stuckerj ,
You can refer this -
Now a days i am using this report to track the utilisation.You can check fabric workspace monitoring a new report currently under preview.
https://learn.microsoft.com/en-us/fabric/fundamentals/workspace-monitoring-overview
Thanks,
Sai Teja
Hi stuckerj,
As far as I know there’s no supported, per-table/per-operation CU or storage meter you can call for warehouses, or any other artifact for that matter. You can get close with a few pragmatic pieces you can script from a Notebook, but none of them give "table X consumed Y CU during this INSERT".
You can get some level of CU metrics from the Capacity Metrics app's semantic model, though last I checked (which to be fair was a while ago) I wasn't able to get timepoint level details for specific items, only the overall usage at a given timepoint. This is because the capacity metrics app uses a live connection to a shared Kusto instance to pull the data, and that connection uses a custom connector that isn't exposed to us. I tried playing around with the XMLA endpoint quite a bit with little success to get the tables that rely on that connector to show me data.
For table size, I'm not aware of a nice way to snapshot this in a warehouse. In a lakehouse, you can use SparkSQL to describe the metadata and get the file size of the parquet files. Given that warehouses also store data in delta parquet format, you might be able to use the onelake APIs to do something similar, but I haven't ever tried.
As for more realtime metrics, I recommend you keep an eye out. Microsoft likes to announce new things at Microsoft Ignite, which is coming up soon, so if anything was in the pipeline, I would expect it to be announced there 😜
If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.
Microsoft Example
Does anyone know how these numbers were gathered?
- tayloramy10 months agoSuper User
Hi stuckerj,
Microsoft has access to capacity metrics data that we do not have access to yet. So they can build things like this that we cannot.
If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.