What is the Interactive "AI Function Evaluation" operation?

[Anonymous]

My gen2 capacity has alerted me to a spike in usage. When reviewing the gen2 capacity usage report, I see the below image.

At a very specific time, multiple occurences of "AI function evaluation" occurred at the exact same time to the second, under "Interactive Operations".

The dataset was not refreshed, so I'm assuming someone is doing something in the report causing this action?

The report itself is very basic. It shows fields in a matrix, but does allow the users to personalize their views.

How do I tell what caused this spike leading to over 75% usage of my capacity?

[Smilo]

According to the microsoft documentation it says that the AI Function Evaluation is caused by copilot activities. I recently investigated this and found that copilot activities are listed as "Copilot in Fabric" when used in both desktop and service. As for the AI Function Evaluation I tested this with a few reports on our one of our capacities and found that if the developers add Q&A or narrative chart in any of the report pages as the users interact with the visuals the narrative updates the insights based on these interactions and the consumption units for the insights generated are listed as the AI Function Evaluation.

You can add these visuals in any of your reports and test

[weizhang]

1. Report Settings – You can disable insights directly in the report settings.
2. Admin Portal (Tenant Settings) – Search for “insights” in Tenant Settings; there should be two relevant settings.

I've attached both settings below for reference.

Details of the Two Settings:

• Receive notifications for top insights (Preview)
  Users in the organization can enable notifications for top insights in report settings.
  Learn more about insights here. 

• Show entry points for insights (Preview)
  Users in the organization can use entry points for requesting insights inside reports.
  Learn more about insights here.
  
  Note: only report creator or tenant admin can view/edit the settings

[SaiTejaTalasila]

Hi @sxz ,

If your tenant admin enabled copilot even the operation can be triggered from copilot.

When Copilot operates within Microsoft Fabric, it utilizes resources from the premium capacity, including CPU and other computational resources. The usage of these resources is tracked and reported as part of the Al function evaluation metrics.

I hope it will be helpful.

Thanks,

Sai Teja 

[sxz]

Are you aware of why this only happens during scheduled refreshes?
Also, I found that disabling the 'Share Q&A synonyms with your organizations' option resolved this. No more AI Function spikes.

Does that resonate with you? Thank you.

[sxz]

I'm having the same issue. However this (AI Function spikes) happens only during scheduled refreshes, which is weird b/c refreshes technically should take background resources other than interactive resources. Very frustrating.

[JessieF]

Hello All,  
I realize this conversation string is a bit old, but I'd like to resurrect if possible. I feel like I presently have the same question as the original poster.  Today our Gateway crashed and we restarted, but during review of the capacity metrics report (yes I upgraded it), I see 81.41% of the CUs at 8:10am and many of these operations are showing AI Function Evaluation and attributed to an end user.  This account is a Business user, not a report publisher, which makes it extra curious that it's a background operation and interactive operation.  I really feel that there needs to be a better definition of the "AI Function Evaluation" operation, especially if we are supposed to use this report to make sense of our tenant activity.

Thanks for the attention in advance.

Jessie.

[v-zhangti]

Hi, @Anonymous 

 

I have searched a lot of information and AI Function Evaluation is probably the general name about AI Function Execution.

AI Function Execution: Total number of executions in the past seven days.
AI Function Execution Reliability (%): Number of successful executions divided by the total number of executions in the past seven days.
CPU Max (%): Max CPU consumption by the AI workload in the past seven days.
Memory Max (GB): Max memory consumption by the AI workload in the past seven days.
AI Function Execution Max Wait Time (MS): Maximum amount of time before starting execution.
AI Function Execution Average Wait Time (MS): Average amount of time before starting execution.
AI Function Execution Max Duration (MS): Maximum amount of time to complete execution.
AI Function Execution Average Duration (MS): Average amount of time to complete execution.

This is the relevant document, hope to help you:

https://docs.microsoft.com/power-bi/admin/service-admin-premium-monitor-capacity 

Best Regards,

Community Support Team _Charlotte

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[Anonymous]

I understand what the metrics mean. However, I'm still unclear on what "AI Functions" are.

As it is the operation taking up 75% utilization of my capacity on 1 report for 5 seconds, that is the part I'm looking for.

There is no explicit AI happening within this report. It has 1 table, and 5 slicers. 2 dax measures doing an average count.

[AlexisOlson]

Interesting question... 

This documentation seems to suggest that AI workloads are not interactive operations:
https://docs.microsoft.com/en-us/power-bi/admin/service-premium-gen2-faq

The following events are interactive operations:

• Datasets workload - Report View, Query, XMLA read
• Dataflows workloads
• Paginated Report workload - paginated report render

The following are background operations:

• Datasets workload - scheduled refresh, on-demand refresh, background query (after refresh)
• Dataflows workload - scheduled dataflow refresh
• Paginated reports workload - data driven subscriptions renders
• Export report to file API calls
• AI workloads

[Anonymous]

Thank you for that link, that didn't turn up in my searches.

This particular report is refreshed overnight, so based on the times it would be something caused by interaction from a user.

I'm also curious as to what it may be that caused high number of 5% usage actions that all occurred at the exact same second in time.

This report is published in sharepoint, so MOST users would not have access to thinks like insights generation or other visuals that would cause azure AI to be used.

Very few people have a pro license to view this report within the portal to allow them to initiate any AI related visuals.