Data activator: Reflex/Event stream using increasing amount of CUs.

[kjellvs]

Dear all,

I am using data activator on my dedicated capacity (event stream + reflex). My consumed capacity is at 3600% of my capacity (35x too high), this started at 400%, but is slowly increasing and increasing. This results in us not being able to create new notebooks, and getting throttling requirements. When looking into my timepoint detail, this is mainly due to (1) event processing and (2) trigger evaluations.

The event stream has the following properties:

2.8k events per hour,  0.8 events per second (< 2 events per second limitations outlined here: https://learn.microsoft.com/en-us/fabric/data-activator/data-activator-limitations) 

0.4KB per event -> 1.12MB per hour. (https://learn.microsoft.com/en-us/fabric/real-time-analytics/event-streams/monitor-capacity-consumpt...

data activator evaluates 10 triggers on 12 different properties, for ~200 object instances that get a message every ~4 minutes (aprox. same throughput as above).

Now I am wondering the following:
1) How should I proceed? I want to use data activator with event stream and reflex; but not require an F2048.

2) will i get billed for this extreme CU utilization?

Hope you can help,

kr, 

Kjell

[kjellvs]

Hi Neeko,

thank you for your reply. is there any information on what type of triggers are expensive? (in terms of logic) or can we see this data emperically? i.e., see how much compute is used per trigger? This would be crucial to understand our improvement efforts. I haven't found such my self.

kind regards,

Kjell

[v-tangjie-msft]

Hi @kjellvs ,

For the triggers, consider if the logic can be simplified, or if the number of triggers can be reduced without impacting the business logic. Consider using more efficient data types or reducing the precision of data where possible to decrease the size of events.

Regarding your second question, refer this, consumption rates are subject to change at any time. Microsoft uses reasonable efforts to provide notice via email or through in-product notification. I would advise you to review the billing terms for your dedicated capacity.

Best Regards,

Neeko Tang

If this post  helps, then please consider Accept it as the solution  to help the other members find it more quickly.