need help
116 TopicsBest practice for handling schema evolution in Fabric Eventstream before data reaches Eventhouse?
I have an Eventstream receiving operational events where the schema may evolve over time. For example, the producer initially sends: DeviceId, Timestamp, Temperature, Status but later adds fields such as: Location, FirmwareVersion, ErrorCode I want the pipeline to continue ingesting events without breaking downstream KQL tables, update policies, materialized views, or Real-Time Dashboards. I am trying to understand where schema evolution should ideally be handled in a production Fabric RTI architecture. Would you: enforce the contract upstream using Schema Registry normalize changing fields inside Eventstream before Eventhouse ingestion land the raw payload first and handle schema evolution inside Eventhouse/KQL maintain separate versioned event schemas/tables How are people handling this in production when producers can add fields without notice? I am particularly interested in avoiding a design where every small upstream schema change forces updates across Eventstream, KQL tables, update policies, and downstream dashboards.Solved81Views0likes2CommentsBest practice for deciding between Eventstream transformations and Eventhouse update policies
Hi Fabric Community, I am exploring a Real-Time Intelligence architecture and would appreciate some guidance on where transformation logic should ideally be placed. The proposed flow is: Azure Event Hubs → Fabric Eventstream → Eventhouse → Real-Time Dashboard / Power BI Fabric Eventstream supports filtering, field management, aggregation and other processing before events are written to the destination. An Eventhouse can also ingest the raw events first and transform them into curated tables through KQL update policies. I am trying to understand the recommended boundary between these two layers. For example, assume the incoming event contains: Device or customer identifier Event timestamp Event type Location Numeric readings Additional JSON properties The required processing includes: Removing events that fail basic validation Renaming and standardizing fields Converting timestamps and data types Flattening selected JSON properties Enriching the event with reference data Creating five-minute aggregates Preserving the original event for auditing and future reprocessing My current thinking is: Use Eventstream for lightweight filtering, routing and simple schema normalization. Land the original event in a Bronze table whenever replay or auditing is required. Use Eventhouse update policies or KQL for enrichment, reusable business logic and curated Silver tables. Use materialized views for frequently queried aggregations rather than calculating them repeatedly in dashboards. However, I am unsure where Microsoft recommends drawing the line. A few questions: Are there transformation types that should generally remain in Eventstream rather than Eventhouse? Is it considered good practice to send both the raw stream and a transformed derived stream into separate Eventhouse tables? When using Eventstream’s Event processing before ingestion mode, what are the trade-offs compared with direct ingestion followed by an Eventhouse update policy? How do teams handle changes to transformation logic when historical events need to be reprocessed? For reference-data enrichment, would you normally perform the lookup in Eventstream or after ingestion with KQL? Are five-minute or hourly aggregations better implemented in Eventstream, through an update policy, or with an Eventhouse materialized view? Microsoft’s Eventstream destination guidance documents both direct ingestion and event processing before ingestion, while the KQL update-policy documentation provides another way to transform ingested data. I would be interested to hear how others divide responsibility between Eventstream and Eventhouse in production, particularly where auditability, reprocessing and maintainability are important. Thanks in advance!Solved90Views0likes2CommentsNew to microsoft fabric- time intelligence.
Hello everyone! I'm new to microsoft fabric and Real -Time Intellligence. I am excited to learn how the real time data processing works and look forward to learning from this community. I am beginner to microsoft fabric . So as a beginner with basic programming knowledge, what are the best beginner resources to start learning Real-Time Intelligence? I hope you will favour me....237Views2likes5CommentsReal time dashboard
Since the latest Microsoft Fabric update, I've noticed a change in how Real-Time Dashboard refresh behaves. Previously, I could configure visuals with Continuous refresh, allowing them to update almost immediately whenever new data arrived (typically within a few seconds, with a maximum delay of around 10 seconds). After the recent update, it appears that continuous refresh is enabled by default, but visuals now refresh at a fixed minimum interval of 10 seconds. In addition, visuals that don't support continuous refresh—or when no new data is received—seem to refresh every 30 seconds. I'm trying to understand whether this is expected behavior or if there's a way to restore the previous experience. Specifically: Is it still possible to configure Real-Time Dashboard visuals to refresh more frequently than every 10 seconds? Has the previous event-driven continuous refresh option been removed, or is it available through another setting? Are there any recommended approaches to achieve near real-time updates when new data arrives instead of relying on fixed refresh intervals? Was this change introduced intentionally as part of the latest Microsoft Fabric release? Our dashboards are used to monitor live operational processes, so even a 10–30 second refresh delay has a noticeable impact compared to the earlier behavior. Any clarification or guidance from the community or Microsoft team would be greatly appreciated.Solved128Views1like1CommentEnable configurable recipients for Activator error notifications - critical for monitoring
According to the official documentation: "If a problem occurs with any of your Fabric Activator events, objects, or rules after you create them, then Fabric Activator sends you an email containing an error code." However: The recipient is not configurable (hardcoded to item creator only) This limitation is not documented No alternative notification channels are available This creates a critical blind spot for production monitoring when: Activators are deployed via service principal (cannot receive emails) The original creator leaves the team or changes roles Multiple team members need visibility into system errors Requested Feature: Enable configuration of error notification recipients through: Workspace-level notification settings Distribution list/group email support Integration with workspace contacts/admins Use Case: Our production Activators are deployed via CI/CD pipelines using service principals. Without configurable notifications, we have no way to receive critical error alerts, making Activator unsuitable for production use4.9KViews2likes6CommentsNot able to creat alert over a table chart without removing columns
I've been creating alerts on a table chart based on different filtering, however it stopped allowing me to create alerts on that table since 3 days ago, as a test I removed all the alerts previously created and now I cannot create any new alert over that table chart. I tested by removing columns and that is the only way i'm able to create alerts again on that table chart card, then republishing the report adding the deleted columns. With all the columns I'm able to select the column to base the alert but after that, the alert it is not created, and I'm not able to proceed:1.9KViews2likes5CommentsWorking with Microsoft Fabric Support - A Collaborative Approach
At Microsoft Fabric Support, our goal is simple: help you resolve issues as quickly and smoothly as possible. Just like the recommendations shared in the Microsoft Fabric Community’s guidance on getting questions answered effectively, providing clear and complete information upfront helps everyone move faster and avoid unnecessary back-and-forth. Support is most effective when it’s a collaborative process. You bring the knowledge of your environment and workloads, and we bring deep platform expertise and diagnostics to help investigate the issue together. Help Us Help You When opening a support case, please include as much of the following information as possible: Issue Details Clear description of the problem Expected behavior vs. actual behavior Is the issue intermittent or consistently reproducible? Approximate timeframe of when the issue occurred (UTC preferred) Error Information Please include: Full error message text Screenshots (highly recommended) Activity IDs / Correlation IDs if available Even small details can significantly speed up investigation. Environment Information That Helps Investigation Depending on the Fabric workload being used, please provide the relevant item IDs and environment details. Workspace Information Workspace ID / Workspace URL To find this: Open the Fabric workspace Copy the URL from your browser Example: ".. https://app.fabric.microsoft.com/groups/<WorkspaceID>/… …" The value after /groups/ is the Workspace ID. Screenshot example above showcases the WorkspaceID and EventhouseID. Fabric Item IDs Please provide the item ID related to the affected workload, such as: EventhouseID ActivatorID LakehouseID WarehouseID SemanticModelID EventstreamID PipelineID These IDs help us locate telemetry and backend diagnostics more efficiently. You can typically find these: In the browser URL while inside the Fabric item Within item settings/details pages From the Fabric portal navigation pane Additional Environment Details Please also include: "… WorkspaceID: Capacity Name: Region …" Example: "… WorkspaceID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx Capacity Name: Fabric-Prod-EastUS Region: East US …" This information helps us identify: Capacity-level issues Regional service impact Configuration-specific behavior Why This Matters Microsoft Fabric is a distributed cloud platform, and troubleshooting often depends on correlating logs, telemetry, timestamps, and resource identifiers together. Providing complete details upfront helps: Reduce delays Avoid repeated clarification requests Accelerate root cause analysis Create a smoother support experience for everyone involved We’re here to partner with you throughout the process, and we truly appreciate your collaboration in helping us investigate issues effectively. We look forward to working with you!1.1KViews5likes3CommentsDigital Twin Builder (preview): Bounding storage growth for an incremental time-series
Is there any supported way to apply a retention horizon to a Digital Twin Builder time-series mapping that uses incremental processing, so the underlying dtdm table doesn't grow without bound? My scenario: high-frequency live telemetry landing in an Eventhouse, surfaced to DTB through a lakehouse table (I've used both shortcuts and pipeline copy activities in my tests). The static layer and relationships are working well. The question is purely about the time-series layer. With incremental mapping enabled, the series appends continuously and the dtdm table grows linearly with no eviction. I've confirmed a couple of things that don't solve it, so I'll pre-empt them: Retention on the source table doesn't bound the twin copy. Incremental processing is forward-only, so source rows aging out are never removed from data the mapping already ingested. A mapping filter governs ingestion, not eviction, and only limits initial backfill (no relative/dynamic date filter). I assume full reload (incremental disabled) over a windowed source table does keep things bounded, since each run replaces the prior load, and that's my current fallback. But I'd like to confirm whether there's any retention/TTL control I've missed for the incremental path specifically, at the mapping, entity, or item level, or any sanctioned way to age out rows from the dtdm time-series table. If the answer is "not currently supported," that's useful to know too, and I'll file it as an Idea. Running on a Fabric capacity, DTB still in preview. Thanks in advanceSolved1KViews0likes5CommentsActivator: issue when set an action by posting a message in Teams channel
Hi, I've succeeded in triggering an alert based on a KPI in a Power BI report, and that works fine. However, I want to send a message to my community in a Teams channel. When I select the appropriate option in the Activator action pane, I can't select the team or the channel, even though I'm signed in with my account. It's strange because email and personal Teams messages work without any issue. Thx for your support!Solved1.1KViews0likes3CommentsActivator Email Delivery Inconsistent — Alerts Recorded in History but Emails Delayed and Missing
Hi, I am currently working with Microsoft Fabric Activator, where I have configured an alert on a KQL function output from an Eventhouse datasource. The alert is designed to trigger every 10 minutes, based on a change detection condition on a computed column (AlertTriggerKey) that increments on every 10-minute window. Observed Behavior Activator is detecting the condition and recording each trigger correctly in the alert history, confirming that the rule evaluation and condition matching are functioning as expected. However, the email delivery cadence is inconsistent and does not align with the expected 10-minute interval. Specifically, the pattern I am observing is inconsistent and does not follow a fixed delay. While some alerts are delivered on time, others arrive with unpredictable delays ranging from 30 minutes to over an hour, and certain alerts are missed entirely with no corresponding email delivery whatsoever. There is no consistent gap between the expected trigger time and the actual email delivery time — the delay varies arbitrarily across alerts and some triggers recorded in Activator history have no corresponding email delivered at all, making it impossible to rely on the notifications for real-time operational decisions. What Has Been Verified Rule configuration is correct Trigger condition (AlertTriggerKey → Changes) is working as expected Activator history confirms alerts are being recorded at the correct 10-minute intervals Throttle limit is not being hit — only 6 alerts per hour Recipients and action configuration are correctly set up Emails are not being filtered into spam or junk Core Issue The alert history and the email delivery are out of sync. Activator is recording the trigger at the correct cadence, but the email notifications are being delivered with delays and in some cases are dropped entirely, suggesting the issue lies specifically in the email dispatch pipeline, not in the rule evaluation or condition detection layer. Has anyone encountered a similar pattern where Activator history is accurate but email delivery is inconsistently delayed or completely missing? Is there a known latency issue or queuing behavior in the Fabric Activator email dispatch system that could explain this? Any guidance or workaround would be greatly appreciated. Thank you.Solved1.4KViews0likes3Comments