User Profile
svelde
Super User
Joined 2 years ago
User Widgets
Contributions
Re: Best practice for handling schema evolution in Fabric Eventstream before data reaches Eventhouse?
Hello ShivekMaharaj , Welcome to this Microsoft Fabric forum. I have used the Eventstream schema support a few times already. I mainly use it to split up messages coming from multiple devices or services but arriving at the same Eventstream source. Think of an Azure IoT Operations device having a connection to an Eventstream custom endpoint. This device sends out multiple types of messages, so I can split them out in the Eventstream towards different destinations. So I can reuse the same 'pipeline' from the edge. The same goes for multiple sources, each having its own kind of message type, and again, these must be redirected to different destinations. There are challenges. I tried to use this on a source with messages from many LoRaWAN devices. Within one hour, I got 50+ message schemas from ~15 devices. This was because some devices sent multiple messages, even partial messages (having a subset of the properties. This could differ. For example, when a GPS sensor had no fix, the latitude and longitude were left out. This abundance of schemas (which keeps growing) made practical usage of schemas impossible. Partial messages, while having schemas enabled, could lead to dropped messages, and there is no dead-letter box. So, I only use the schemas for filtering if I'm in control of the message formats and the flow demands splitting the stream in the Eventstream, too. Otherwise, I advise just ingesting and passing on all (raw) messages to an Eventhouse (as a dynamic) and making use of table update policies so at least the raw messages are persisted. Then, in case downstream functionality fails due to a changing format, the changed (raw) messages are still persisted, and you can act on the changed formats. --- If this answer helps you, marking it as the accepted answer is appreciated. All community members with similar questions will benefit by doing so. Your contribution is highly appreciated.Re: Capacity overview events and Timepoints details
Hello rgsalido Welcome to this Fabric Community forum. Regarding the Fabric Capacity overview events, I checked the documentation. There, you can access these events via the Real-Time Hub: I tested this shortly, and it seems available as an Eventstream source. Unfortunately, I do not have the right permissions. You need to be an administrator: These rights limitations seem fair due to the wish to access the capacity itself. --- If this answer helps you, marking it as the accepted answer is appreciated. All community members with similar questions will benefit by doing so. Your contribution is highly appreciated.Re: Best Way to Monitor Marketing Campaign Performance in Real Time Using Microsoft Fabric or Power BI?
Hello iluvsmmpanel welcome to this Fabric community portal. You want to collect datetime-based events, observations, and facts that are immutable. For this, Eventhouse is the perfect database because it's a time series database, optimized for this kind of data. I do not have all the details about the sources, but depending on the source, you should either go for: Data pipelines (which as many connectors out of the box) Python notebooks (which is a pro-code solution, so very flexible but you need some Python programming skills or vibe code it along the way) Eventstream if there is already a streaming solution offered by the sources. This is a no-code approach I recommend storing data from each source in separate KQL Database tables. If the incoming data needs transformations, check out table update policies. This offers in-flight transformations from one table to another table, ideal for a 'medaillion architecture' approach. From there, you can deduplicate data via materialized views. Or add custom KQL functions for special (joined) subsets of the incoming, transformed data. You could use Power BI with DirectQuery access to the data for visualization. But you could also invest some time in real-time dashboards, which give more control (you write KQL directly for each data pane) and have a smarter refresh policy. You can demonstrate your solution as a blog post on this forum later on. If you need pointers to a learning path or training, please let us know. --- If this answer helps you, marking it as the accepted answer is appreciated. All community members with similar questions will benefit by doing so. Your contribution is highly appreciated.Re: New to microsoft fabric- time intelligence.
Hello Anj_il_a- Welcome to this community forum. I recommend looking at this MS Learn Learning path for 'Implement Real-Time Intelligence with Microsoft Fabric'. This gives access to several learning modules, enough for a full day of learning Fabric RTI. Next to an introduction to working with real-time data, you learn about the different tools in Fabric for eg., ingesting data, storing data, querying data, visualizing data, and acting on data. Notice that these learning modules lead to badges and trophies. If you want to make your new knowledge measurable, take a look at the Microsoft Applied Skill 'Implement a Real-Time Intelligence solution with Microsoft Fabric'. Once completed, you get a badge which is marked as ‘online verifiable’. This assessment will use an interactive lab to evaluate your performance. See also this blog post for details. But most of all, try tinkering in Fabric RTI with the new Fabric RTI Items and learn by doing. --- If this answer helps you, marking it as the accepted answer is appreciated. All community members with similar questions will benefit by doing so. Your contribution is highly appreciated.
