real-time analytics

9 Topics

High latency in Fabric Eventstream using event processing with KQL Database
Hello! I am testing Fabric real-time analytics with KQL databases and eventstreaming from Event Hub. Using Fabric Eventstream with direct ingestion to a KQL database, I get the latency I expect (a couple of seconds) and it seems to work fine. However when I add event processing before ingestion, with a simple filter operation and mapping columns operation, the ingestion latency into the KQL database seems to go up to 30 seconds. In my test I have about 20 small events being sent every second. The watermark delay graph in the EventStream still shows 3 second, but as I said the latency is higher. The inserts into the KQL database seems to get batched together in larger groups and inserted with much higher delay than when using Eventstream with direct ingestion. I am thinking it maybe uses Kusto Ingestion batching, instead of Streaming Ingestion? Is this something that is expected, and is there a way to alter settings to reduce the delay? Best regards, Emil
Solved
Anonymous
2 years ago Place Activator
5.7KViews
1like
2Comments
How to feed Streaming data to Custom App in Eventstream in Realtime Analytics Fabric using Fabric
Hi, I want to stream data stored in a csv file in Azure Storage account by feeding data to Custom App Event Hubs data source in Eventstream in Real time Analytics in Fabric. I want to use a Fabric solution to implement this. Any help on this would be appreciated. Thanks.
Anonymous
2 years ago Place Activator
3KViews
0likes
2Comments
Microsoft Fabric - Feature Roadmap and Data Activator Public Preview
Hi All - any available roadmap for Microsoft Fabric feature roadmap? Also, when shall we get our hands on experience with Data Activator?
Solved
AnshumanB
2 years ago Place Activator
1.5KViews
1like
1Comment
Anomaly detection in Microsoft Fabric
Hi there We're trying to implement an anomaly detection pipeline that detects sensor data anomalies in real-time. We previously used Azure Anomaly Detector service for building the PoC of the pipeline but we cannot create that service anymore since 20th of September 2023 and it's getting retired in October 2026. I was hoping to see an alternative service to perform anomaly detection and I came across this documentation on performing multi-variate anomaly detection in Microsoft Fabric but the article wants us to create an Anomaly Detector resource which is not possible anymore. Would you please let me know how to perform anomaly detection in Azure? since the documentation is a bit confusing
Solved
Mohammadreza-24
2 years ago Place Activator
3.8KViews
1like
2Comments
Ingest data from Azure Event Hubs into Fabric KQL DB
Hi, I'm trying to I'm trying ingest data from Azure Event Hubs to Fabric KQL DB but I'm having issues with the connection. (FYI this is working in Azure Stream Analytics with the same event hub as a source) First it can't find any exisiting connections, even though there are EventHub connections created And second, even if I create a new connection it's not possible to continue. The consumer group is greyed out and I can't click next
Solved
DC07
2 years ago Place Activator
3.4KViews
0likes
3Comments
Real-Time Analytics Tutorial on MS Learn Fabric dataset not real time
Hi, I just finished the tutorial and notice that the Yellow Taxi dataset (high data-rate) is not at all high rate or even updated, its only 2022 data. In the tutorial we set the refresh rate to 10 seconds indicating that more data will appear. Have i done something wring or is this just a "dry" practice? Thank you Tutorial from MS Learn: https://learn.microsoft.com/en-us/fabric/real-time-analytics/tutorial-introduction
Solved
Dejo
2 years ago Place Activator
3.4KViews
0likes
4Comments
Calling Custom App endpoint for Eventstream in Fabric from Postman
I've created an Custom App endpoint for my eventstream and can easily call this from Power Automate/Logic App. However, I've some issues calling the same Event Hub endpoint from e.g., Postman seems to be harder. Does anyone have a sample/tutorial/guide on how to push data from Postman to my Eventstream eventhub endpoint?
Solved
trondb
2 years ago Place Activator
4.5KViews
0likes
2Comments
Cloudfiles error
Py4JJavaError: An error occurred while calling o4364.load. : org.apache.spark.SparkClassNotFoundException: [DATA_SOURCE_NOT_FOUND] Failed to find the data source: cloudFiles. Please find packages at `https://spark.apache.org/third-party-projects.html`.
sunilmaghanuru
2 years ago Place Activator
6.5KViews
0likes
1Comment
Eventstream flow of data
Hi, Is there an easy way to (1) check the flow of data on an eventstream, identifying, for example, there was an interruption of 10 minutes? and (2) create an alert if this kind of interruption happens ? I imagine I can do this with data activator, but I was looking for something easier. Kind Regards, Dennes
DennesTorres
2 years ago Place Activator
2.2KViews
0likes
2Comments