eventhouse

47 Topics

Changing Ingestions from JSON
I have an EventStream receiving IoT Data out of JSONs. In my KQL table the columns timestamp, deviceId and for example x and y exist. The data from one sensor gets ingested in the columns timestamp, deviceId and y. No x since there is no data in the JSON for that. I dont work with a Mapping because it only makes it more complicated and the names in the JSON and in the KQL table are identical. No I changed the named for data from y to x when creating the JSON. Since there is no mapping and the column already exists in the KQL table the data should be saved now in column x. My problem is, it doesn't the data is not in x nor in y. It seems to be gone. Is that a bug in fabric? Can I solve this?
MoFabricIoT
18 days ago Place Eventhouse and KQL
Views
0likes
Comments
Fabric Eventstream MySQL CDC scans all databases and does not emit row-level events
Hello Fabric Team, We are testing the Microsoft Fabric Eventstream MySQL CDC connector with an AWS RDS MySQL database. The MySQL instance is accessible through a private IP, and Fabric connects through a VNet data gateway/private network connection. Connectivity is successful, and the connector remains Active without reporting processing errors. Our MySQL environment contains the same table structure across multiple databases, following a pattern similar to: application_<tenant>.sample_table We configured the source to capture only one fully qualified table: application_tenant1.sample_table The MySQL configuration is: log_bin = ON binlog_format = ROW binlog_row_image = FULL The connection user has SELECT, SHOW DATABASES, REPLICATION CLIENT, REPLICATION SLAVE/REPLICA, and the required snapshot permissions. Each Fabric source uses a unique Server ID. CDC from the same RDS instance has previously worked successfully through another CDC platform. However, we are seeing the following behavior: Although only one fully qualified table is selected, Fabric/Debezium scans schema metadata for every accessible database and table on the MySQL server. More than 45,000 schema events were generated across the tenant and history databases. The schema events contain DDL such as DROP TABLE IF EXISTS and CREATE TABLE. Snapshot mode is configured as Initial. The selected table contains approximately 16,000 records, but Fabric emitted zero snapshot r events. We committed a new insert into the selected table after schema discovery finished, but no c event was emitted. Eventhouse receives the schema events successfully, confirming that the source-to-Eventhouse path is operational. The Eventhouse destination stores the Debezium payload in a dynamic column using a mapper from payload to RawEvent. Could you please confirm the following? Is it expected for the Fabric-managed MySQL CDC connector to scan and publish schema events for every database when only one fully qualified table is selected? Does Fabric expose or support settings equivalent to: database.include.list table.include.list schema.history.internal.store.only.captured.databases.ddl schema.history.internal.store.only.captured.tables.ddl Why would the connector finish schema discovery but not emit existing rows from the selected table when Snapshot mode is Initial? Why are committed inserts not emitted after the schema phase appears to finish? Is there a known limitation involving AWS RDS MySQL accessed through a private IP and VNet data gateway? What is the recommended supported destination pattern for this CDC stream? Our main objective is reliable ongoing CDC capture. We are flexible about the landing destination and can use: Eventhouse ADLS Gen2 in JSON or Parquet Fabric Lakehouse/OneLake Snowflake as the final destination Please let us know which logs, connector identifiers, timestamps, or additional configuration details would help investigate this behavior. Thank you.
Solved
girishtharwani2
1 month ago Place Eventhouse and KQL
Views
0likes
Comments
Unable to create a Monitoring Eventhouse in a Microsoft Fabric workspace.
I am trying to enable Workspace Monitoring for a Microsoft Fabric workspace. The user has Fabric Administrator access and is also a Workspace Administrator. However, under: Workspace Settings → Monitoring the + Eventhouse option is disabled/not available. We have verified the tenant-level settings and confirmed that “Workspace admins can turn on monitoring for their workspaces” is enabled.
Solved
jonypanchal7
1 month ago Place Eventhouse and KQL
Views
2likes
Comments
Anomaly Detector is disabled
Hello, I have a big Table in which we have our TimeSeries data from various devices. Since not every device sends the same data we have a big schema of numeric columns that are not filled for every deviceId. Is that the reason why there is no Anomaly Detector in my EventHouse? In the schema I can see that columns for the data are int or real. Some columns are string, but that shoudn't be the reason, right? Has anyone faced the same issue?
MoFabricIoT
1 month ago Place Eventhouse and KQL
Views
0likes
Comments
Eventhouse Capacity Planner minimum CU not reflected in UI/API
Hi all, We've set a minimum of 32 CU on our Eventhouse via Capacity Planner (autoscale alone isn't sufficient - we need guaranteed baseline capacity to protect a large bulk-ingestion workload from destination-side OutOfMemory during a migration). We've been told the 32 CU setting has been applied on the backend, but the Fabric UI and REST API (.show cluster / .show diagnostics) still report capacity/behavior consistent with a much lower tier. Can anyone confirm: Whether the a custom minimum CU value is actually enforced server-side even though the UI/API don't show it, and When UI/API reporting is expected to catch up to reflect the configured minimum? Any insight or similar experience would be appreciated.
Solved
WorkFull22
1 month ago Place Eventhouse and KQL
Views
0likes
Comments
Fabric Eventhouse: Capacity policy for .ingest inline is hard-capped at 1 concurrent operation
Problem When using the KQL Activity in Fabric Data Pipelines to execute .ingest inline commands against an Eventhouse KQL database, the effective ingestion capacity is always 1 concurrent operation — regardless of the Fabric capacity SKU (tested on F2 and F8) and regardless of the cluster capacity policy setting. Running .alter-merge cluster policy capacity with ClusterMaximumConcurrentOperations: 16 succeeds without error, but .show capacity still reports an effective ingestion capacity of 1. Any pipeline pattern that executes .ingest inline within a parallel ForEach immediately hits 429 TooManyRequests / ControlCommandThrottledException errors. Highlighting that the capacity is 1. This makes the KQL Activity (KustoQueryLanguage type) unusable for parallel ingestion scenarios — which is an important use case in data pipelines (logging, event emission, watermark updates during parallel table processing). Question / Request Do you recognize this issue? if so I have the below request: Increase the effective ingestion capacity for Fabric Eventhouse to match what the capacity policy allows (respect ClusterMaximumConcurrentOperations)
Solved
hal-apeno91
2 months ago Place Eventhouse and KQL
Views
0likes
Comments
Copilot for EventHouse Queryset
Hi, RTI has introduced the preview feature of Copilot for the Eventhouse Queryset, but is there any way to automate the Copilot usage, via API endpoint or SDKs? The idea is to programmatically generate relevant and meaningful Kusto queries for business insights, from the tables and datasets present in the KQL DB in Eventhouse using Copilot. Thanks in advance!
sandeep022
2 months ago Place Eventhouse and KQL
Views
1like
Comments
Delay in Onelake Shortcut from Eventhouse to Lakehouse
I have a table in a KQL Database in an eventhouse that is hooked up to an Azure Eventhub. I have turned on One Lake storage for this table and added a shortcut to the lakehouse for this table. The table receives around 10-15k events per day and I have set the retention policy for the table to 30 days. The eventhouse table data was updating and getting reflected in the shortcut table within a few minutes when I first set it up. But now it takes almost an hour or more to see the latest event data in the shortcut table in my lakehouse. I turned the one lake availability off and then on again, and also tried updating the table policies but nothing seems to work
Solved
Shashwot
3 months ago Place Eventhouse and KQL
Views
1like
Comments
BCDR support for Fabric Eventhouse (KQL DB) – cross-region data replication and failover
Hi team, A few quick questions regarding BCDR options for Fabric Eventhouse (KQL): Does Fabric Eventhouse provide built-in cross-region redundancy/failover in case of regional outages? If yes, what is required to implement this, and what does the cost model look like? Does Fabric support native data replication/ copy capabilities? Specifically, is there support (or a roadmap) for: Azure Data Explorer (ADX/Kusto) → Fabric Eventhouse (KQL DB) Fabric Eventhouse (KQL DB) → Fabric Eventhouse (KQL DB) across regions (e.g., westcentralus ↔ westus) Could you please share any relevant documentation, known limitations, or guidance for these scenarios? Thanks in advance for your help!
Solved
egirma
4 months ago Place Eventhouse and KQL
Views
2likes
Comments
Is there a way to deactivate an Eventhouse/KQL DB?
Greetings, all. I have eventhouses and KQL DBs in Dev and Test workspaces that are part of a deployment pipeline. They generate capacity usage even though no data is flowing in and no one uses them. I want to deactivate or turn off the Dev/Test objects until we want to use them for code changes for deployments so Prod isn't broken. Is there a way to do this?
Solved
arpost
9 months ago Place Eventhouse and KQL
Views
0likes
Comments