kql queryset

39 Topics

Date slicer
Hi, I need some help with Power BI. I want to use a Between Date Slicer and display the dates in this format: DD-MMM-YYYY Example: 01-Jan-2020 I don’t want to use the default dropdown/date format. I specifically want the Between slicer with two date inputs (From Date and To Date), but I need the displayed dates to appear as 01-Jan-2020 instead of the default format. Could you please guide me on how to achieve this? If the standard Power BI slicer does not support this format, is there any custom visual or alternative solution that can provide the same Between Date functionality with the DD-MMM-YYYY format? Thanks!
Solved
pradipir
16 days ago Place Real-Time Intelligence
Views
1like
Comments
Fabric Eventstream MySQL CDC scans all databases and does not emit row-level events
Hello Fabric Team, We are testing the Microsoft Fabric Eventstream MySQL CDC connector with an AWS RDS MySQL database. The MySQL instance is accessible through a private IP, and Fabric connects through a VNet data gateway/private network connection. Connectivity is successful, and the connector remains Active without reporting processing errors. Our MySQL environment contains the same table structure across multiple databases, following a pattern similar to: application_<tenant>.sample_table We configured the source to capture only one fully qualified table: application_tenant1.sample_table The MySQL configuration is: log_bin = ON binlog_format = ROW binlog_row_image = FULL The connection user has SELECT, SHOW DATABASES, REPLICATION CLIENT, REPLICATION SLAVE/REPLICA, and the required snapshot permissions. Each Fabric source uses a unique Server ID. CDC from the same RDS instance has previously worked successfully through another CDC platform. However, we are seeing the following behavior: Although only one fully qualified table is selected, Fabric/Debezium scans schema metadata for every accessible database and table on the MySQL server. More than 45,000 schema events were generated across the tenant and history databases. The schema events contain DDL such as DROP TABLE IF EXISTS and CREATE TABLE. Snapshot mode is configured as Initial. The selected table contains approximately 16,000 records, but Fabric emitted zero snapshot r events. We committed a new insert into the selected table after schema discovery finished, but no c event was emitted. Eventhouse receives the schema events successfully, confirming that the source-to-Eventhouse path is operational. The Eventhouse destination stores the Debezium payload in a dynamic column using a mapper from payload to RawEvent. Could you please confirm the following? Is it expected for the Fabric-managed MySQL CDC connector to scan and publish schema events for every database when only one fully qualified table is selected? Does Fabric expose or support settings equivalent to: database.include.list table.include.list schema.history.internal.store.only.captured.databases.ddl schema.history.internal.store.only.captured.tables.ddl Why would the connector finish schema discovery but not emit existing rows from the selected table when Snapshot mode is Initial? Why are committed inserts not emitted after the schema phase appears to finish? Is there a known limitation involving AWS RDS MySQL accessed through a private IP and VNet data gateway? What is the recommended supported destination pattern for this CDC stream? Our main objective is reliable ongoing CDC capture. We are flexible about the landing destination and can use: Eventhouse ADLS Gen2 in JSON or Parquet Fabric Lakehouse/OneLake Snowflake as the final destination Please let us know which logs, connector identifiers, timestamps, or additional configuration details would help investigate this behavior. Thank you.
Solved
girishtharwani2
30 days ago Place Eventhouse and KQL
Views
0likes
Comments
Unable to create a Monitoring Eventhouse in a Microsoft Fabric workspace.
I am trying to enable Workspace Monitoring for a Microsoft Fabric workspace. The user has Fabric Administrator access and is also a Workspace Administrator. However, under: Workspace Settings → Monitoring the + Eventhouse option is disabled/not available. We have verified the tenant-level settings and confirmed that “Workspace admins can turn on monitoring for their workspaces” is enabled.
Solved
jonypanchal7
1 month ago Place Eventhouse and KQL
Views
2likes
Comments
Best practice for handling schema evolution in Fabric Eventstream before data reaches Eventhouse?
I have an Eventstream receiving operational events where the schema may evolve over time. For example, the producer initially sends: DeviceId, Timestamp, Temperature, Status but later adds fields such as: Location, FirmwareVersion, ErrorCode I want the pipeline to continue ingesting events without breaking downstream KQL tables, update policies, materialized views, or Real-Time Dashboards. I am trying to understand where schema evolution should ideally be handled in a production Fabric RTI architecture. Would you: enforce the contract upstream using Schema Registry normalize changing fields inside Eventstream before Eventhouse ingestion land the raw payload first and handle schema evolution inside Eventhouse/KQL maintain separate versioned event schemas/tables How are people handling this in production when producers can add fields without notice? I am particularly interested in avoiding a design where every small upstream schema change forces updates across Eventstream, KQL tables, update policies, and downstream dashboards.
Solved
ShivekMaharaj
2 months ago Place Real-Time Intelligence
Views
0likes
Comments
Copilot for EventHouse Queryset
Hi, RTI has introduced the preview feature of Copilot for the Eventhouse Queryset, but is there any way to automate the Copilot usage, via API endpoint or SDKs? The idea is to programmatically generate relevant and meaningful Kusto queries for business insights, from the tables and datasets present in the KQL DB in Eventhouse using Copilot. Thanks in advance!
sandeep022
2 months ago Place Eventhouse and KQL
Views
1like
Comments
Error in Microsoft Applied Skills: Implement a Real-Time Intelligence Assessment Lab : Need help
Hello All, since a week I was stuck on one error in Microsoft Applied Skills: Implement a Real-Time Intelligence solution with Microsoft Fabric Lab. Did anyone face the same issue if so, can anyone help me out how to resolve it. Attaching the error and related images below. Error: It says " unable to load container blobs. Verify that workspace identity is enabled and has storage blob data reader access to the storage account" . I am familiar with this permission and have enabled it in my own azure environment for my own blob that I have created but since these are pre created by Microsoft, I don't have access to these storage blobs and hence I am not sure how to proceed here. I have attempted this twice but I was stuck at the same point. I really appreciate your help and taking time in looking into my post. Thank you so much.
Solved
AshikaAlvakonda
5 months ago Place Real-Time Intelligence
Views
0likes
Comments
Issue Writing to OneLake-Enabled KQL Database Delta Tables
I enabled OneLake availability for my KQL database so the tables are exposed as Delta or Parquet files. I’m able to read the table using Parquet without any issues: df = spark.read.format("parquet").load(abfsspath) However, when I try to overwrite or append data to the table, it fails: append_df.write.format("delta").mode("append").save(abfsspath) I receive the following error: Operation failed: "Forbidden", 403, AuthorizationPermissionMismatch "This request is not authorized to perform this operation using this permission." I am the workspace admin, and I don’t want to ingest data using the conventional KQL ingestion methods because they are too slow for my use case. I specifically need to write records directly using Delta or Parquet. Could you help me understand how to enable this write operation for Delta tables?
Solved
Dinzz
8 months ago Place Real-Time Intelligence
Views
0likes
Comments
Architecture & GPS hardware
Hello Fabric community, I'm building a fleet monitoring platform for trucks/vehicles and need architectural guidance for using Microsoft Fabric Real-Time Intelligence, as well as GPS hardware recommendations. MY SCENARIO: ------------ - GPS trackers sending location data every 10-60 seconds via TCP - Data includes: GPS coordinates, speed, fuel level, engine RPM, temperature sensors, driver behavior (harsh braking, acceleration) - Need to support multiple tenants (each customer sees only their own fleet) - Planning to scale from 50 vehicles to 1000+ vehicles - Want real-time dashboards + historical analytics + automated alerts - Future: add video/dashcam capabilities MY PLANNED ARCHITECTURE: ------------------------ GPS Devices → TCP Server (Traccar/custom) → Azure Event Hub → Fabric Eventstream → Eventhouse → Real-Time Dashboard QUESTIONS: GPS Hardware: ------------- 1. What GPS tracker brands/models do you recommend that integrate well with custom backends and Microsoft Fabric? I'm considering Teltonika FMC920, Queclink, or Concox - any experience with these? 2. Which GPS devices have the best open protocols for parsing data and sending to Azure/Fabric? 3. Any recommendations for GPS trackers with OBD-II support to read vehicle diagnostics (fuel, RPM, error codes)? Microsoft Fabric Architecture: ------------------------------ 4. Is Eventstream + Eventhouse the right approach for high-frequency GPS telemetry data? 5. What's the recommended way to implement multi-tenancy (row-level security) for different fleet customers? 6. Should I use KQL Database or Lakehouse for storing historical GPS data (months/years)? 7. Any recommendations for handling geofencing alerts in real-time using Activator? 8. What Fabric SKU (F4, F8, F16?) would you recommend starting with for ~100 vehicles sending data every 30 seconds? I'm based in Mexico and planning to deploy this commercially for fleet owners. Any guidance from people who have built similar IoT/telematics solutions with Fabric would be greatly appreciated! Thanks in advance!
Solved
jhrjdata
8 months ago Place Real-Time Intelligence
Views
0likes
Comments
Is there a way to deactivate an Eventhouse/KQL DB?
Greetings, all. I have eventhouses and KQL DBs in Dev and Test workspaces that are part of a deployment pipeline. They generate capacity usage even though no data is flowing in and no one uses them. I want to deactivate or turn off the Dev/Test objects until we want to use them for code changes for deployments so Prod isn't broken. Is there a way to do this?
Solved
arpost
9 months ago Place Eventhouse and KQL
Views
0likes
Comments
Issue creating a KQL materialized view
Hi, I'm testing an eventstream that reads source events from an Azure Event Hubs and write them into a KQL table. I've created successfully a KQL materialized view (using summarize and arg_max clauses) with the backfill=true option, without specifying the async clause, while the eventstream was active. After two days, for debug purposes, I've tried to create a parallel materialized view with a different name using the same KQL code and I've obtained this error: But firts it isn't required the async keyword. So, I've tried to create an async materialized view using .create async materialized-view with (backfill=true) mymaterview on table mytable an OperationId is shown but it doesn't appear any materialized views. Now, I don't understand because first it isn't necessary to specify the async keyword and then I need to write it. Then, I don't understand because an async materialized view doesn't appear. I don't understand because I cannot create a parallel clone materialized view with a different name. I hope that a materialized view, created with or without the back fill option, keep updating with new source event data without executing any alter action on it. Any helps to me, please? Many thanks
Solved
pmscorca
9 months ago Place Eventhouse and KQL
Views
0likes
Comments