kql queryset
32 TopicsCopilot for EventHouse Queryset
Hi, RTI has introduced the preview feature of Copilot for the Eventhouse Queryset, but is there any way to automate the Copilot usage, via API endpoint or SDKs? The idea is to programmatically generate relevant and meaningful Kusto queries for business insights, from the tables and datasets present in the KQL DB in Eventhouse using Copilot. Thanks in advance!6.7KViews1like7CommentsIs there a way to deactivate an Eventhouse/KQL DB?
Greetings, all. I have eventhouses and KQL DBs in Dev and Test workspaces that are part of a deployment pipeline. They generate capacity usage even though no data is flowing in and no one uses them. I want to deactivate or turn off the Dev/Test objects until we want to use them for code changes for deployments so Prod isn't broken. Is there a way to do this?Solved9.8KViews0likes4CommentsIssue creating a KQL materialized view
Hi, I'm testing an eventstream that reads source events from an Azure Event Hubs and write them into a KQL table. I've created successfully a KQL materialized view (using summarize and arg_max clauses) with the backfill=true option, without specifying the async clause, while the eventstream was active. After two days, for debug purposes, I've tried to create a parallel materialized view with a different name using the same KQL code and I've obtained this error: But firts it isn't required the async keyword. So, I've tried to create an async materialized view using .create async materialized-view with (backfill=true) mymaterview on table mytable an OperationId is shown but it doesn't appear any materialized views. Now, I don't understand because first it isn't necessary to specify the async keyword and then I need to write it. Then, I don't understand because an async materialized view doesn't appear. I don't understand because I cannot create a parallel clone materialized view with a different name. I hope that a materialized view, created with or without the back fill option, keep updating with new source event data without executing any alter action on it. Any helps to me, please? Many thanksSolved10KViews0likes2CommentsT-SQL executing directly in Eventhouse KQL Queryset without comment (--) prefix
Hey everyone, I was just exploring Eventhouse and noticed something weird. The docs say you need to add "--" on its own line before T-SQL queries in KQL querysets, but I'm running T-SQL commands just fine without it. I never added the "--" prefix and all T-SQL queries executed normally. Is this a recent change? The documentation still mentions needing the comment prefix to switch to T-SQL mode, but it seems like the query editor is just auto-detecting it now? Reference: https://learn.microsoft.com/en-us/kusto/query/t-sql?view=microsoft-fabricSolved3.9KViews0likes7Commentskusto error- partial query failure
hi all, error - Details: "Query execution has exceeded the allowed limits (80DA0001): Partial query failure: Runaway query (E_RUNAWAY_QUERY). (message: The Join output block has exceeded the memory budget during evaluation. Results may be incorrect or incomplete (E_RUNAWAY_QUERY; see https://aka.ms/kustoquerylimits).: ). [0]Kusto.Data.Exceptions.KustoServicePartialQueryFailureLimitsExceededException: Query execution has exceeded the allowed limits (80DA0001): Partial query failure: Runaway I am trying to join 5 tables in kusto to fetch all required columns , 3 tables from 1 cluster ans 2 from another cluster. But query is failing with memory error Dummy query : let onboarded = Pln | project STId; let regAss = external_table('PR') | where Region == 'France' | where Status !in ("Error", "Rejected") | project STId; let exeAppr = external_table('PR') | where Region == "France" and Status in ("Ready", "Approved") | project STId, Region, Status; let QReferenced= external_table('PQR') | where Region == 'France' and Status == "Completed" | project STId, Region, Status,RequestId; let Regionpsllls = cluster("cp.ws.kusto.windows.net").database("pslll").pslll | where State == "Active" and pslllType == "Region" | project pslllId, pslllName = Name, pslll_Reg = RegionName; let pslllServices = cluster("cp.ws.kusto.windows.net").database("pslll").PFL | where ProductType == "Service" | project pslllId, SOId; let pslll_Reg_Serv = Regionpsllls | join kind=inner pslllServices on pslllId | project pslll_Reg, pslllName, SOId; // Join onboarded with pslll let Onboardedpslll = onboarded | join kind=inner pslll_Reg_Serv on $left.STId == $right.SOId; // Join region assigned with pslll let regAsspslll = regAss | join kind=inner pslll_Reg_Serv on $left.STId == $right.SOId; // Join execution approved with pslll let exeApprpslll = exeAppr | join kind=inner pslll_Reg_Serv on ($left.STId == $right.SOId and $left.Region == $right.pslll_Reg); //join quota refernced with pslll let QReferencedpslll = QReferenced | join kind=inner pslll_Reg_Serv on ($left.STId == $right.SOId and $left.Region == $right.pslll_Reg); // Combine using inner joins (only psllls present in all three sets) Onboardedpslll | join kind=inner regAsspslll on pslll_Reg, pslllName | join kind=inner exeApprpslll on pslll_Reg, pslllName | join kind=inner QReferencedpslll on pslll_Reg, pslllName | project pslll_Reg, pslllName ( i want many more cols to project)Solved7.8KViews0likes4CommentsPartitioning policy is not copied to DatabaseSchema.kql
I have added a custom partitioning policy to some materialized views and tables like this: .alter materialized-view DimResponse policy partitioning "{\"PartitionKeys\":[{\"ColumnName\":\"OrgId\",\"Kind\":\"Hash\",\"Properties\":{\"Function\":\"XxHash64\",\"MaxPartitionCount\":128,\"Seed\":1,\"PartitionAssignmentMode\":\"Uniform\"}}],\"EffectiveDateTime\":\"2023-01-01T00:00:00.000000Z\",\"MinRowCountPerOperation\":0,\"MaxRowCountPerOperation\":0,\"MaxOriginalSizePerOperation\":0}" I have connected to source control, but in DatabaseSchema.kql, the policy is not in the schema, is this a limitation or a bug? My update policies are applied in the DatabaseSchema.kql, so something is working when altering materialized views and tables.Solved3.2KViews0likes1CommentKQL json mapping from Date Pipeline
Hi everyone. I’m running into an issue when ingesting data from MongoDB into my Kusto database using Azure Data Factory’s Copy Activity. Two of the fields in my source collection are Unix timestamps, but when they arrive in Kusto they end up stored as long instead of datetime. To fix this, I created a JSON ingestion mapping on my Kusto table that should convert those two timestamp columns into proper datetimes during ingestion. However, when I specify that mapping in the Copy Activity’s Sink settings, I get the following error: ErrorCode=KustoMappingReferenceHasWrongKind Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException Message=Mapping reference should be of kind 'Csv'. Mapping reference: 'UnixToDateTimeMapping'. Kind 'Json'. Source=Microsoft.DataTransfer.Runtime.KustoConnector I’ve tried passing the mapping name and adding various sink properties, but I can’t get ADF to apply my JSON mapping. Has anyone successfully configured a Copy Activity to use a Kusto JSON ingestion mapping? The code of my mapping as well .create-or-alter table info_connections_teste ingestion json mapping "UnixToDateTimeMapping" '[' ' { "column":"_id", "datatype":"string", "Properties":{"Path":"$._id"} },' ' { "column":"latency", "datatype":"real", "Properties":{"Path":"$.latency"} },' ' { "column":"packetloss_percentage","datatype":"int", "Properties":{"Path":"$.packetloss_percentage"} },' ' { "column":"download_occupancy","datatype":"real", "Properties":{"Path":"$.download_occupancy"} },' ' { "column":"upload_occupancy", "datatype":"real", "Properties":{"Path":"$.upload_occupancy"} },' ' { "column":"connection_id", "datatype":"string", "Properties":{"Path":"$.connection_id"} },' ' { "column":"createdAt", "datatype":"long", "Properties":{"Path":"$.createdAt"} },' ' { "column":"createdAtDate", "datatype":"datetime", "Properties":{"Path":"$.createdAt","Transform":"DateTimeFromUnixSeconds"} },' ' { "column":"updatedAt", "datatype":"long", "Properties":{"Path":"$.updatedAt"} },' ' { "column":"updatedAtDate", "datatype":"datetime", "Properties":{"Path":"$.updatedAt","Transform":"DateTimeFromUnixSeconds"} },' ' { "column":"__v", "datatype":"int", "Properties":{"Path":"$.__v"} }' ']'Solved9.5KViews0likes4CommentsHow can I split a variable-depth, slash-delimited path_column field into separate col in KQL?
Q : How can I split a variable-depth, slash-delimited path_column field into separate path_lvl_1, path_lvl_2, … columns in Kusto (Fabric) so that Power BI (in DirectQuery mode) can consume a fixed schema without requiring manual updates when the number of segments grows? Details: My raw table is named data_table and has this schema: id_column : string device_column : string event_name : string data_type : string unit_column : string path_column : string // e.g. "A/B/C/D" or "X/Y" or "Region/Zone/Section/Shelf/…" value_column : string timestamp_column : datetime static_flag : bool message_type : string path_column can have an arbitrary number of “/” segments—today might be 3 levels, tomorrow 10 or 50. We need Power BI (in DirectQuery) to see columns like: path_lvl_1, path_lvl_2, path_lvl_3, …, path_lvl_N where N grows automatically whenever any row’s path_column has more segments than before. Approaches tried and limitations: Split to a dynamic array (extend segments_array = split(path_column, "/")) then client-side “List → Record → Expand.” Works in Import mode, but breaks in DirectQuery because the M step (using List.Count, Record.FromList, Expand) cannot fold. Split into a fixed number of columns (e.g. assume max of 10 levels via path_lvl_1 = iff(array_length(parts) >= 1, parts[0], ""), …, path_lvl_10 = iff(array_length(parts) >= 10, parts[9], "")). Works in DirectQuery, but if a row ever has 11 segments, you lose data beyond level 10—and must manually alter the function/table to add path_lvl_11. mv-expand + evaluate pivot at query time for a truly dynamic set of columns. KQL function returns a result with as many columns as needed (0..max index), but the schema is not stable, so Power BI (DirectQuery) cannot bind to it. What I need: A way in Fabric/Kusto to produce a “wide” result that always has a fixed schema of path_lvl_1…path_lvl_N, automatically increasing N whenever deeper paths appear—without having to manually ALTER the table or function each time the maximum depth increases. Ideally, a pure KQL solution (since DirectQuery won’t let me do client-side M transformations), or some Fabric feature I’m missing (update policies, etc.) that can “auto-add” new columns. Why this matters: We want the Fabric → Power BI pipeline to be maintenance-free. If tomorrow a device reports path_column = "A/B/C/D/E/F/G/H" the next Power BI refresh should automatically see path_lvl_7 and path_lvl_8 without a manual “.alter table” or “re-publish function.” Actually I need these different columns to create hierarchy in one of power bi report. so that the dynamic schema can be handled properly. or if you can provide a solution to handle it at etl level, that will also work, like may be implementing any function or script so that if each time no of values getting inccreased in path_column, then automatically functions or script run and creates same table with new schema with updated policy. Has anyone in the Fabric community solved this? Is there a recommended pattern to handle truly variable-depth slash-delimited hierarchies in KQL so that downstream Power BI (DirectQuery) always sees a stable, “growing” set of path_lvl_X columns?Solved7.2KViews0likes9CommentsI want to achieve a chart in real time dashboard where metrics and chart both should there
Hi Team I am trying to create a chart where i want keep the metrics and chart both in a single tile, is it possible or not if yes can you help me through it, i am trying with plotly also but got nothing till now attaching sc of requirement and slo one more thing there is a chart where 3 stocks live data is coming , so i want the each graph to be separated by one other like same in above sc instead of giving the combined chart togetherSolved5.7KViews0likes5CommentsUrgent question about Policy update
Hi, I'm trying to implement the medallion architecture in RT, so I was thinking I'd have one KQL DB for bronze, one for silver, and one for gold. The question is how I can adjust the update policy to bring the bronze transformed data to the silver table that is in its own DB. Currently it did so as follows: .alter table SilverDataSensores policy update @'[{"IsEnabled": true, "Source": "Bronce2DataSensores", "Query": "UpdateGoldTable()", "IsTransactional": true}]' So as I mention I want to have a different KQL DB for each layer, how can I reference that SilverDataSensors is in another DB, or in its defense that Soruce and Query are from the bronze DB? Thanks 😄Solved10KViews0likes4Comments