When working with raw JSON data in Eventhouse, one common challenge is comparing two records to identify what changed—especially when the structure isn’t fixed. Fields may appear, disappear, or shift...
Published 1 year ago
Version 1.0
Blog Post
Adding another neat little easy way to compare using set_difference function in KQL
This KQL does a similar approach of extracting each key value pair to do comparison and then uses make_set and set_difference.
let device_mappings = datatable(col1:dynamic)
[dynamic({"displayName":"conditional","oldvalue":{"id":"123","Name":"Surya","dept":"maths"},"newvalue":{"id":"123","Name":"teja","dept":"science"}})];
device_mappings
| where col1.displayName=="conditional"
| extend oldvalue=col1.oldvalue, newvalue=col1.newvalue
| mv-expand oldvalue, newvalue // to explode old and new values to multiple records
| summarize make_set(oldvalue), make_set(newvalue) // for use in set_difference
| project newValues= set_difference(set_newvalue,set_oldvalue) // compares new and old values and gets the New/changed key value pair