Forum Discussion
tekulka
6 years agoNew Member
Need to implement incremental refresh where DataSource is kusto
Hi Team, So I want to implement incremental refresh for power BI premium. This is because we have around 300M+ records in our telemetry and we need to fetch it all in order to generate reports. N...
- 6 years ago
Kusto does not emit the native query. It does not mean it is not foldable. Add the step to see if it is folding:
Value.Metadata(PreviousStep)[QueryFolding]
Enable diagnonsis and I think you should be able to find the native kusto query being run.
Not sure if this helps, but this is what I'm using
let FormatArgument = (arg as any) => if arg is date then "datetime(" & Date.ToText(arg, "yyyy-MM-dd") & ")" else if arg is datetime then "datetime(" & DateTime.ToText(arg, "yyyy-MM-dd HH:mm:ss") & ")" else if arg is datetimezone then "datetime(" & DateTime.ToText(DateTimeZone.RemoveZone(DateTimeZone.ToUtc(arg)), "yyyy-MM-dd HH:mm:ss") & ")" else if arg is duration then "time(" & Duration.ToText(arg) & ")" else if arg = null then error "null value passed to function" else Text.From(arg) in FormatArgumentlet KustoFn = (fn as text, args as list, options as nullable record) => Kusto.Contents("yourcluster", "yourdb", fn & "(" & Text.Combine(List.Transform(args, #"FormatKustoArgs"), "," ) & ")" , options) in KustoFnLet me know if you see any bugs with using the kusto connector, I do have contributer access to this.
artemus
6 years agoMicrosoft Employee
Kusto does not emit the native query. It does not mean it is not foldable. Add the step to see if it is folding:
Value.Metadata(PreviousStep)[QueryFolding]
Enable diagnonsis and I think you should be able to find the native kusto query being run.
Not sure if this helps, but this is what I'm using
let
FormatArgument = (arg as any) =>
if arg is date then
"datetime(" & Date.ToText(arg, "yyyy-MM-dd") & ")"
else if arg is datetime then
"datetime(" & DateTime.ToText(arg, "yyyy-MM-dd HH:mm:ss") & ")"
else if arg is datetimezone then
"datetime(" & DateTime.ToText(DateTimeZone.RemoveZone(DateTimeZone.ToUtc(arg)), "yyyy-MM-dd HH:mm:ss") & ")"
else if arg is duration then
"time(" & Duration.ToText(arg) & ")"
else if arg = null then
error "null value passed to function"
else
Text.From(arg)
in
FormatArgumentlet
KustoFn = (fn as text, args as list, options as nullable record) =>
Kusto.Contents("yourcluster", "yourdb",
fn & "(" & Text.Combine(List.Transform(args, #"FormatKustoArgs"), "," ) & ")"
, options)
in
KustoFnLet me know if you see any bugs with using the kusto connector, I do have contributer access to this.