Hi All,
I was doing some testing with excution times of various M code solutions, and that proved more difficult than I first imaged.
The main reason is that M code is not excuted in the order of the statements, but is evaluated lazily. Just in time so to speak.
I cam accross a number of 'obvious' solutions that did not work (or were very complicated), so I created my own.
The main function is this:
Benchmark.Time = (f as function) => // making 1 call to the function, returning the number of seconds it took
let
f = () => Logical.From(f()), // to cast the result to a logical
start_time = DateTime.LocalNow(), // not evaluated yet!!
end_time = DateTime.LocalNow(), // not evaluated yet!!
duration =
if start_time <> null then // start_time evaluates here
if f() ?? true <> null then // function is called here and never returns null
(end_time - start_time) // so, this is always executed and evaluates the end_time to calculate the duration
else null // never executed
else null // never executed
in
Duration.TotalSeconds(duration) // in seconds as a floating point number
Note the hoops I go through to make sure start_time and end_time are evaluated at the right moment...
The full solution includes running many samples of a list of functions and producing some basix statistics:
// Function.Benchmark Compare produces some basic statistics comparing 2 benchmark samples
(functionList as list, sampleSize as number) => // a list of functions and the number of times to call each function
let
Benchmark.Time = (f as function) => // making 1 call to the function, returning the number of seconds it took
let
f = () => Logical.From(f()), // to cast the result to a logical
start_time = DateTime.LocalNow(), // not evaluated yet!!
end_time = DateTime.LocalNow(), // not evaluated yet!!
duration =
if start_time <> null then // start_time evaluates here
if f() ?? true <> null then // function is called here and never returns null
(end_time - start_time) // so, this is always executed and evaluates the end_time to calculate the duration
else null // never executed
else null // never executed
in
Duration.TotalSeconds(duration) // in seconds as a floating point number
,
Benchmark.Sample = (f as function) => // repeatedly call, and time, the function
List.Generate(
() => 0, // from 0
each _ < sampleSize, // to sampleSize -1 (taken from the enlosing function)
each _ + 1, // by 1
each Benchmark.Time(f)), // call and time
Benchmark.Statistics = (runTimes as list) => // calculate some basic statistics and return them as a record
[
Average = List.Average(runTimes),
StandardDeviation = List.StandardDeviation(runTimes),
Median = List.Median(runTimes),
Percentile25 = List.Percentile(runTimes, 0.25),
Percentile75 = List.Percentile(runTimes, 0.75)
],
results = List.Accumulate( // loop through the list of functions
functionList,
{}, // start with an empty list
(state,current) =>
state & {Benchmark.Statistics(Benchmark.Sample(current))}) // take the sample and append the statistics
in
results // a list of records
Enjoy!
