Forum Discussion
Calculate Equipment State Duration per Shift
- Anonymous6 years ago
here a solution "complete" (corresponding to the status of the information received 😁) avoiding the use of the function List.Accumulate.
starting table:
output Table:
the code for the output table:
let ct = Table.TransformColumnTypes(Table,{{"StateDateTime", type datetime}, {"EquipmentState", type text}, {"ShiftIndex", Int64.Type}}), idx=List.RemoveLastN(List.Distinct(ct[ShiftIndex]),1), lstR=List.Generate( ()=>[r=1,pos=List.PositionOf(ct[ShiftIndex],idx{0},Occurrence.Last), lr=sfht(ct{pos},ct{pos+1})], each [r]<=List.Count(idx), each [r=[r]+1, pos=List.PositionOf(ct[ShiftIndex],idx{[r]},Occurrence.Last),lr=sfht(ct{pos},ct{pos+1}) ], each [lr] ), tc= Table.Combine({Table.FromRecords(List.Combine(lstR)),Table}), grp = Table.Group(tc, {"ShiftIndex"}, {{"shIdx", each duration(_)}}), #"Expanded shIdx" = Table.ExpandTableColumn(grp, "shIdx", {"StateDateTime", "EquipmentState", "Duration"}, {"StateDateTime", "EquipmentState", "Duration"}) in #"Expanded shIdx"the two functions used:
sfht
let listRows=(rL,rF)=> let shift=#duration(0,0,720,0), dL=rL[StateDateTime], dF=rF[StateDateTime], lr=List.Generate( ()=>[r=rL&[StateDateTime=#datetime(Date.Year(dL),Date.Month(dL),Date.Day(dL)+sh{1},sh{0},0,0)],idx=1], each [r][StateDateTime]<=dF, each [r=if Number.Mod(idx,2)=1 then [r]& [StateDateTime=[r][StateDateTime]+shift] else [r]&[ShiftIndex=[r][ShiftIndex]+1],idx=[idx]+1], each [r] ), sh=if Time.Hour(dL) < 6 then {6,0} else if Time.Hour(dL)<18 then {18,0} else {6,1} in lr in listRowsand finally duration:
let count=(tab)=> let ts=Table.Sort(tab,{"StateDateTime"}), ai = Table.AddIndexColumn(ts, "i", 0, 1) in Table.RemoveColumns(Table.AddColumn(ai, "Duration", each try if [ShiftIndex]=ai[ShiftIndex]{[i]+1} then Duration.TotalMinutes(ai[StateDateTime]{[i]+1}-[StateDateTime]) else "" otherwise""),{"i"}) in count
DYd you try the code on your real dataset?
How many rows you table has?
If you explain all relevant details of you problem, you have chance to get more spwcific and usefull answer.
If needs; I can modify the code to manage big dataset.
Thanks Anonymous ,
I ran the code against a sub-set of the actual Dataset (limited to 2000 rows) - in xlsx format and it took approx 20min to run... Most of the results seem ok, except if we have a case where we log an Event before the close of a shift, and only log another event 2 shifts later... Example we log a State at 16:00:00 (DayShift or ShiftIndex =4 for example) and the next State is only logged at 08:00:00 the following day (DayShift or ShiftIndex = 6 for example) - then the code does not make an entry at 18:00:00 - so we have not entry for ShiftIndex = 5.
The current complete Dataset contains approx 5000 rows, and would grow by roughly 2000 entires each month.
I'm not sure how I get you a copy of the actual dataset, but happy to share.
- Anonymous6 years agoNot applicable
Hi EricSteynMMD
my suggestions:
"I ran the code against a sub-set of the actual Dataset (limited to 2000 rows) - in xlsx format and it took approx 20min to run..."
If you give some other info (for example other columns) we could use this to partion (by month, by some id or some other varaible) the problem in many little sub-problem and gain efficiency.
"Most of the results seem ok, except if we have a case where we log an Event before the close of a shift, and only log another event 2 shifts later... Example we log a State at 16:00:00 (DayShift or ShiftIndex =4 for example) and the next State is only logged at 08:00:00 the following day (DayShift or ShiftIndex = 6 for example) - then the code does not make an entry at 18:00:00 - so we have not entry for ShiftIndex = 5."
I'm not sure to understand what "log an event" really means. Could you give specific use case and expected result.
"The current complete Dataset contains approx 5000 rows, and would grow by roughly 2000 entires each month.
I'm not sure how I get you a copy of the actual dataset, but happy to share."
you could share the structure of your dataset eventually using dummy values, like e1, e2, ... e45, e46 for event, similarly for state change s1, s2, ... and so on, giving all the different cases and the real dimension of your dataset.
You could create a dummy dataset as done by LaurentZ here
Peraphs an approach similar to the one used for this daset is repeatable and effective also in your case, even though LaurentZ has not yet stated that it had the solution