Forum Discussion
detecting overlapping activities with time and date
- 2 years ago
Hi Joaomatos2002, check this:
Comment: there is a mistake in your sample data (first row). There should be [Inicio] 2024-07-31 23:00:00 in my opinion.
Remove these 3 steps if you don't need same row order:
Result
v1
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("ldBRCoAgEATQq8h+K+zOWlZXEe9/jUohyrYokP0YhsdgziTkSXi/YMTAKag46MK8vSOdAosTNlIqPhMqgmudZxOx0oq0JdrVYSLWvtOS+AmZ3pHhNlzvCB7/RCsydkj8t6QhqasnC4GYSFkB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [ID_Card = _t, #"ID_Act Inicio" = _t, Inicio = _t, Fim = _t, DATe = _t]), ChangedType = Table.TransformColumnTypes(Source,{{"ID_Card", Int64.Type}, {"ID_Act Inicio", Int64.Type}, {"Inicio", type datetime}, {"Fim", type datetime}, {"DATe", type date}}), AddedIndex = Table.AddIndexColumn(ChangedType, "Index", 0, 1, Int64.Type), Fn_CheckOverlaping = (myTable as table)=> [ // _Detail = GroupedRows{[ID_Card=2]}[All], _Detail = myTable, _AddedIndex = Table.AddIndexColumn(_Detail, "IndexHelper", 0, 1, Int64.Type), _Zipped = List.Zip({ _AddedIndex[Inicio], _AddedIndex[Fim] }), _StepBack = _AddedIndex, _Ad_CheckOverlaping = Table.AddColumn(_StepBack, "Check Overlapping", each [ a = List.Buffer(List.RemoveRange(_Zipped, [IndexHelper])), //Removed current row from _Zipped b = List.AnyTrue(List.Transform(a, (x)=> if [Inicio] >= x{0} and [Inicio] < x{1} then true //Inicio check else if [Fim] > x{0} and [Fim] <= x{1} then true //Fim check else if [Inicio] < x{0} and [Fim] > x{1} then true //Outside Check else false) ) ][b], type logical), _RemovedColumns = Table.RemoveColumns(_Ad_CheckOverlaping, {"IndexHelper"}) ][_RemovedColumns], GroupedRows = Table.Group(AddedIndex, {"ID_Card"}, {{"Fn", Fn_CheckOverlaping, type table}}), CombinedFn = Table.Combine(GroupedRows[Fn]), SortedRows = Table.Sort(CombinedFn,{{"Index", Order.Ascending}}), RemovedIndex = Table.RemoveColumns(SortedRows,{"Index"}, MissingField.Ignore) in RemovedIndexv2 (this one should be faster)
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("ldBRCoAgEATQq8h+K+zOWlZXEe9/jUohyrYokP0YhsdgziTkSXi/YMTAKag46MK8vSOdAosTNlIqPhMqgmudZxOx0oq0JdrVYSLWvtOS+AmZ3pHhNlzvCB7/RCsydkj8t6QhqasnC4GYSFkB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [ID_Card = _t, #"ID_Act Inicio" = _t, Inicio = _t, Fim = _t, DATe = _t]), ChangedType = Table.TransformColumnTypes(Source,{{"ID_Card", Int64.Type}, {"ID_Act Inicio", Int64.Type}, {"Inicio", type datetime}, {"Fim", type datetime}, {"DATe", type date}}), AddedIndexHelper = Table.AddIndexColumn(ChangedType, "IndexHelper", 0, 1, Int64.Type), fn_CheckOverlapping = (myTable as table)=> [ // _Detail = GroupedRows{[ID_Card=2]}[Fn], _Detail = myTable, _Ad_DateTimes = Table.AddColumn(_Detail, "DateTimes", each { [IndexHelper], List.DateTimes([Inicio], Duration.TotalMinutes([Fim]-[Inicio])+1, #duration(0,0,1,0)) }, type list), _DateTimes = _Ad_DateTimes[DateTimes], _StepBack = _Ad_DateTimes, _Ad_CheckOverlapping = Table.AddColumn(_StepBack, "Check Overlapping", each [ a = List.Select(_DateTimes, (x)=> x{0} <> [IndexHelper]), b = List.Combine(List.Transform(a, (x)=> x{1})), c = if List.ContainsAny(b, [DateTimes]{1}) then true else false ][c], type logical), _RemovedColumns = Table.RemoveColumns(_Ad_CheckOverlapping,{"IndexHelper", "DateTimes"}) ][_RemovedColumns], GroupedRows = Table.Group(AddedIndexHelper, {"ID_Card"}, {{"Fn", fn_CheckOverlapping, type table}}), CombinedFn = Table.Combine(GroupedRows[Fn]) in CombinedFn
Hi Joaomatos2002, check this:
Comment: there is a mistake in your sample data (first row). There should be [Inicio] 2024-07-31 23:00:00 in my opinion.
Remove these 3 steps if you don't need same row order:
Result
v1
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("ldBRCoAgEATQq8h+K+zOWlZXEe9/jUohyrYokP0YhsdgziTkSXi/YMTAKag46MK8vSOdAosTNlIqPhMqgmudZxOx0oq0JdrVYSLWvtOS+AmZ3pHhNlzvCB7/RCsydkj8t6QhqasnC4GYSFkB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [ID_Card = _t, #"ID_Act Inicio" = _t, Inicio = _t, Fim = _t, DATe = _t]),
ChangedType = Table.TransformColumnTypes(Source,{{"ID_Card", Int64.Type}, {"ID_Act Inicio", Int64.Type}, {"Inicio", type datetime}, {"Fim", type datetime}, {"DATe", type date}}),
AddedIndex = Table.AddIndexColumn(ChangedType, "Index", 0, 1, Int64.Type),
Fn_CheckOverlaping =
(myTable as table)=>
[
// _Detail = GroupedRows{[ID_Card=2]}[All],
_Detail = myTable,
_AddedIndex = Table.AddIndexColumn(_Detail, "IndexHelper", 0, 1, Int64.Type),
_Zipped = List.Zip({ _AddedIndex[Inicio], _AddedIndex[Fim] }),
_StepBack = _AddedIndex,
_Ad_CheckOverlaping = Table.AddColumn(_StepBack, "Check Overlapping", each
[ a = List.Buffer(List.RemoveRange(_Zipped, [IndexHelper])), //Removed current row from _Zipped
b = List.AnyTrue(List.Transform(a, (x)=>
if [Inicio] >= x{0} and [Inicio] < x{1} then true //Inicio check
else if [Fim] > x{0} and [Fim] <= x{1} then true //Fim check
else if [Inicio] < x{0} and [Fim] > x{1} then true //Outside Check
else false) )
][b], type logical),
_RemovedColumns = Table.RemoveColumns(_Ad_CheckOverlaping, {"IndexHelper"})
][_RemovedColumns],
GroupedRows = Table.Group(AddedIndex, {"ID_Card"}, {{"Fn", Fn_CheckOverlaping, type table}}),
CombinedFn = Table.Combine(GroupedRows[Fn]),
SortedRows = Table.Sort(CombinedFn,{{"Index", Order.Ascending}}),
RemovedIndex = Table.RemoveColumns(SortedRows,{"Index"}, MissingField.Ignore)
in
RemovedIndex
v2 (this one should be faster)
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("ldBRCoAgEATQq8h+K+zOWlZXEe9/jUohyrYokP0YhsdgziTkSXi/YMTAKag46MK8vSOdAosTNlIqPhMqgmudZxOx0oq0JdrVYSLWvtOS+AmZ3pHhNlzvCB7/RCsydkj8t6QhqasnC4GYSFkB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [ID_Card = _t, #"ID_Act Inicio" = _t, Inicio = _t, Fim = _t, DATe = _t]),
ChangedType = Table.TransformColumnTypes(Source,{{"ID_Card", Int64.Type}, {"ID_Act Inicio", Int64.Type}, {"Inicio", type datetime}, {"Fim", type datetime}, {"DATe", type date}}),
AddedIndexHelper = Table.AddIndexColumn(ChangedType, "IndexHelper", 0, 1, Int64.Type),
fn_CheckOverlapping =
(myTable as table)=>
[
// _Detail = GroupedRows{[ID_Card=2]}[Fn],
_Detail = myTable,
_Ad_DateTimes = Table.AddColumn(_Detail, "DateTimes", each { [IndexHelper], List.DateTimes([Inicio], Duration.TotalMinutes([Fim]-[Inicio])+1, #duration(0,0,1,0)) }, type list),
_DateTimes = _Ad_DateTimes[DateTimes],
_StepBack = _Ad_DateTimes,
_Ad_CheckOverlapping = Table.AddColumn(_StepBack, "Check Overlapping", each
[ a = List.Select(_DateTimes, (x)=> x{0} <> [IndexHelper]),
b = List.Combine(List.Transform(a, (x)=> x{1})),
c = if List.ContainsAny(b, [DateTimes]{1}) then true else false
][c], type logical),
_RemovedColumns = Table.RemoveColumns(_Ad_CheckOverlapping,{"IndexHelper", "DateTimes"})
][_RemovedColumns],
GroupedRows = Table.Group(AddedIndexHelper, {"ID_Card"}, {{"Fn", fn_CheckOverlapping, type table}}),
CombinedFn = Table.Combine(GroupedRows[Fn])
in
CombinedFn
- Joaomatos20022 years agoFrequent Visitor
Thank you for your effort and time, I thank you from the bottom of my heart. Its working now. You had already succeeded, but what you did was what you originally wanted.
I also leave here a code that I found in this community:
Time_Check_OverLap = VAR _Person = 'YourTable'[PersonID] -- Variable to hold the current person ID VAR _start = 'YourTable'[StartTime] -- Variable to hold the start time of the current record VAR _end = 'YourTable'[EndTime] -- Variable to hold the end time of the current record -- Variable to calculate the overlap condition VAR _condition_overlap = CALCULATE( COUNT('YourTable'[PersonID]), -- Counting the number of records with the same person ID that meet the overlap conditions FILTER( 'YourTable', 'YourTable'[PersonID] = _Person -- Ensuring the same person ID && ( (_start < 'YourTable'[StartTime] && _end > 'YourTable'[EndTime]) -- Case 1: Current record fully overlaps another record || _start = 'YourTable'[StartTime] -- Case 2: Current record starts at the same time as another record || (_start > 'YourTable'[StartTime] && _start < 'YourTable'[EndTime]) -- Case 3: Current record starts within the duration of another record ) ) ) -- Returning "n" if there is no overlap or only one occurrence (itself), otherwise returning "s" RETURN IF(_condition_overlap <= 1, "n", "s")! P E A C E F A M !
(͡• ͜ʖ ͡•)
- dufoq32 years agoCommunity Champion
I've added v2 into my previous post. V2 should be faster.