Find previous event in an event log using Power Query

[magnus_b]

Hi,

I have data like this in a direct query source:

case_id	timestamp	event
99288	28.10.2020 08:55:32	created
71263	28.10.2020 09:28:32	created
99288	28.10.2020 09:29:00	assigned
99288	28.10.2020 09:39:04	solved
71263	28.10.2020 10:44:02	solved
87100	28.10.2020 10:55:01	created
71263	28.10.2020 11:22:46	reopened

I want to calculate a new column that contains the previous event for the same case id:

case_id	timestamp	event	previous_event
99288	28.10.2020 08:55:32	created
71263	28.10.2020 09:28:32	created
99288	28.10.2020 09:29:00	assigned	created
99288	28.10.2020 09:39:04	solved	assigned
71263	28.10.2020 10:44:02	solved	created
87100	28.10.2020 10:55:01	created
71263	28.10.2020 11:22:46	reopened	solved

How can I achieve this using Power Query?

I did find a solution using DAX (https://community.powerbi.com/t5/DAX-Commands-and-Tips/Find-previous-event-in-an-event-log-using-DAX...) , but that did not work when using direct query, which is a requirement.

[mahoneypat]

Given the limitations of direct query, you may need to do this in a measure.  How do you plan to use this column once generated?  If it will just be a value in a table/matrix, then a measure would work.  If it will be used as a legend or on an axis, a column is needed.

Regards,

Pat

[Jimmy801]

Hello @magnus_b 

You can group your data applying the function of the whole table. Then with a Table.TransformColumns you can manipulate the grouped table by sorting it, adding index column and then adding a new column, where the grouped table is filtered by [Index]-1.

here a praticable example (in case, substitue the source and the changes-type step with your existing query

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("hc/NCoNADATgVyk5iySzq2bzKosHaZdSKLW4pc9v8GZ/r8NHJpMzpQRVagjaCrdg8IHVus4CPD0uZXqUE41NpkHQhxeZDPouP910mYzZ06nWy/n2kwan0dM6X5/f24UtRmPsoQ6y1eyhD2L5P0jEAIu9p0uZ72V7c1wB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [case_id = _t, timestamp = _t, event = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"case_id", Int64.Type}, {"timestamp", type datetime}, {"event", type text}}),
    #"Grouped Rows" = Table.Group(#"Changed Type", {"case_id"}, {{"AllRows", each _, type table [case_id=number, timestamp=datetime, event=text]}}),
    SortAndAddColumn = Table.TransformColumns
    (
        #"Grouped Rows",
        {
            {
                "AllRows",
                (tbl)=> let 
                    Sort = Table.Sort(tbl, {{"timestamp", Order.Ascending}}),
                    Index = Table.AddIndexColumn(Sort, "Index"),
                    AddPrevious= Table.AddColumn(Index, "Previous", (row)=> try Table.SelectRows(Index, each [Index]= row[Index]-1)[event]{0} otherwise "")
                    in 
                    AddPrevious
            }
        }
    ),
    #"Expanded AllRows" = Table.ExpandTableColumn(SortAndAddColumn, "AllRows", {"timestamp", "event", "Previous"}, {"timestamp", "event", "Previous"}),
    #"Sorted Rows" = Table.Sort(#"Expanded AllRows",{{"timestamp", Order.Ascending}})



in
    #"Sorted Rows"

Copy paste this code to the advanced editor in a new blank query to see how the solution works

If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too

Have fun

Jimmy

[magnus_b]

Thank you @Jimmy801 ! 

Unfortunately it does not work - the SortAndAddColum step is not compatible with direct query. Any ideas?

[Jimmy801]

Hello @magnus_b 

try this code. I don't know if this is supported in direct mode and how the performance is. Hope it helps

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("hc/NCoNADATgVyk5iySzq2bzKosHaZdSKLW4pc9v8GZ/r8NHJpMzpQRVagjaCrdg8IHVus4CPD0uZXqUE41NpkHQhxeZDPouP910mYzZ06nWy/n2kwan0dM6X5/f24UtRmPsoQ6y1eyhD2L5P0jEAIu9p0uZ72V7c1wB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [case_id = _t, timestamp = _t, event = _t]),
    ChangeType = Table.TransformColumnTypes(Source,{{"case_id", Int64.Type}, {"timestamp", type datetime}, {"event", type text}}),
    #"Grouped Rows" = Table.Buffer(Table.Group(ChangeType, {"case_id"}, {{"AllRows", each _, type table [case_id=number, timestamp=datetime, event=text]}})),
    AddPrevisous = Table.AddColumn
    (
        ChangeType,
        "Previous",
        (row)=> 
        let 
            GetTable = Table.SelectRows(#"Grouped Rows", each [#"case_id"]= row[#"case_id"])[AllRows]{0},
            SelectTimeStamp = Table.SelectRows(GetTable, each [timestamp]< row[timestamp]),
            GetLastTimeStamp = try Table.Last(SelectTimeStamp)[event] otherwise ""
        in 
            GetLastTimeStamp
    )
in
    AddPrevisous

Copy paste this code to the advanced editor in a new blank query to see how the solution works.

If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too

Have fun

Jimmy

[magnus_b]

@Jimmy801 Thank you for the suggestion. Unfortunately this query is also not compatible with direct query.