Forum Discussion

magnus_b's avatar
magnus_b
Advocate II
5 years ago

Find previous event in an event log using Power Query

Hi,

 

I have data like this in a direct query source:

 

case_idtimestampevent
9928828.10.2020 08:55:32created
7126328.10.2020 09:28:32created
9928828.10.2020 09:29:00assigned
9928828.10.2020 09:39:04solved
7126328.10.2020 10:44:02solved
8710028.10.2020 10:55:01created
7126328.10.2020 11:22:46reopened

 

I want to calculate a new column that contains the previous event for the same case id:

 

case_idtimestampeventprevious_event
9928828.10.2020 08:55:32created 
7126328.10.2020 09:28:32created 
9928828.10.2020 09:29:00assignedcreated
9928828.10.2020 09:39:04solvedassigned
7126328.10.2020 10:44:02solvedcreated
8710028.10.2020 10:55:01created 
7126328.10.2020 11:22:46reopenedsolved

 

How can I achieve this using Power Query?

I did find a solution using DAX (https://community.powerbi.com/t5/DAX-Commands-and-Tips/Find-previous-event-in-an-event-log-using-DAX/m-p/1460986#M27547) , but that did not work when using direct query, which is a requirement.

5 Replies

  • Jimmy801's avatar
    Jimmy801
    Community Champion

    Hello magnus_b 

     

    You can group your data applying the function of the whole table. Then with a Table.TransformColumns you can manipulate the grouped table by sorting it, adding index column and then adding a new column, where the grouped table is filtered by [Index]-1.

    here a praticable example (in case, substitue the source and the changes-type step with your existing query

     

     

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("hc/NCoNADATgVyk5iySzq2bzKosHaZdSKLW4pc9v8GZ/r8NHJpMzpQRVagjaCrdg8IHVus4CPD0uZXqUE41NpkHQhxeZDPouP910mYzZ06nWy/n2kwan0dM6X5/f24UtRmPsoQ6y1eyhD2L5P0jEAIu9p0uZ72V7c1wB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [case_id = _t, timestamp = _t, event = _t]),
        #"Changed Type" = Table.TransformColumnTypes(Source,{{"case_id", Int64.Type}, {"timestamp", type datetime}, {"event", type text}}),
        #"Grouped Rows" = Table.Group(#"Changed Type", {"case_id"}, {{"AllRows", each _, type table [case_id=number, timestamp=datetime, event=text]}}),
        SortAndAddColumn = Table.TransformColumns
        (
            #"Grouped Rows",
            {
                {
                    "AllRows",
                    (tbl)=> let 
                        Sort = Table.Sort(tbl, {{"timestamp", Order.Ascending}}),
                        Index = Table.AddIndexColumn(Sort, "Index"),
                        AddPrevious= Table.AddColumn(Index, "Previous", (row)=> try Table.SelectRows(Index, each [Index]= row[Index]-1)[event]{0} otherwise "")
                        in 
                        AddPrevious
                }
            }
        ),
        #"Expanded AllRows" = Table.ExpandTableColumn(SortAndAddColumn, "AllRows", {"timestamp", "event", "Previous"}, {"timestamp", "event", "Previous"}),
        #"Sorted Rows" = Table.Sort(#"Expanded AllRows",{{"timestamp", Order.Ascending}})
    
    
    
    in
        #"Sorted Rows"

     

     

    Copy paste this code to the advanced editor in a new blank query to see how the solution works

    If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
    Kudoes are nice too

    Have fun

    Jimmy

     

    • magnus_b's avatar
      magnus_b
      Advocate II

      Thank you Jimmy801 ! 

      Unfortunately it does not work - the SortAndAddColum step is not compatible with direct query. Any ideas?

       

      • Jimmy801's avatar
        Jimmy801
        Community Champion

        Hello magnus_b 

         

        try this code. I don't know if this is supported in direct mode and how the performance is. Hope it helps

        let
            Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("hc/NCoNADATgVyk5iySzq2bzKosHaZdSKLW4pc9v8GZ/r8NHJpMzpQRVagjaCrdg8IHVus4CPD0uZXqUE41NpkHQhxeZDPouP910mYzZ06nWy/n2kwan0dM6X5/f24UtRmPsoQ6y1eyhD2L5P0jEAIu9p0uZ72V7c1wB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [case_id = _t, timestamp = _t, event = _t]),
            ChangeType = Table.TransformColumnTypes(Source,{{"case_id", Int64.Type}, {"timestamp", type datetime}, {"event", type text}}),
            #"Grouped Rows" = Table.Buffer(Table.Group(ChangeType, {"case_id"}, {{"AllRows", each _, type table [case_id=number, timestamp=datetime, event=text]}})),
            AddPrevisous = Table.AddColumn
            (
                ChangeType,
                "Previous",
                (row)=> 
                let 
                    GetTable = Table.SelectRows(#"Grouped Rows", each [#"case_id"]= row[#"case_id"])[AllRows]{0},
                    SelectTimeStamp = Table.SelectRows(GetTable, each [timestamp]< row[timestamp]),
                    GetLastTimeStamp = try Table.Last(SelectTimeStamp)[event] otherwise ""
                in 
                    GetLastTimeStamp
            )
        in
            AddPrevisous

         

        Copy paste this code to the advanced editor in a new blank query to see how the solution works.

        If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
        Kudoes are nice too

        Have fun

        Jimmy

         

  • mahoneypat's avatar
    mahoneypat
    Microsoft Employee

    Given the limitations of direct query, you may need to do this in a measure.  How do you plan to use this column once generated?  If it will just be a value in a table/matrix, then a measure would work.  If it will be used as a legend or on an axis, a column is needed.

     

    Regards,

    Pat