Forum Discussion

OzzyM91's avatar
OzzyM91
Frequent Visitor
3 years ago
Solved

Find difference between current and previous row - performance issue

Hello, I've got a dataset of transactions, including various salesmen with transactions times. I need to create a calculated column returning datetime of previous transaction, allowing me to subsequ...
  • PaulDBrown's avatar
    PaulDBrown
    3 years ago

    Ok, try a different approach in Power Query. Basically it entails creating a single table with the transaction time and the next transacion time. Then leave the calculations to measures (which are simple since both transaction times are on the same row of the table. 

    To do this, you need to sort the table by salesman and Transaction time (in ascending order), and add an index column starting at 1

    Now duplicate the table and change the index order to start at 0

    Now you can merge both tables by selecting salesman and index, and keep only the new transaction row from the second table:

    (You can disable load for the second table since you don't need it in the model)

    If you prefer to do this whole process with a single query, here is the M code:

    let
      Source = Table.FromRows(
        Json.Document(
          Binary.Decompress(
            Binary.FromText(
              "i45WCsnPVdJRMjLSNzDUNzIwMlIwNLIyMFCK1cEmZQqT8s1MzkhMzQFJGyOkTfBLm1kZoxiMJGVuZWKKR6cFmpNMsLgWSacJfhebkGYwVMoUWRfII7EA",
              BinaryEncoding.Base64
            ),
            Compression.Deflate
          )
        ),
        let
          _t = ((type nullable text) meta [Serialized.Text = true])
        in
          type table [Salesman = _t, #"Transaction time" = _t]
      ),
      #"Changed Type" = Table.TransformColumnTypes(
        Source,
        {{"Salesman", type text}, {"Transaction time", type datetime}}
      ),
      #"Sorted Rows" = Table.Sort(
        #"Changed Type",
        {{"Salesman", Order.Ascending}, {"Transaction time", Order.Ascending}}
      ),
      #"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "Index", 1, 1, Int64.Type),
      Source1 = Source,
      #"Changed Type1" = Table.TransformColumnTypes(
        Source1,
        {{"Salesman", type text}, {"Transaction time", type datetime}}
      ),
      #"Sorted Rows1" = Table.Sort(
        #"Changed Type1",
        {{"Salesman", Order.Ascending}, {"Transaction time", Order.Ascending}}
      ),
      #"Added Index1" = Table.AddIndexColumn(#"Sorted Rows1", "Index", 0, 1, Int64.Type),
      #"Merged Queries" = Table.NestedJoin(
        #"Added Index",
        {"Salesman", "Index"},
        #"Added Index1",
        {"Salesman", "Index"},
        "Next transaction",
        JoinKind.LeftOuter
      ),
      #"Expanded Next Transaction" = Table.ExpandTableColumn(
        #"Merged Queries",
        "Next transaction",
        {"Transaction time"},
        {"Next transaction"}
      ),
      #"Removed Columns" = Table.RemoveColumns(#"Expanded Next Transaction", {"Index"})
    in
      #"Removed Columns"

    Now the measures are much simpler:

    Difference in Minutes =
    DATEDIFF (
        MAX ( 'TD on rows'[Transaction time] ),
        MAX ( 'TD on rows'[Next transaction] ),
        MINUTE
    )
    
    Diff vs Next transaction time (HH:MM) =
    VAR _MinutesDiff =
        DATEDIFF (
            MAX ( 'TD on rows'[Transaction time] ),
            MAX ( 'TD on rows'[Next transaction] ),
            MINUTE
        )
    VAR _FinalHours =
        INT ( DIVIDE ( _MinutesDiff, 60 ) )
    VAR _FinalMinutes =
        FORMAT ( MOD ( _MinutesDiff, 60 ), "00" )
    RETURN
        IF (
            ISBLANK ( MAX ( 'TD on rows'[Next transaction] ) ),
            BLANK (),
            _FinalHours & ":" & _FinalMinutes
        )
    

     

    New file attached