Forum Discussion

OptiBill's avatar
OptiBill
New Member
3 years ago
Solved

Building a History Table from a Change Log and Current State Table

I am attempting to build a "History" table for a Power BI report from a "Change Log" table and a "Current State" table. As an example, the "Change Log" table has the following data:

 

Column1CreatedFieldFromStringToString
ABCD10/1/2022 12:00AssigneeUser1User2
ABCD10/1/2022 16:00StatusOpenIn Progress
ABCD10/2/2022 13:00AssigneeUser2User3
ABCD10/3/2022 6:00Priority2 - High1 - Blocking
ABCD10/9/2022 14:00StatusIn ProgressClosed

 

The "Current State" table has the record as it currently exists:

CreatedDatetimeUpdatedDatetimeAssigneeKeyStatusPriority
10/1/2022 10:0010/9/2022 14:00User3ABCDClosed1 - Blocking

 

And my desired "History" table would be:

CreatedDatetimeUpdatedDatetimeAssigneeKeyStatusPriority
10/1/2022 10:0010/1/2022 10:00User1ABCDOpen2 - High
10/1/2022 10:0010/1/2022 12:00User2ABCDOpen2 - High
10/1/2022 10:0010/1/2022 16:00User2ABCDIn Progress2 - High
10/1/2022 10:0010/2/2022 13:00User3ABCDIn Progress2 - High
10/1/2022 10:0010/3/2022 6:00User3ABCDIn Progress1 - Blocking
10/1/2022 10:0010/9/2022 14:00User3ABCDClosed1 - Blocking

 

Is this achievable in Power Query? 

4 Replies

  • ImkeF's avatar
    ImkeF
    Community Champion

    Hi OptiBill ,
    yes, that's perfectly doable in PQ. This is the logic:

    let
        Source = Excel.CurrentWorkbook(){[Name = "ChangeLog"]}[Content], 
        #"Changed Type" = Table.TransformColumnTypes(
            Source, 
            {
                {"Column1", type text}, 
                {"Created", type datetime}, 
                {"Field", type text}, 
                {"FromString", type text}, 
                {"ToString", type text}
            }
        ), 
        #"Removed Duplicates" = Table.Distinct(#"Changed Type", {"Field"}), 
        #"Removed Columns" = Table.RemoveColumns(#"Removed Duplicates", {"ToString", "Created"}), 
        #"Pivoted Column1" = Table.Pivot(
            #"Removed Columns", 
            List.Distinct(#"Removed Columns"[Field]), 
            "Field", 
            "FromString"
        ), 
        #"Merged Queries" = Table.NestedJoin(
            #"Pivoted Column1", 
            {"Column1"}, 
            CurrentState, 
            {"Key"}, 
            "CurrentState", 
            JoinKind.LeftOuter
        ), 
        _Initial = Table.ExpandTableColumn(#"Merged Queries", "CurrentState", {"CreatedDatetime"}, {"Created"}), 
        #"Pivoted Column" = #"Changed Type", 
        #"Removed Columns1" = Table.RemoveColumns(#"Pivoted Column", {"FromString"}), 
        _Ongoing = Table.Pivot(
            #"Removed Columns1", 
            List.Distinct(#"Removed Columns1"[Field]), 
            "Field", 
            "ToString"
        ), 
        Custom1 = _Initial & _Ongoing, 
        #"Filled Down" = Table.FillDown(Custom1, {"Assignee", "Status", "Priority"})
    in
        #"Filled Down"

    Please check out the file enclosed as well.

    • OptiBill's avatar
      OptiBill
      New Member

      Thanks ImkeF! This looks amazing and it directly answers my initial question. However, I didn't realize that my example wasn't inclusive of all scenarios, which was my fault. When including additional data which may have some, all, or none of the same changes it looks like the current solution is breaking down.

       

      For example, if an additional "Change Log" entry is added for Key EFGH so that the "Change Log" now reads as 

      Column1CreatedFieldFromStringToString
      ABCD10/1/2022 12:00AssigneeUser1User2
      ABCD10/1/2022 16:00StatusOpenIn Progress
      ABCD10/2/2022 13:00AssigneeUser2User3
      ABCD10/3/2022 6:00Priority2 - High1 - Blocking
      ABCD10/9/2022 14:00StatusIn ProgressClosed
      EFGH10/3/2022 14:00Assignee User1

       

      And a corresponding additional "Current State" entry is added for Key EFGH, as well as an entry for Key IJKL (which has no entries in the "Change Log" since no changes have been made since it's initial creation) so that the "Current State" reads as

      CreatedDatetimeUpdatedDatetimeAssigneeKeyStatusPriority
      10/1/2022 10:0010/9/2022 14:00User3ABCDClosed1 - Blocking
      10/2/2022 11:0010/3/2022 14:00User1EFGHOpen1 - Blocking
      10/1/2022 8:0010/1/2022 8:00 IJKLOpen2 - High

       

      The resulting "History" table is showing as

      Column1AssigneeStatusPriorityCreated

      ABCDUser1Open2 - High10/1/2022 10:00:00 AM
      ABCDUser2Open2 - High10/1/2022 12:00:00 PM
      ABCDUser2In Progress2 - High10/1/2022 4:00:00 PM
      ABCDUser3In Progress2 - High10/2/2022 1:00:00 PM
      ABCDUser3In Progress1 - Blocking10/3/2022 6:00:00 AM
      ABCDUser3Closed1 - Blocking10/9/2022 2:00:00 PM
      EFGHUser1Closed1 - Blocking10/3/2022 2:00:00 PM

       

      The desired "History" table would instead be

      Column1AssigneeStatusPriorityCreated
      ABCDUser1Open2 - High10/1/2022 10:00:00 AM
      ABCDUser2Open2 - High10/1/2022 12:00:00 PM
      ABCDUser2In Progress2 - High10/1/2022 4:00:00 PM
      ABCDUser3In Progress2 - High10/2/2022 1:00:00 PM
      ABCDUser3In Progress1 - Blocking10/3/2022 6:00:00 AM
      ABCDUser3Closed1 - Blocking10/9/2022 2:00:00 PM
      EFGH Open1 - Blocking10/2/2022 11:00:00 AM
      EFGHUser1Open1 - Blocking10/3/2022 2:00:00 PM
      IJKL Open2 - High10/1/2022 8:00:00 AM

       

      It looks like the EFGH Key is missing a row AND has the wrong Status for the row that is included. Additionally, the row for IJKL is missing altogether.

       

      Can your solution be enhanced to handle these additional uniquenesses?

    • GARAVENA's avatar
      GARAVENA
      Frequent Visitor

      Is this achievable with dax? Im trying to do the same, but working with a DB in direct query.