Forum Discussion

kblommer's avatar
kblommer
Frequent Visitor
6 years ago
Solved

Consolidate Rows based on Dates within range of each other

Hello, I have a table with two columns: ID and Date.   The ID column is a 20-digit whole number The Date column is in the format MM/DD/YYYY HH:MM:SS.   There are some rows with matching IDs. Of...
  • edhans's avatar
    6 years ago

    Ok kblommer - see if this helps.

     

    Original Data I used as a test (the comment is to show you what I will recognize as a repeat repair after the query. I didn't cheat and put that in my source queries 😉 )

     

    ID Date Comment
    1 1/1/2020  
    1 1/3/2020 Repeat
    2 2/1/2020  
    3 1/13/2020  
    4 2/20/2020  
    5 1/30/2020  
    5 1/31/2020 Repeat
    6 1/21/2020  
    6 2/21/2020  
    6 3/4/2020  
    7 1/18/2020  
    7 3/3/2020  
    8 1/1/2020  
    9 1/28/2020  
    10 3/31/2020  
    10 4/1/2020 Repeat
    10 5/1/2020  

     

    In the file below, I did three queries:

    1. Repairs - this is the original table and I did nothing to it really other than the formatting and get rid of my cheating comment field.
    2. Repairs Near Date
      1. This is using Repairs as the source
      2. Added an index to keep track of the original data.
      3. Added a list of the next two dates. So Jan 1 in the date column would generate a list of Jan 2 and Jan 3. You could adjust that logic in the "Added Near Dates" step.
      4. Expanded the Near Dates column. I now have massive duplication.
    3. Matching Date Range
      1. Used Repairs Near Date query above as the source
      2. Did a left join to the original Repairs query matching the date and repair ID number, then expanded the query. You'll note now most are null. Those that are not null are the culprits.
      3. I grouped the ID, Date, and Index column and threw everything else into an All Rows nested table. You'll note I'm back to my original 17 rows in the source data. All duplications are in the nested table.
      4. I added a new column called Matching Row Record that uses Table.Max to get the maximum value for the repeated Reapair.ID (not the original ID) column. I just wanted to get rows with data and nulls where there were no matches. 
      5. Expanded that maximum record. Now I know what is matching but I need to find a way to filter that out from the source. For example, I now know that the ID#1 for Jan 1 has a match for ID#1 for Jan 3, but don't yet have a way to get rid of the ID#1 Jan 3 record in the original ID/Date columns.
      6. Filled the expanded records down, filling in all of the nulls.
      7. Added a column that returns TRUE/FALSE if the ID and Date match. It returns 3 trues, my repeat repairs. I then filter out all TRUE values.
      8. Now down to 14 records - 17 original - 3 repeats.
      9. Used the "Matching Repair Date" function below to find those that are the repeats in the 3 day range. If they are, keep the repair date, otherwise return null.
      10. Kept the ID, Date, and Matching Repair date columns.
      11. The end.

     

     

    Matching Repair Date Formula
    
    if ([ID] = [Repairs.ID]) and (Number.Abs(Number.From([Repairs.Date]) - Number.From([Date])) < 3)
    then [Repairs.Date]
    else null

     

     

    So my end data looks like this:

     

    I did all of this in Excel, but it is the exact same logic in Power BI's Power Query tool. Here is my Excel file.

     

    Let me know if that works for you.