Forum Discussion

Varan_15's avatar
Varan_15
Icon for Helper III rankHelper III
1 year ago
Solved

single rows multiple IN/OUT time change to column level

Hi All,

 

I have raw data which is contains same naming worksteps with IN/OUT time  for that i need to convert into IN-OUT columns with respective timing using in power query kindly suggest please.

example source data:

Reference NoBarcodeProcess IDWorkstepstatusDate_Time
13/16850ABC121212120PendingOUT01Aug2024 14:10:52
13/16850ABC121212120InputIN01Aug2024 15:35:52
13/16850ABC121212120InputOUT01Aug2024 15:43:39
13/16850ABC121212120PendingIN01Aug2024 15:43:39
13/16850ABC121212120pendingOUT05Aug2024 13:27:37
13/16850ABC121212120InputIN05Aug2024 13:27:37
13/16850ABC121212120InputOUT05Aug2024 13:52:53
13/16850ABC121212120PendingIN05Aug2024 13:52:53
13/16850ABC121212120PendingOUT06Aug2024 14:05:55
13/16850ABC121212120InputIN06Aug2024 14:05:55
13/16850ABC121212120InputOUT06Aug2024 15:22:47
13/16850ABC121212120PendingIN06Aug2024 15:35:47

 

expected output table :

Reference NoBarcodeProcess IDWorkstepINOUT
13/16850ABC121212120PendingNULL01Aug2024 14:10:52
13/16850ABC121212120Input01Aug2024 15:35:5201Aug2024 15:43:39
13/16850ABC121212120Pending01Aug2024 15:43:3905Aug2024 13:27:37
13/16850ABC121212120Input05Aug2024 13:27:3705Aug2024 13:52:53
13/16850ABC121212120Pending05Aug2024 13:52:5306Aug2024 14:05:55
13/16850ABC121212120Input06Aug2024 14:05:5506Aug2024 15:22:47
13/16850ABC121212120Pending06Aug2024 15:35:47NULL

 

how to do in power query ?

 

Thanks,

Varan

 

 

  • Try this - 

    To transform your data in Power Query, you can follow these steps:

    Load your data into Power Query:
    Open Excel and load your data into Power Query by selecting your data range and clicking on Data > From Table/Range.
    Add an Index Column:
    In Power Query, go to the Add Column tab and select Index Column > From 0.
    Pivot the Data:
    Select the Workstep column, then go to the Transform tab and click on Pivot Column.
    In the Pivot Column dialog box, select status as the Values Column and Date_Time as the Values.
    Fill Down the Values:
    Select the columns that were created by the pivot (IN and OUT columns).
    Go to the Transform tab and click on Fill > Down.
    Remove Duplicates:
    Select the Reference No, Barcode, Process ID, and Workstep columns.
    Go to the Home tab and click on Remove Rows > Remove Duplicates.
    Reorder and Clean Up Columns:
    Reorder the columns as needed and remove the Index column if itโ€™s no longer necessary.


    Best Regards
    Saud Ansari
    If this post helps, please Accept it as a Solution to help other members find it. I appreciate your Kudos!

13 Replies

  • saud968's avatar
    saud968
    Icon for Memorable Member rankMemorable Member

    Load Data into Power Query:
    Open Excel and load your data into Power Query by selecting your data range and clicking on Data > From Table/Range.
    Split the Data into IN and OUT:
    Duplicate your query by right-clicking on the query name in the Queries pane and selecting Duplicate.
    In the first query, filter the status column to show only IN values.
    In the second query, filter the status column to show only OUT values.
    Rename Columns:
    In the IN query, rename the Date_Time column to IN.
    In the OUT query, rename the Date_Time column to OUT.
    Merge Queries:
    Go to the IN query and select Home > Merge Queries.
    Merge the IN query with the OUT query on Reference No, Barcode, Process ID, and Workstep.
    Choose the Left Outer join type to keep all rows from the IN query.
    Expand the Merged Table:
    After merging, expand the OUT table to include the OUT column.
    Clean Up:
    Remove any unnecessary columns and ensure the data types are correct.

    Best Regards
    Saud Ansari
    If this post helps, please Accept it as a Solution to help other members find it. I appreciate your Kudos!

    • Varan_15's avatar
      Varan_15
      Icon for Helper III rankHelper III

      saud968 ,

       

      Thanks for the suggesstions but data source is SQL table which is contains 4 million records it may get hang up, Is this will work out ?

       

      I have tried above steps but Reference No, Barcode, Process ID, and Workstep all are same hence OUT time creating duplicate records with back and forth.

       

      If some cases are IN/OUT time is null but due to same reference ,barcode number fetching previous IN/OUT times kindly help alterway power query functions if any.

       

      Thanks,

      Varan

       

  • Dear All,

     

    Can anyone help me on this above request how to achive using power query need to fix urgently

     

    Thanks,

    Varan

    • MSuser5's avatar
      MSuser5
      Icon for Helper III rankHelper III

      Try this also

      let

          // Load the data

          Source = Excel.CurrentWorkbook(){[Name="Data"]}[Content],

          

          // Convert Workstep to uppercase for consistency

          #"Uppercased Text" = Table.TransformColumns(Source, {{"Workstep", Text.Upper, type text}}),

          

          // Sort by Process ID and Process datetime to ensure order

          #"Sorted Rows" = Table.Sort(#"Uppercased Text", {{"Process ID", Order.Ascending}, {"Process datetime", Order.Ascending}}),

          

          // Add an index column to maintain original row order

          #"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "OriginalIndex", 1, 1, Int64.Type),

          

          // Separate tables for entries and exits

          EntryTable = Table.AddIndexColumn(Table.SelectRows(#"Added Index", each [status] = "Entry"), "EntryIndex", 1, 1),

          ExitTable = Table.AddIndexColumn(Table.SelectRows(#"Added Index", each [status] = "Exit"), "ExitIndex", 1, 1),

          

          // Rename columns to avoid conflicts

          RenamedEntryTable = Table.RenameColumns(EntryTable, {{"Process ID", "Process ID_Entry"}, {"Process datetime", "Entry datetime"}, {"Workstep", "Workstep_Entry"}}),

          RenamedExitTable = Table.RenameColumns(ExitTable, {{"Process ID", "Process ID_Exit"}, {"Process datetime", "Exit datetime"}, {"Workstep", "Workstep_Exit"}}),

          

          // Perform a full outer join on the indexed columns to align Entry and Exit

          Combined = Table.Join(RenamedEntryTable, {"Process ID_Entry", "EntryIndex"}, RenamedExitTable, {"Process ID_Exit", "ExitIndex"}, JoinKind.FullOuter),

          

          // Add a final Process ID and Workstep column by combining the respective values from both tables

          #"Added Combined Process ID" = Table.AddColumn(Combined, "Process ID", each if [Process ID_Entry] <> null then [Process ID_Entry] else [Process ID_Exit]),

          #"Added Combined Workstep" = Table.AddColumn(#"Added Combined Process ID", "Workstep", each if [Workstep_Entry] <> null then [Workstep_Entry] else [Workstep_Exit]),

          

          // Remove unnecessary columns after combining

          #"Removed Temporary Columns" = Table.RemoveColumns(#"Added Combined Workstep", {"Process ID_Entry", "Process ID_Exit", "EntryIndex", "ExitIndex", "Workstep_Entry", "Workstep_Exit"}),

          

          // Sort the final table by Process ID and entry datetime for better organization

          #"Sorted Final Rows" = Table.Sort(#"Removed Temporary Columns", {{"Process ID", Order.Ascending}, {"Entry datetime", Order.Ascending}})

      in

          #"Sorted Final Rows"

       

  • Try,

    let
    // Load the data
    Source = Excel.CurrentWorkbook(){[Name="Data"]}[Content],

    // Convert Workstep to uppercase for consistency
    #"Uppercased Text" = Table.TransformColumns(Source, {{"Workstep", Text.Upper, type text}}),

    // Sort by Process ID and Process datetime to ensure order
    #"Sorted Rows" = Table.Sort(#"Uppercased Text", {{"Process ID", Order.Ascending}, {"Process datetime", Order.Ascending}}),

    // Group by Process ID and Workstep, adding conditional indexes for Entry and Exit separately
    #"Added Conditional Index" = Table.Group(#"Sorted Rows", {"Process ID", "Workstep"}, {
    {"AllRows", each
    let
    // Separate tables for Entry and Exit, with conditional indexing
    EntryTable = Table.AddIndexColumn(Table.SelectRows(_, each [status] = "Entry"), "EntryIndex", 1, 1),
    ExitTable = Table.AddIndexColumn(Table.SelectRows(_, each [status] = "Exit"), "ExitIndex", 1, 1),

    // Rename columns to avoid conflicts
    RenamedEntryTable = Table.RenameColumns(EntryTable, {{"Process ID", "Process ID_Entry"}, {"Process datetime", "Entry datetime"}, {"status", "Entry status"}, {"Workstep", "Workstep_Entry"}}),
    RenamedExitTable = Table.RenameColumns(ExitTable, {{"Process ID", "Process ID_Exit"}, {"Process datetime", "Exit datetime"}, {"status", "Exit status"}, {"Workstep", "Workstep_Exit"}}),

    // Merge the two indexed tables back together using a join
    Combined = Table.Join(RenamedEntryTable, {"Process ID_Entry", "Workstep_Entry"}, RenamedExitTable, {"Process ID_Exit", "Workstep_Exit"}, JoinKind.FullOuter)
    in
    Combined
    }
    }, GroupKind.Local),

    // Expand the nested table to flatten the structure
    #"Expanded AllRows" = Table.ExpandTableColumn(#"Added Conditional Index", "AllRows", {"Entry datetime", "Exit datetime", "EntryIndex", "ExitIndex", "Process ID_Entry", "Process ID_Exit", "Entry status", "Exit status", "Workstep_Entry", "Workstep_Exit"}, {"Entry datetime", "Exit datetime", "EntryIndex", "ExitIndex", "Process ID_Entry", "Process ID_Exit", "Entry status", "Exit status", "Workstep_Entry", "Workstep_Exit"}),

    // Combine the Process ID and Workstep columns into one and remove the temporary columns
    #"Added Combined Process ID" = Table.AddColumn(#"Expanded AllRows", "Process ID", each if [Process ID_Entry] <> null then [Process ID_Entry] else [Process ID_Exit]),
    #"Added Combined Workstep" = Table.AddColumn(#"Added Combined Process ID", "Workstep", each if [Workstep_Entry] <> null then [Workstep_Entry] else [Workstep_Exit]),

    // Remove temporary columns used for processing
    #"Removed Temporary Columns" = Table.RemoveColumns(#"Added Combined Workstep", {"Process ID_Entry", "Process ID_Exit", "EntryIndex", "ExitIndex", "Entry status", "Exit status", "Workstep_Entry", "Workstep_Exit"}),

    // Sort the final table by Process ID for proper organization
    #"Sorted Final Rows" = Table.Sort(#"Removed Temporary Columns", {{"Process ID", Order.Ascending}})
    in
    #"Sorted Final Rows"

  • To achieve the desired transformation in Power Query, follow these steps:

    ### Steps:

    1. **Sort the Data**:
    - Ensure the data is sorted by **Reference No**, **Process ID**, and **Date_Time**.

    2. **Add a Custom Column**:
    - Create a custom column in Power Query to mark each `IN` and `OUT` row.

    3. **Pivot the Data**:
    - Select the **Workstep** and **Status** columns.
    - Use the **Pivot Column** option, setting **Date_Time** as the values column.

    4. **Adjust the Output**:
    - Rename columns as **IN** and **OUT**, and handle null values for proper representation.

    This process will reshape your data as expected.

  • saud968's avatar
    saud968
    Icon for Memorable Member rankMemorable Member

    Try this - 

    To transform your data in Power Query, you can follow these steps:

    Load your data into Power Query:
    Open Excel and load your data into Power Query by selecting your data range and clicking on Data > From Table/Range.
    Add an Index Column:
    In Power Query, go to the Add Column tab and select Index Column > From 0.
    Pivot the Data:
    Select the Workstep column, then go to the Transform tab and click on Pivot Column.
    In the Pivot Column dialog box, select status as the Values Column and Date_Time as the Values.
    Fill Down the Values:
    Select the columns that were created by the pivot (IN and OUT columns).
    Go to the Transform tab and click on Fill > Down.
    Remove Duplicates:
    Select the Reference No, Barcode, Process ID, and Workstep columns.
    Go to the Home tab and click on Remove Rows > Remove Duplicates.
    Reorder and Clean Up Columns:
    Reorder the columns as needed and remove the Index column if itโ€™s no longer necessary.


    Best Regards
    Saud Ansari
    If this post helps, please Accept it as a Solution to help other members find it. I appreciate your Kudos!

    • MSuser5's avatar
      MSuser5
      Icon for Helper III rankHelper III

      ExtraTimeInWorkingHours =
      VAR HasPending =
      CALCULATE(
      COUNTROWS(FILTER('Table', 'Table'[Workstep] = "Pending")),
      ALLEXCEPT('Table', 'Table'[CustomerID])
      ) > 0
      VAR HasAction =
      CALCULATE(
      COUNTROWS(FILTER('Table', 'Table'[Workstep] = "Action")),
      ALLEXCEPT('Table', 'Table'[CustomerID])
      ) > 0

      VAR PendingHours = IF(HasPending, 9, 0) // Add 9 hours only once if "Pending" is present
      VAR ActionHours = IF(HasAction, 6, 0) // Add 6 hours only once if "Action" is present

      VAR TotalHours = PendingHours + ActionHours
      VAR FullDaysToAdd = INT(TotalHours / ๐Ÿ˜Ž // Convert total hours to full working days
      VAR RemainingHours = MOD(TotalHours, ๐Ÿ˜Ž // Get remaining hours that don't form a full day

      RETURN
      (FullDaysToAdd * ๐Ÿ˜Ž + RemainingHours // Total extra hours accounting for full days and remaining hours

      • MSuser5's avatar
        MSuser5
        Icon for Helper III rankHelper III

        ExtraTimeInWorkingHours =
        VAR HasPending =
        CALCULATE(
        COUNTROWS(FILTER('Table', 'Table'[Workstep] = "Pending")),
        ALLEXCEPT('Table', 'Table'[CustomerID])
        ) > 0
        VAR HasAction =
        CALCULATE(
        COUNTROWS(FILTER('Table', 'Table'[Workstep] = "Action")),
        ALLEXCEPT('Table', 'Table'[CustomerID])
        ) > 0

        VAR PendingHours = IF(HasPending, 9, 0) // Add 9 hours only once if "Pending" is present
        VAR ActionHours = IF(HasAction, 6, 0) // Add 6 hours only once if "Action" is present

        VAR TotalHours = PendingHours + ActionHours

        // Calculate additional working days and hours based on the total hours
        VAR AdditionalDays = INT(TotalHours / ๐Ÿ˜Ž // Full days from total hours
        VAR RemainingHours = MOD(TotalHours, ๐Ÿ˜Ž // Remaining hours after full days

        RETURN
        IF(
        RemainingHours > 0,
        AdditionalDays + 1, // Add one more day if there's any remaining time
        AdditionalDays
        )

  • To check if the tickets are within or beyond the SLA based on the given conditions, you will need to calculate the allocated SLA time considering working hours (8 AM to 4 PM) and incorporate the additional hours for specific worksteps (like "Pending" and "Action"). Hereโ€™s how you can approach it in Power BI DAX:

    Step 1: Calculate the Base SLA Deadline

    Create a calculated column or measure to calculate the SLA deadline for each ticket based on the start time:

    BaseSLADeadline =
    VAR StartTime = 'Table'[StartDateTime]
    VAR StartDate = DATEVALUE(StartTime)
    VAR SLADeadline =
    IF(
    HOUR(StartTime) < 11,
    StartDate + TIME(16, 0, 0), // Same day 4 PM if started before 11 AM
    StartDate + 1 + TIME(16, 0, 0) // Next day 4 PM if started after 11 AM
    )
    RETURN
    SLADeadline

    Step 2: Add Extra Time for Specific Worksteps

    Now, you need to account for additional hours based on specific worksteps ("Pending" adds 8 hours, "Action" adds 5 hours).

    Create a measure or calculated column to calculate the total additional time:

    ExtraTimeInHours =
    SUMX(
    FILTER(
    'Table',
    'Table'[CustomerID] = EARLIER('Table'[CustomerID]) &&
    ('Table'[Workstep] = "Pending" || 'Table'[Workstep] = "Action")
    ),
    SWITCH(
    'Table'[Workstep],
    "Pending", 8,
    "Action", 5,
    0
    )
    )

    Step 3: Calculate the SLA Deadline Including Additional Hours

    You need to adjust the base SLA deadline considering working hours (8 AM to 4 PM). The additional hours should be added only during the working hours.

    Create a measure to calculate the adjusted SLA deadline:

    AdjustedSLADeadline =
    VAR BaseDeadline = [BaseSLADeadline]
    VAR ExtraHours = [ExtraTimeInHours]
    VAR WorkingStartTime = TIME(8, 0, 0)
    VAR WorkingEndTime = TIME(16, 0, 0)
    VAR FullDaysToAdd = INT(ExtraHours / ๐Ÿ˜Ž
    VAR RemainingHours = MOD(ExtraHours, ๐Ÿ˜Ž
    VAR ExtraDays = IF(HOUR(BaseDeadline) + RemainingHours > 16, 1, 0)
    VAR NewDate = BaseDeadline + FullDaysToAdd + ExtraDays
    VAR NewTime = TIME(HOUR(BaseDeadline) + RemainingHours, MINUTE(BaseDeadline), SECOND(BaseDeadline))

    RETURN
    IF(
    NewTime > WorkingEndTime,
    NewDate + 1 + WorkingStartTime,
    NewDate + NewTime
    )

    Step 4: Compare the Actual Completion Time Against the SLA Deadline

    Finally, you can create a measure to check if the ticket is within or beyond the SLA:

    SLAStatus =
    IF(
    'Table'[EndDateTime] <= [AdjustedSLADeadline],
    "Within SLA",
    "Beyond SLA"
    )

    Summary

    1. BaseSLADeadline calculates the initial SLA deadline based on the start time.


    2. ExtraTimeInHours adds additional time for specific worksteps.


    3. AdjustedSLADeadline recalculates the SLA deadline considering working hours.


    4. SLAStatus compares the actual end time against the adjusted SLA deadline to check if it is within or beyond SLA.

     

    This approach ensures that SLA calculations account for working hours and the specified rules for different scenarios.

     

  • ExtraTimeInHours =
    VAR HasPending =
    CALCULATE(
    COUNTROWS(FILTER('Table', 'Table'[Workstep] = "Pending")),
    ALLEXCEPT('Table', 'Table'[CustomerID])
    ) > 0
    VAR ActionHours =
    CALCULATE(
    SUMX(
    FILTER('Table', 'Table'[Workstep] = "Action"),
    5
    ),
    ALLEXCEPT('Table', 'Table'[CustomerID])
    )
    RETURN
    IF(HasPending, 8, 0) + ActionHours