Forum Discussion
single rows multiple IN/OUT time change to column level
Hi All,
I have raw data which is contains same naming worksteps with IN/OUT time for that i need to convert into IN-OUT columns with respective timing using in power query kindly suggest please.
example source data:
| Reference No | Barcode | Process ID | Workstep | status | Date_Time |
| 13/16850 | ABC1212 | 12120 | Pending | OUT | 01Aug2024 14:10:52 |
| 13/16850 | ABC1212 | 12120 | Input | IN | 01Aug2024 15:35:52 |
| 13/16850 | ABC1212 | 12120 | Input | OUT | 01Aug2024 15:43:39 |
| 13/16850 | ABC1212 | 12120 | Pending | IN | 01Aug2024 15:43:39 |
| 13/16850 | ABC1212 | 12120 | pending | OUT | 05Aug2024 13:27:37 |
| 13/16850 | ABC1212 | 12120 | Input | IN | 05Aug2024 13:27:37 |
| 13/16850 | ABC1212 | 12120 | Input | OUT | 05Aug2024 13:52:53 |
| 13/16850 | ABC1212 | 12120 | Pending | IN | 05Aug2024 13:52:53 |
| 13/16850 | ABC1212 | 12120 | Pending | OUT | 06Aug2024 14:05:55 |
| 13/16850 | ABC1212 | 12120 | Input | IN | 06Aug2024 14:05:55 |
| 13/16850 | ABC1212 | 12120 | Input | OUT | 06Aug2024 15:22:47 |
| 13/16850 | ABC1212 | 12120 | Pending | IN | 06Aug2024 15:35:47 |
expected output table :
| Reference No | Barcode | Process ID | Workstep | IN | OUT |
| 13/16850 | ABC1212 | 12120 | Pending | NULL | 01Aug2024 14:10:52 |
| 13/16850 | ABC1212 | 12120 | Input | 01Aug2024 15:35:52 | 01Aug2024 15:43:39 |
| 13/16850 | ABC1212 | 12120 | Pending | 01Aug2024 15:43:39 | 05Aug2024 13:27:37 |
| 13/16850 | ABC1212 | 12120 | Input | 05Aug2024 13:27:37 | 05Aug2024 13:52:53 |
| 13/16850 | ABC1212 | 12120 | Pending | 05Aug2024 13:52:53 | 06Aug2024 14:05:55 |
| 13/16850 | ABC1212 | 12120 | Input | 06Aug2024 14:05:55 | 06Aug2024 15:22:47 |
| 13/16850 | ABC1212 | 12120 | Pending | 06Aug2024 15:35:47 | NULL |
how to do in power query ?
Thanks,
Varan
Try this -
To transform your data in Power Query, you can follow these steps:
Load your data into Power Query:
Open Excel and load your data into Power Query by selecting your data range and clicking on Data > From Table/Range.
Add an Index Column:
In Power Query, go to the Add Column tab and select Index Column > From 0.
Pivot the Data:
Select the Workstep column, then go to the Transform tab and click on Pivot Column.
In the Pivot Column dialog box, select status as the Values Column and Date_Time as the Values.
Fill Down the Values:
Select the columns that were created by the pivot (IN and OUT columns).
Go to the Transform tab and click on Fill > Down.
Remove Duplicates:
Select the Reference No, Barcode, Process ID, and Workstep columns.
Go to the Home tab and click on Remove Rows > Remove Duplicates.
Reorder and Clean Up Columns:
Reorder the columns as needed and remove the Index column if itโs no longer necessary.
Best Regards
Saud Ansari
If this post helps, please Accept it as a Solution to help other members find it. I appreciate your Kudos!
13 Replies
- saud968
Memorable Member
Load Data into Power Query:
Open Excel and load your data into Power Query by selecting your data range and clicking on Data > From Table/Range.
Split the Data into IN and OUT:
Duplicate your query by right-clicking on the query name in the Queries pane and selecting Duplicate.
In the first query, filter the status column to show only IN values.
In the second query, filter the status column to show only OUT values.
Rename Columns:
In the IN query, rename the Date_Time column to IN.
In the OUT query, rename the Date_Time column to OUT.
Merge Queries:
Go to the IN query and select Home > Merge Queries.
Merge the IN query with the OUT query on Reference No, Barcode, Process ID, and Workstep.
Choose the Left Outer join type to keep all rows from the IN query.
Expand the Merged Table:
After merging, expand the OUT table to include the OUT column.
Clean Up:
Remove any unnecessary columns and ensure the data types are correct.
Best Regards
Saud Ansari
If this post helps, please Accept it as a Solution to help other members find it. I appreciate your Kudos!- Varan_15
Helper III
saud968 ,
Thanks for the suggesstions but data source is SQL table which is contains 4 million records it may get hang up, Is this will work out ?
I have tried above steps but Reference No, Barcode, Process ID, and Workstep all are same hence OUT time creating duplicate records with back and forth.
If some cases are IN/OUT time is null but due to same reference ,barcode number fetching previous IN/OUT times kindly help alterway power query functions if any.
Thanks,
Varan
- Varan_15
Helper III
Dear All,
Can anyone help me on this above request how to achive using power query need to fix urgently
Thanks,
Varan
- MSuser5
Helper III
Try this also
let
// Load the data
Source = Excel.CurrentWorkbook(){[Name="Data"]}[Content],
// Convert Workstep to uppercase for consistency
#"Uppercased Text" = Table.TransformColumns(Source, {{"Workstep", Text.Upper, type text}}),
// Sort by Process ID and Process datetime to ensure order
#"Sorted Rows" = Table.Sort(#"Uppercased Text", {{"Process ID", Order.Ascending}, {"Process datetime", Order.Ascending}}),
// Add an index column to maintain original row order
#"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "OriginalIndex", 1, 1, Int64.Type),
// Separate tables for entries and exits
EntryTable = Table.AddIndexColumn(Table.SelectRows(#"Added Index", each [status] = "Entry"), "EntryIndex", 1, 1),
ExitTable = Table.AddIndexColumn(Table.SelectRows(#"Added Index", each [status] = "Exit"), "ExitIndex", 1, 1),
// Rename columns to avoid conflicts
RenamedEntryTable = Table.RenameColumns(EntryTable, {{"Process ID", "Process ID_Entry"}, {"Process datetime", "Entry datetime"}, {"Workstep", "Workstep_Entry"}}),
RenamedExitTable = Table.RenameColumns(ExitTable, {{"Process ID", "Process ID_Exit"}, {"Process datetime", "Exit datetime"}, {"Workstep", "Workstep_Exit"}}),
// Perform a full outer join on the indexed columns to align Entry and Exit
Combined = Table.Join(RenamedEntryTable, {"Process ID_Entry", "EntryIndex"}, RenamedExitTable, {"Process ID_Exit", "ExitIndex"}, JoinKind.FullOuter),
// Add a final Process ID and Workstep column by combining the respective values from both tables
#"Added Combined Process ID" = Table.AddColumn(Combined, "Process ID", each if [Process ID_Entry] <> null then [Process ID_Entry] else [Process ID_Exit]),
#"Added Combined Workstep" = Table.AddColumn(#"Added Combined Process ID", "Workstep", each if [Workstep_Entry] <> null then [Workstep_Entry] else [Workstep_Exit]),
// Remove unnecessary columns after combining
#"Removed Temporary Columns" = Table.RemoveColumns(#"Added Combined Workstep", {"Process ID_Entry", "Process ID_Exit", "EntryIndex", "ExitIndex", "Workstep_Entry", "Workstep_Exit"}),
// Sort the final table by Process ID and entry datetime for better organization
#"Sorted Final Rows" = Table.Sort(#"Removed Temporary Columns", {{"Process ID", Order.Ascending}, {"Entry datetime", Order.Ascending}})
in
#"Sorted Final Rows"
- MSuser5
Helper III
Try,
let
// Load the data
Source = Excel.CurrentWorkbook(){[Name="Data"]}[Content],
// Convert Workstep to uppercase for consistency
#"Uppercased Text" = Table.TransformColumns(Source, {{"Workstep", Text.Upper, type text}}),
// Sort by Process ID and Process datetime to ensure order
#"Sorted Rows" = Table.Sort(#"Uppercased Text", {{"Process ID", Order.Ascending}, {"Process datetime", Order.Ascending}}),
// Group by Process ID and Workstep, adding conditional indexes for Entry and Exit separately
#"Added Conditional Index" = Table.Group(#"Sorted Rows", {"Process ID", "Workstep"}, {
{"AllRows", each
let
// Separate tables for Entry and Exit, with conditional indexing
EntryTable = Table.AddIndexColumn(Table.SelectRows(_, each [status] = "Entry"), "EntryIndex", 1, 1),
ExitTable = Table.AddIndexColumn(Table.SelectRows(_, each [status] = "Exit"), "ExitIndex", 1, 1),
// Rename columns to avoid conflicts
RenamedEntryTable = Table.RenameColumns(EntryTable, {{"Process ID", "Process ID_Entry"}, {"Process datetime", "Entry datetime"}, {"status", "Entry status"}, {"Workstep", "Workstep_Entry"}}),
RenamedExitTable = Table.RenameColumns(ExitTable, {{"Process ID", "Process ID_Exit"}, {"Process datetime", "Exit datetime"}, {"status", "Exit status"}, {"Workstep", "Workstep_Exit"}}),
// Merge the two indexed tables back together using a join
Combined = Table.Join(RenamedEntryTable, {"Process ID_Entry", "Workstep_Entry"}, RenamedExitTable, {"Process ID_Exit", "Workstep_Exit"}, JoinKind.FullOuter)
in
Combined
}
}, GroupKind.Local),
// Expand the nested table to flatten the structure
#"Expanded AllRows" = Table.ExpandTableColumn(#"Added Conditional Index", "AllRows", {"Entry datetime", "Exit datetime", "EntryIndex", "ExitIndex", "Process ID_Entry", "Process ID_Exit", "Entry status", "Exit status", "Workstep_Entry", "Workstep_Exit"}, {"Entry datetime", "Exit datetime", "EntryIndex", "ExitIndex", "Process ID_Entry", "Process ID_Exit", "Entry status", "Exit status", "Workstep_Entry", "Workstep_Exit"}),
// Combine the Process ID and Workstep columns into one and remove the temporary columns
#"Added Combined Process ID" = Table.AddColumn(#"Expanded AllRows", "Process ID", each if [Process ID_Entry] <> null then [Process ID_Entry] else [Process ID_Exit]),
#"Added Combined Workstep" = Table.AddColumn(#"Added Combined Process ID", "Workstep", each if [Workstep_Entry] <> null then [Workstep_Entry] else [Workstep_Exit]),
// Remove temporary columns used for processing
#"Removed Temporary Columns" = Table.RemoveColumns(#"Added Combined Workstep", {"Process ID_Entry", "Process ID_Exit", "EntryIndex", "ExitIndex", "Entry status", "Exit status", "Workstep_Entry", "Workstep_Exit"}),
// Sort the final table by Process ID for proper organization
#"Sorted Final Rows" = Table.Sort(#"Removed Temporary Columns", {{"Process ID", Order.Ascending}})
in
#"Sorted Final Rows" - PavanLalwani
Resolver II
To achieve the desired transformation in Power Query, follow these steps:
### Steps:
1. **Sort the Data**:
- Ensure the data is sorted by **Reference No**, **Process ID**, and **Date_Time**.2. **Add a Custom Column**:
- Create a custom column in Power Query to mark each `IN` and `OUT` row.3. **Pivot the Data**:
- Select the **Workstep** and **Status** columns.
- Use the **Pivot Column** option, setting **Date_Time** as the values column.4. **Adjust the Output**:
- Rename columns as **IN** and **OUT**, and handle null values for proper representation.This process will reshape your data as expected.
- sanalytics
Super User
Hello Varan_15
Below is the pbix file of my slotuion.
Hope it helps.
Regards
sanalytics
If it is your solution then please like and accept it as solution
- saud968
Memorable Member
Try this -
To transform your data in Power Query, you can follow these steps:
Load your data into Power Query:
Open Excel and load your data into Power Query by selecting your data range and clicking on Data > From Table/Range.
Add an Index Column:
In Power Query, go to the Add Column tab and select Index Column > From 0.
Pivot the Data:
Select the Workstep column, then go to the Transform tab and click on Pivot Column.
In the Pivot Column dialog box, select status as the Values Column and Date_Time as the Values.
Fill Down the Values:
Select the columns that were created by the pivot (IN and OUT columns).
Go to the Transform tab and click on Fill > Down.
Remove Duplicates:
Select the Reference No, Barcode, Process ID, and Workstep columns.
Go to the Home tab and click on Remove Rows > Remove Duplicates.
Reorder and Clean Up Columns:
Reorder the columns as needed and remove the Index column if itโs no longer necessary.
Best Regards
Saud Ansari
If this post helps, please Accept it as a Solution to help other members find it. I appreciate your Kudos!- MSuser5
Helper III
ExtraTimeInWorkingHours =
VAR HasPending =
CALCULATE(
COUNTROWS(FILTER('Table', 'Table'[Workstep] = "Pending")),
ALLEXCEPT('Table', 'Table'[CustomerID])
) > 0
VAR HasAction =
CALCULATE(
COUNTROWS(FILTER('Table', 'Table'[Workstep] = "Action")),
ALLEXCEPT('Table', 'Table'[CustomerID])
) > 0VAR PendingHours = IF(HasPending, 9, 0) // Add 9 hours only once if "Pending" is present
VAR ActionHours = IF(HasAction, 6, 0) // Add 6 hours only once if "Action" is presentVAR TotalHours = PendingHours + ActionHours
VAR FullDaysToAdd = INT(TotalHours / ๐ // Convert total hours to full working days
VAR RemainingHours = MOD(TotalHours, ๐ // Get remaining hours that don't form a full dayRETURN
(FullDaysToAdd * ๐ + RemainingHours // Total extra hours accounting for full days and remaining hours- MSuser5
Helper III
ExtraTimeInWorkingHours =
VAR HasPending =
CALCULATE(
COUNTROWS(FILTER('Table', 'Table'[Workstep] = "Pending")),
ALLEXCEPT('Table', 'Table'[CustomerID])
) > 0
VAR HasAction =
CALCULATE(
COUNTROWS(FILTER('Table', 'Table'[Workstep] = "Action")),
ALLEXCEPT('Table', 'Table'[CustomerID])
) > 0VAR PendingHours = IF(HasPending, 9, 0) // Add 9 hours only once if "Pending" is present
VAR ActionHours = IF(HasAction, 6, 0) // Add 6 hours only once if "Action" is presentVAR TotalHours = PendingHours + ActionHours
// Calculate additional working days and hours based on the total hours
VAR AdditionalDays = INT(TotalHours / ๐ // Full days from total hours
VAR RemainingHours = MOD(TotalHours, ๐ // Remaining hours after full daysRETURN
IF(
RemainingHours > 0,
AdditionalDays + 1, // Add one more day if there's any remaining time
AdditionalDays
)
- MSuser5
Helper III
To check if the tickets are within or beyond the SLA based on the given conditions, you will need to calculate the allocated SLA time considering working hours (8 AM to 4 PM) and incorporate the additional hours for specific worksteps (like "Pending" and "Action"). Hereโs how you can approach it in Power BI DAX:
Step 1: Calculate the Base SLA Deadline
Create a calculated column or measure to calculate the SLA deadline for each ticket based on the start time:
BaseSLADeadline =
VAR StartTime = 'Table'[StartDateTime]
VAR StartDate = DATEVALUE(StartTime)
VAR SLADeadline =
IF(
HOUR(StartTime) < 11,
StartDate + TIME(16, 0, 0), // Same day 4 PM if started before 11 AM
StartDate + 1 + TIME(16, 0, 0) // Next day 4 PM if started after 11 AM
)
RETURN
SLADeadlineStep 2: Add Extra Time for Specific Worksteps
Now, you need to account for additional hours based on specific worksteps ("Pending" adds 8 hours, "Action" adds 5 hours).
Create a measure or calculated column to calculate the total additional time:
ExtraTimeInHours =
SUMX(
FILTER(
'Table',
'Table'[CustomerID] = EARLIER('Table'[CustomerID]) &&
('Table'[Workstep] = "Pending" || 'Table'[Workstep] = "Action")
),
SWITCH(
'Table'[Workstep],
"Pending", 8,
"Action", 5,
0
)
)Step 3: Calculate the SLA Deadline Including Additional Hours
You need to adjust the base SLA deadline considering working hours (8 AM to 4 PM). The additional hours should be added only during the working hours.
Create a measure to calculate the adjusted SLA deadline:
AdjustedSLADeadline =
VAR BaseDeadline = [BaseSLADeadline]
VAR ExtraHours = [ExtraTimeInHours]
VAR WorkingStartTime = TIME(8, 0, 0)
VAR WorkingEndTime = TIME(16, 0, 0)
VAR FullDaysToAdd = INT(ExtraHours / ๐
VAR RemainingHours = MOD(ExtraHours, ๐
VAR ExtraDays = IF(HOUR(BaseDeadline) + RemainingHours > 16, 1, 0)
VAR NewDate = BaseDeadline + FullDaysToAdd + ExtraDays
VAR NewTime = TIME(HOUR(BaseDeadline) + RemainingHours, MINUTE(BaseDeadline), SECOND(BaseDeadline))RETURN
IF(
NewTime > WorkingEndTime,
NewDate + 1 + WorkingStartTime,
NewDate + NewTime
)Step 4: Compare the Actual Completion Time Against the SLA Deadline
Finally, you can create a measure to check if the ticket is within or beyond the SLA:
SLAStatus =
IF(
'Table'[EndDateTime] <= [AdjustedSLADeadline],
"Within SLA",
"Beyond SLA"
)Summary
1. BaseSLADeadline calculates the initial SLA deadline based on the start time.
2. ExtraTimeInHours adds additional time for specific worksteps.
3. AdjustedSLADeadline recalculates the SLA deadline considering working hours.
4. SLAStatus compares the actual end time against the adjusted SLA deadline to check if it is within or beyond SLA.This approach ensures that SLA calculations account for working hours and the specified rules for different scenarios.
- MSuser5
Helper III
ExtraTimeInHours =
VAR HasPending =
CALCULATE(
COUNTROWS(FILTER('Table', 'Table'[Workstep] = "Pending")),
ALLEXCEPT('Table', 'Table'[CustomerID])
) > 0
VAR ActionHours =
CALCULATE(
SUMX(
FILTER('Table', 'Table'[Workstep] = "Action"),
5
),
ALLEXCEPT('Table', 'Table'[CustomerID])
)
RETURN
IF(HasPending, 8, 0) + ActionHours