Supplies are limited. Contact info@espc.tech right away to save your spot before the conference sells out.
Get your discountScore big with last-minute savings on the final tickets to FabCon Vienna. Secure your discount
Hey Team!
I have very complex problem I'm struggling to resolve. I assume there is several ways to do it. I'm looking for performance wise aproach that can handle milions of rows.
Goal: summarize real edit time (from Agent perspective)
Problem: When editing a Case system create duplicate record if agent goes from Case to WO. New row/record is created for same Case.
Duplicate: duplicate record is row with Work Order ID. At least part of the time edit is simultaneous with Case edit. When Agent end edit mode (End Time) then Case Edit is automatically ended usually 7-9 seconds later. The Work Order edit is thus duplicate and for tracking purposes we need only Case row
Solution: I tried chatgpt proposal adding new custom column that returns end time of previous Case edit (based on matching Case Number and Start by which is same agent. After I sorted data and used Buffer.Table to keep correct sorting.
#"Added End Time From Case Row" = Table.AddColumn(#"Added Index", "End Time (From Case)", each
if [Work Order] <> null then
let
CurrentCase = [#"Case Number (Case) (Case)"],
CurrentStartBy = [Start By],
// Find the first row where "Work Order" is null and Case Number and Start By match
MatchedRow = Table.SelectRows(#"Added Index", each
[#"Case Number (Case) (Case)"] = CurrentCase and
[Start By] = CurrentStartBy and
[Work Order] = null
){0}?, // Use "{0}?" to safely return the first matching row or null if no match
EndTimeValue = if MatchedRow <> null then MatchedRow[End Time] else null
in
EndTimeValue
else
null
Do you think there is better solution to this problem that is more performance friendly or in general better approach that consider row calculation based on same condition ? Thank you for your help!
Hi @MatusiSK ,
The M code you provided is not complete, so I may not know whether there is room for improvement. However, for Power Query to optimize query performance, the most effective method may still be to use Group by Function.
Table.Group - PowerQuery M | Microsoft Learn
Please check whether your data can be grouped first and then perform other data transformation operations.
Best Regards,
Dino Tao
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Case Number | Work Order | Creation Mode | Start By | End By | Start Time | End Time | Sec. Difference |
2020098993 | Manual | Agent 4 | Agent 4 | 9/2/2024 09:49:20 | 9/2/2024 10:04:11 | ||
2020098993 | Auto | Agent 4 | SYSTEM | 9/2/2024 15:15:02 | 9/2/2024 15:16:15 | ||
2020098993 | 4014643198 | Manual | Agent 4 | Agent 4 | 9/2/2024 15:15:21 | 9/2/2024 15:16:08 | 7.00 |
2020098993 | 4014643198 | Manual | Agent 4 | Agent 4 | 9/2/2024 15:16:27 | 9/2/2024 15:18:49 | |
2020098993 | Manual | Agent 4 | Agent 4 | 9/4/2024 08:48:02 | 9/4/2024 08:48:17 | ||
2020098993 | Manual | Agent 4 | Agent 4 | 9/6/2024 09:17:30 | 9/6/2024 09:22:37 | ||
2020098993 | Manual | Agent 4 | Agent 4 | 9/9/2024 09:50:50 | 9/9/2024 09:51:36 | ||
2020098993 | Manual | Agent 4 | Agent 4 | 9/11/2024 16:28:53 | 9/11/2024 16:28:57 | ||
2020099685 | Auto | Agent 5 | SYSTEM | 9/9/2024 11:26:29 | 9/9/2024 11:28:46 | ||
2020099685 | 4014736440 | Manual | Agent 5 | Agent 5 | 9/9/2024 11:26:35 | 9/9/2024 11:28:39 | 7.00 |
2020099685 | 4014736440 | Manual | Agent 5 | Agent 5 | 9/9/2024 11:29:07 | 9/9/2024 11:30:39 | |
2020100256 | Auto | Agent 5 | SYSTEM | 9/13/2024 09:21:59 | 9/13/2024 09:23:13 | ||
2020100256 | 4014816836 | Manual | Agent 5 | Agent 5 | 9/13/2024 09:22:18 | 9/13/2024 09:23:06 | 7.00 |
2020100256 | 4014816836 | Manual | Agent 5 | Agent 5 | 9/13/2024 09:23:24 | 9/13/2024 09:24:52 | |
2020102423 | Manual | Agent 6 | Agent 6 | 9/12/2024 10:50:59 | 9/12/2024 10:52:21 | ||
2020102423 | Manual | Agent 6 | Agent 6 | 9/11/2024 13:37:35 | 9/11/2024 13:38:22 | ||
2020116155 | Auto | Agent 7 | SYSTEM | 9/2/2024 11:17:36 | 9/2/2024 11:32:39 | ||
2020116155 | Auto | Agent 7 | SYSTEM | 9/2/2024 12:59:04 | 9/2/2024 12:59:43 | ||
2020116155 | 4014640968 | Manual | Agent 7 | Agent 7 | 9/2/2024 12:59:19 | 9/2/2024 12:59:36 | 7.00 |
2020116155 | 4014640968 | Manual | Agent 7 | Agent 7 | 9/2/2024 13:00:01 | 9/2/2024 13:01:01 | |
2020116155 | Auto | Agent 5 | SYSTEM | 9/11/2024 09:18:47 | 9/11/2024 09:25:20 | ||
2020116155 | 4014777495 | Manual | Agent 5 | Agent 5 | 9/11/2024 09:24:20 | 9/11/2024 09:25:11 | 9.00 |
2020116155 | 4014777495 | Manual | Agent 5 | Agent 5 | 9/11/2024 09:25:28 | 9/11/2024 09:26:54 | |
2020123991 | Manual | Agent 8 | Agent 8 | 9/2/2024 15:29:04 | 9/2/2024 15:33:36 | ||
2020123991 | Auto | Agent 8 | SYSTEM | 9/10/2024 15:50:10 | 9/10/2024 15:54:01 | ||
2020123991 | 4014761470 | Manual | Agent 8 | Agent 8 | 9/10/2024 15:52:54 | 9/10/2024 15:53:54 | 7.00 |
2020123991 | 4014761470 | Manual | Agent 8 | Agent 8 | 9/10/2024 15:54:03 | 9/10/2024 15:54:17 | |
2020143741 | Auto | Agent 6 | SYSTEM | 9/9/2024 08:32:41 | 9/9/2024 08:50:33 | ||
2020143741 | Manual | Agent 6 | Agent 6 | 9/6/2024 10:41:54 | 9/6/2024 10:43:44 | ||
2020148485 | Auto | Agent 9 | SYSTEM | 9/3/2024 16:15:44 | 9/3/2024 16:30:47 | ||
2020148485 | Auto | Agent 9 | SYSTEM | 9/3/2024 16:33:55 | 9/3/2024 17:01:57 | ||
2020148485 | Auto | Agent 9 | SYSTEM | 9/3/2024 17:15:55 | 9/3/2024 17:30:57 | ||
2020151763 | Auto | Agent 9 | SYSTEM | 9/2/2024 10:54:42 | 9/2/2024 10:55:50 | ||
2020151763 | 4014638922 | Manual | Agent 9 | Agent 9 | 9/2/2024 10:54:57 | 9/2/2024 10:55:41 | 9.00 |
2020151763 | 4014638922 | Manual | Agent 9 | Agent 9 | 9/2/2024 10:56:17 | 9/2/2024 10:56:39 | |
2020152082 | Auto | Agent 10 | SYSTEM | 9/2/2024 12:31:43 | 9/2/2024 12:32:35 | ||
2020152082 | 4014640527 | Manual | Agent 10 | Agent 10 | 9/2/2024 12:32:04 | 9/2/2024 12:32:28 | |
2020152082 | 4014640527 | Manual | Agent 10 | Agent 10 | 9/2/2024 12:32:40 | 9/2/2024 12:35:46 | |
2020152082 | 4014640527 | Manual | Agent 10 | Agent 10 | 9/2/2024 12:35:58 | 9/2/2024 12:36:11 | |
2020155768 | Auto | Agent 11 | SYSTEM | 9/11/2024 12:22:00 | 9/11/2024 12:23:22 | ||
2020155768 | 4014780830 | Manual | Agent 11 | Agent 11 | 9/11/2024 12:22:53 | 9/11/2024 12:23:15 | 7.00 |
2020155768 | 4014780830 | Manual | Agent 11 | Agent 11 | 9/11/2024 12:23:30 | 9/11/2024 12:24:03 |
Table with sample data