Forum Discussion

ivec's avatar
ivec
New Member
1 year ago
Solved

Help Optimize Query: Finding matching Answer in a log of Queries and Answers

Hi,
Being very new and naive to Power Query, I've managed to create a query that works, but that ends up being unacceptably slow.   I'm sure there is a much better way, and I would appreciate your guidance !

 

Context: my source table contains a log of events, where the main columns are:

  1. [QueryID] - a string than identifies a transaction/query
  2. [Status] - The type of log entry associated with the Query, one of "Opened" (=request issued) or "Answered" or "Closed" (= responses to the request ... "Closed" is for now equivalent to "Answered").
  3. [Changed At] - a timestamp

Small complication: within a single [QueryID] there can be more than one request "Opened" and then "Answered".

Goal: I want to extract a list of recorded transactions (opened & answered), along with the time it took for the transaction to be answered.

 

Step-by-step, I have manged to write the following query which delivers the correct result, but gets inacceptably slow when confronted with large data sets:

= Table.AddColumn(#"Sorted Rows", "End At", each if [Status] <> "Opened" then "--" else
let qq=[QueryID], minT=[Changed At]
in List.Min( Table.Column( Table.SelectRows(#"Sorted Rows", each ( [QueryID]=qq and ([Status]="Answered" or [Status]="Closed") and [Changed At]>=minT ) ), "Changed At") ) )

For all "Opened" message entries, this will find a corresponding "Answered" entry (the closest subsequent entry in chronological order), retrieve its timestamp, and store it in a new colum "End At".  This way I will get a list of all the intervals.

Of course this gets really slow for large logs.  I can think of ways to optimize this by creating separate intermediate tables and queries.  But I strongly suspect there is a much more straightforward way to perform the above operation, which may avoid manual optimization...
Can somebody help, and teach me proper tricks ?

 

4 Replies

  • v-menakakota's avatar
    v-menakakota
    Community Support

    Hi ivec ,

    I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please accept it as a solution  so other members can easily find it.

    Thank you.

     

    • v-menakakota's avatar
      v-menakakota
      Community Support

      Hi ivec ,

      Thank you lbendlin  for providing possible solution.I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.

      Thank you.

       

      • v-menakakota's avatar
        v-menakakota
        Community Support

        Hi ivec ,


        I hope this information is helpful. Please let me know if you have any further questions or if you'd like to discuss this further. If this answers your question, please Accept it as a solution and give it a 'Kudos' so others can find it easily.

        Thank you.