Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago
Solved

Write update query in power query

Hi,

 

I am new to PowerBI & Powerquery and I need some help.

 

I have below table as source: 

 

job_idstep_idrun_daterun_timeduration
09F901AD-0B49-48A7020240723160000165051
09F901AD-0B49-48A7120240723160001338
09F901AD-0B49-48A7220240723160339164711
10O901XT-1234-4PO80202407306000029
10O901XT-1234-4PO81202407306000028

 

Table contains all job steps (can be different steps for each job).

when step_id = 0, it's the job itselve, in this case value of duration refers to complete duration including all steps of this schedule.

What is missing in this table is the schedule_id, I have no link between all steps for 1 specific job.

So I would like to create this new column to have below result:

 

job_idstep_idrun_daterun_timedurationschedule_id
09F901AD-0B49-48A702024072316000016505120240723160000
09F901AD-0B49-48A712024072316000133820240723160000
09F901AD-0B49-48A722024072316033916471120240723160000
10O901XT-1234-4PO8020240730600002920240730060000
10O901XT-1234-4PO8120240730600002820240730060000

 

Column schedule_id  is the run_date & run_time of step_id = 0 of that schedule. So I need to start from records with step_id = 0 and retrieve all steps > 0 for that job_id where run_datetime is between start_rundatetime and end_rundatetime.

=> start_rundatetime = rundate & runtime of step 0

=> end_rundatetime = rundate & runtime + duration of step 0

 

Thanx,

 

Regards

 

 

  • dufoq3's avatar
    dufoq3
    2 years ago

    Hi, I haven't used your logic, but result is the same as yours. Let me know if it works with real data.

     

    Output:

     

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("nZLBCsMgEET/xXOE3VlT12NK6TU99FAI+f/fqBFKqKklqyAMwmMeg8viKN0T8XTzdA3JB52iGxzlC0KgCMmRL5RPCSON7NahwfEPbnsT0TaEAySSSgiR/5TVkrpLwiKpPZK6SyaDJKhvycJZJTPUtSTQt2ThzJI4Lsk0Z+719AwJPjxm/ZKULX6+JFIb4Qaip1rY3sLWFiVzS4Wca4G9BVXL+gY=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [job_id = _t, step_id = _t, run_date = _t, run_time = _t, duration = _t]),
        ChangedType = Table.TransformColumnTypes(Source,{{"step_id", Int64.Type}, {"run_date", Int64.Type}, {"run_time", Int64.Type}, {"duration", Int64.Type}}),
        GroupedRows = Table.Group(ChangedType, {"job_id", "step_id"}, {{"All", each 
            [ a = Table.SelectRows(_, (x)=> x[step_id] = 0){0}?,
              b = Table.AddColumn(_, "schedule_id", (x)=> Text.From(a[run_date]?) & Text.PadStart(Text.From(a[run_time]?), 6, "0"), type text)
            ][b], type table}}, GroupKind.Local,
            (x,y)=> Number.From(y[step_id] = 0) ),
        Combined = Table.Combine(GroupedRows[All])
    in
        Combined

     

6 Replies

  •  

    where run_datetime is between start_rundatetime and end_rundatetime.

     

    There is no end_rundatetime column in your sample data. Please provide sample data that fully covers your issue.
    Please show the expected outcome based on the sample data you provided.

     

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("dY49CoAwDEbv0tnClx9tMyriWgcHQbz/NYyKg2gDgUfgkbdtATYZqB8jBrWouU+hCfBlsCKxOFIHnwtatBT2puLRj3feRHJd4o8kYhdoovsZobi3LpFYNOpc8itSTnwa2eoKVRSv2w8=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [job_id = _t, step_id = _t, run_date = _t, run_time = _t, duration = _t]),
        #"Grouped Rows" = Table.Group(Source, {"job_id"}, {{"Rows", each _, type table [job_id=nullable text, step_id=nullable text, run_date=nullable text, run_time=nullable text, duration=nullable text]}, {"schedule_id", each List.Min([run_date])&List.Min([run_time]), type nullable text}}),
        #"Expanded Rows" = Table.ExpandTableColumn(#"Grouped Rows", "Rows", {"step_id", "run_date", "run_time", "duration"}, {"step_id", "run_date", "run_time", "duration"})
    in
        #"Expanded Rows"

    How to use this code: Create a new Blank Query. Click on "Advanced Editor". Replace the code in the window with the code provided here. Click "Done". Once you examined the code, replace the Source step with your own source.

    • Anonymous's avatar
      Anonymous
      Not applicable

      lbendlin  dufoq3  thanks for the code. It's working.

      In my case my table contains all job runs, my initial example was a bit too simple I see now. Here is another example with multiple runs for a job, I already included the result column schedule_id

       

      job_idstep_idrun_daterun_timedurationschedule_id
      09F901AD-0B49-48A702024072316000016505120240723160000
      09F901AD-0B49-48A712024072316000133820240723160000
      09F901AD-0B49-48A722024072316033916471120240723160000
      09F901AD-0B49-48A702024072318000016525120240723180000
      09F901AD-0B49-48A712024072318000133820240723180000
      09F901AD-0B49-48A722024072318033916491120240723180000
      09F901AD-0B49-48A702024072320000016505120240723200000
      09F901AD-0B49-48A712024072320000133820240723200000
      09F901AD-0B49-48A722024072320033916471120240723200000
      09F901AD-0B49-48A702024072322000016525120240723220000
      09F901AD-0B49-48A712024072322000133820240723220000
      09F901AD-0B49-48A722024072322033916491120240723220000
      10O901XT-1234-4PO8020240730600002920240730060000
      10O901XT-1234-4PO8120240730600002820240730060000
      10O901XT-1234-4PO8020240731600002920240731060000
      10O901XT-1234-4PO8120240731600002820240731060000
      10O901XT-1234-4PO8020240801600002920240801060000
      10O901XT-1234-4PO8120240801600002820240801060000
      10O901XT-1234-4PO8020240802600002920240802060000
      10O901XT-1234-4PO8120240802600002820240802060000

       

      startrun_time & endrun_time a jobstep can be calculated as follows: 

          startrun_time => run_date & run_time

          endrun_time => run_date & run_time + duration

      format of time is HHMMSS

      format of duration is HHMMSS (165051 means 16 hours 50 minuntes 51 seconds and 339 means 3 minutes and 39 seconds)

      Step 0 is the job itselve so in case of 1st jobrun of 09F901AD-0B49-48A7, step 1 & 2 falls between 20240723160000 (startrun_time step 0) & 20240724085051 (endrun_time step 0).

       

      Do you guys have any idea how to retrieve the schedule_id in these cases?

      • dufoq3's avatar
        dufoq3
        Community Champion

        Hi, I haven't used your logic, but result is the same as yours. Let me know if it works with real data.

         

        Output:

         

        let
            Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("nZLBCsMgEET/xXOE3VlT12NK6TU99FAI+f/fqBFKqKklqyAMwmMeg8viKN0T8XTzdA3JB52iGxzlC0KgCMmRL5RPCSON7NahwfEPbnsT0TaEAySSSgiR/5TVkrpLwiKpPZK6SyaDJKhvycJZJTPUtSTQt2ThzJI4Lsk0Z+719AwJPjxm/ZKULX6+JFIb4Qaip1rY3sLWFiVzS4Wca4G9BVXL+gY=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [job_id = _t, step_id = _t, run_date = _t, run_time = _t, duration = _t]),
            ChangedType = Table.TransformColumnTypes(Source,{{"step_id", Int64.Type}, {"run_date", Int64.Type}, {"run_time", Int64.Type}, {"duration", Int64.Type}}),
            GroupedRows = Table.Group(ChangedType, {"job_id", "step_id"}, {{"All", each 
                [ a = Table.SelectRows(_, (x)=> x[step_id] = 0){0}?,
                  b = Table.AddColumn(_, "schedule_id", (x)=> Text.From(a[run_date]?) & Text.PadStart(Text.From(a[run_time]?), 6, "0"), type text)
                ][b], type table}}, GroupKind.Local,
                (x,y)=> Number.From(y[step_id] = 0) ),
            Combined = Table.Combine(GroupedRows[All])
        in
            Combined

         

  • dufoq3's avatar
    dufoq3
    Community Champion

    Hi Anonymous, check this:

     

    Output

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("dY49CoAwDEbv0tnClx9tMyriWgcHQbz/NYyKg2gDgUfgkbdtATYZqB8jBrWouU+hCfBlsCKxOFIHnwtatBT2puLRj3feRHJd4o8kYhdoovsZobi3LpFYNOpc8itSTnwa2eoKVRSv2w8=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [job_id = _t, step_id = _t, run_date = _t, run_time = _t, duration = _t]),
        ChangedType = Table.TransformColumnTypes(Source,{{"step_id", Int64.Type}, {"run_date", Int64.Type}, {"run_time", Int64.Type}, {"duration", Int64.Type}}),
        GroupedRows = Table.Group(ChangedType, {"job_id"}, {{"All", each 
            [ a = Table.SelectRows(_, (x)=> x[step_id] = 0){0}?,
              b = Table.AddColumn(_, "schedule_id", (x)=> Text.From(a[run_date]?) & Text.From(a[run_time]?), type text)
            ][b], type table}}),
        Combined = Table.Combine(GroupedRows[All])
    in
        Combined