Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Be one of the first to start using Fabric Databases. View on-demand sessions with database experts and the Microsoft product team to learn just how easy it is to get started. Watch now

Reply
abdsk
New Member

Skipping overlapping date & get date difference.

Hi there,

I got a requirement where I need to skip overlapping dates & only consider minimum start date & max end date e.g.

One Machine started at 12-05-2023 02:00 & ended at 12-05-2023 10:30 meanwhile other machine (row no 2) started at 12-05-2023 09:30 & ended at 12-05-2023 16:00 so here 9.30 comes before the earlier machines end time (overlapping) so expected output will be 14 hrs (end time of row no 2 - start time row no 1  i.e. 12-05-2023 16:00 - 12-05-2023 02:00) but the third time when machine goes down (row no 3) is started at 17 (no overlapping) & ends at 20 so it is 3 hrs.
So similarly row no 4 & 5; where start time to be consider as 13-05-2023 10:00 & end time is 13-06-2023 10:00 as the second start date is overlapping.

 

How to get the output in hrs for such data; your help is much appreciated.

 

 Downtime  
EquipmentStart TimeEnd TimeExpected Output (hrs) 
Machine12-05-2023 02:0012-05-2023 10:300 
Machine12-05-2023 09:3012-05-2023 16:0014 
Machine12-05-2023 17:0012-05-2023 20:003 
Machine13-05-2023 10:0010-06-2023 00:000 
Machine04-06-2023 03:0013-06-2023 10:0072030 days * 24 hrs
2 ACCEPTED SOLUTIONS
ImkeF
Super User
Super User

Hello @abdsk ,
this can be achieved by the magical 5th parameter in the Table.Group-function.
If you paste the following M-code into the advanced editor of a blank query, you can follow the steps along:

 

let
  Source = Table.FromRows(
    Json.Document(
      Binary.Decompress(
        Binary.FromText(
          "i45W8k1MzsjMS1XSUTI00jcw1TcyMDJWMDCyMjAAIlRRQwMrY7BorA4ufZZQFaj6zKCm4dRnaI7NPiMD7PqMkV0E12egb2AGdQV2fQYmCBXGCH3GcFG4abGxAA==",
          BinaryEncoding.Base64
        ),
        Compression.Deflate
      )
    ),
    let
      _t = ((type nullable text) meta [Serialized.Text = true])
    in
      type table [Equipment = _t, #"Start Time" = _t, #"End Time" = _t]
  ),
  #"Changed Type" = Table.TransformColumnTypes(
    Source,
    {{"Start Time", type datetime}, {"End Time", type datetime}}
  ),
  #"Grouped Rows" = Table.Group(
    #"Changed Type",
    {"Equipment", "Start Time", "End Time"},
    {
      {"Count", each _},
      {"MinStart", each List.Min([Start Time]), type nullable datetime},
      {"MaxEnd", each List.Max([End Time]), type nullable datetime}
    },
    GroupKind.Local,
    (x, y) => Number.From(x[End Time] < y[Start Time])
  ),
  #"Inserted Time Subtraction" = Table.AddColumn(
    #"Grouped Rows",
    "Duration",
    each Duration.TotalHours([MaxEnd] - [MinStart])
  )
in
  #"Inserted Time Subtraction"

 

 

Imke Feldmann (The BIccountant)

If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!

How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries

View solution in original post

ImkeF
Super User
Super User

Hi @abdsk ,
unfortunately this is not so easy any more, as it has a different logic than I originally thought.
Please use this code instead:

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("xdNBDsIwDATAr6Ccg+S4LbS8BKj6/2/UvsGu1hInpJyckRPbyb635+vderNxjeXmfjF7mGXoBqGjV3r5SU9ab6THXerBuSuNx9YaS6o1bpXajbTPWjtrPR3nKnM6LjQeG6FV58Zje5uwlA/NPcnHoG7Ck883KDRtRb9N65W1a72R9v9oKilCs9aYKEK6g/kbUGObvn8D3pv6fZw=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Equipment = _t, #"Start Time" = _t, #"End Time" = _t]),
ChangedType = Table.TransformColumnTypes(
    Source, 
    {{"Equipment", type text}, {"Start Time", type datetime}, {"End Time", type datetime}}
  ), 
  #"Grouped Rows" = Table.Group(
    ChangedType, 
    {"Equipment"}, 
    {
      {
        "Result", 
        each 
          let
            InputTable = Table.Buffer(Table.SelectColumns(_, {"Start Time", "End Time"})), 
            Custom1 = List.Generate(
              () => [
                MinStart = InputTable{0}[Start Time], 
                MaxEnd   = InputTable{0}[End Time], 
                Group    = 1, 
                Counter  = 0
              ], 
              each [Counter] <= Table.RowCount(InputTable), 
              each [
                CurrentStart = InputTable{[Counter]}[Start Time], 
                CurrentEnd = InputTable{[Counter]}[End Time], 
                GroupChange = CurrentStart > [MaxEnd] and CurrentEnd > [MinStart], 
                MinStart = 
                  if GroupChange then
                    CurrentStart
                  else
                    List.Min({CurrentStart, [MinStart]}), 
                MaxEnd = if GroupChange then CurrentEnd else List.Max({CurrentEnd, [MaxEnd]}), 
                Group = if GroupChange then [Group] + 1 else [Group], 
                Counter = [Counter] + 1
              ]
            ), 
            #"Converted to Table" = Table.FromList(
              Custom1, 
              Splitter.SplitByNothing(), 
              null, 
              null, 
              ExtraValues.Error
            ), 
            #"Expanded Column1" = Table.ExpandRecordColumn(
              #"Converted to Table", 
              "Column1", 
              {"MinStart", "MaxEnd", "Group"}
            ), 
            #"Grouped Rows" = Table.Group(
              #"Expanded Column1", 
              {"Group"}, 
              {
                {"MinStart", each List.Min([MinStart]), type datetime}, 
                {"MaxEnd", each List.Max([MaxEnd]), type datetime}
              }
            )
          in
            #"Grouped Rows"
      }
    }
  ), 
  #"Expanded Result" = Table.ExpandTableColumn(
    #"Grouped Rows", 
    "Result", 
    {"MinStart", "MaxEnd"}
  )
in
  #"Expanded Result"

Imke Feldmann (The BIccountant)

If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!

How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries

View solution in original post

7 REPLIES 7
ImkeF
Super User
Super User

Hi @abdsk ,
unfortunately this is not so easy any more, as it has a different logic than I originally thought.
Please use this code instead:

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("xdNBDsIwDATAr6Ccg+S4LbS8BKj6/2/UvsGu1hInpJyckRPbyb635+vderNxjeXmfjF7mGXoBqGjV3r5SU9ab6THXerBuSuNx9YaS6o1bpXajbTPWjtrPR3nKnM6LjQeG6FV58Zje5uwlA/NPcnHoG7Ck883KDRtRb9N65W1a72R9v9oKilCs9aYKEK6g/kbUGObvn8D3pv6fZw=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Equipment = _t, #"Start Time" = _t, #"End Time" = _t]),
ChangedType = Table.TransformColumnTypes(
    Source, 
    {{"Equipment", type text}, {"Start Time", type datetime}, {"End Time", type datetime}}
  ), 
  #"Grouped Rows" = Table.Group(
    ChangedType, 
    {"Equipment"}, 
    {
      {
        "Result", 
        each 
          let
            InputTable = Table.Buffer(Table.SelectColumns(_, {"Start Time", "End Time"})), 
            Custom1 = List.Generate(
              () => [
                MinStart = InputTable{0}[Start Time], 
                MaxEnd   = InputTable{0}[End Time], 
                Group    = 1, 
                Counter  = 0
              ], 
              each [Counter] <= Table.RowCount(InputTable), 
              each [
                CurrentStart = InputTable{[Counter]}[Start Time], 
                CurrentEnd = InputTable{[Counter]}[End Time], 
                GroupChange = CurrentStart > [MaxEnd] and CurrentEnd > [MinStart], 
                MinStart = 
                  if GroupChange then
                    CurrentStart
                  else
                    List.Min({CurrentStart, [MinStart]}), 
                MaxEnd = if GroupChange then CurrentEnd else List.Max({CurrentEnd, [MaxEnd]}), 
                Group = if GroupChange then [Group] + 1 else [Group], 
                Counter = [Counter] + 1
              ]
            ), 
            #"Converted to Table" = Table.FromList(
              Custom1, 
              Splitter.SplitByNothing(), 
              null, 
              null, 
              ExtraValues.Error
            ), 
            #"Expanded Column1" = Table.ExpandRecordColumn(
              #"Converted to Table", 
              "Column1", 
              {"MinStart", "MaxEnd", "Group"}
            ), 
            #"Grouped Rows" = Table.Group(
              #"Expanded Column1", 
              {"Group"}, 
              {
                {"MinStart", each List.Min([MinStart]), type datetime}, 
                {"MaxEnd", each List.Max([MaxEnd]), type datetime}
              }
            )
          in
            #"Grouped Rows"
      }
    }
  ), 
  #"Expanded Result" = Table.ExpandTableColumn(
    #"Grouped Rows", 
    "Result", 
    {"MinStart", "MaxEnd"}
  )
in
  #"Expanded Result"

Imke Feldmann (The BIccountant)

If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!

How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries

Thanks again @ImkeF; though difficult to sallow but gives the expected result.

ImkeF
Super User
Super User

Hi @abdsk ,
please give the desired output as a table form and open a new thread for the 2nd topic.
I'm not sure if I have time to look into it, so someone else might pick it up instead.

Imke Feldmann (The BIccountant)

If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!

How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries

Thanks for reply @ImkeF,
here is the expected output

EquipmentMinStartMaxEnd
XYZ01-01-2022 00:00:0006-01-2022 00:00:00
XYZ09-01-2022 00:00:0017-01-2022 00:00:00
XYZ20-01-2022 00:00:0020-02-2022 00:00:00
abdsk
New Member

Hey Hi @ImkeF ,
 
The above logic is working for most of the data but failing in some scenarios.
 
PFB PQ FYR 
 
=========================================
 
let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("xdNBDsIwDATAr6Ccg+S4LbS8BKj6/2/UvsGu1hInpJyckRPbyb635+vderNxjeXmfjF7mGXoBqGjV3r5SU9ab6THXerBuSuNx9YaS6o1bpXajbTPWjtrPR3nKnM6LjQeG6FV58Zje5uwlA/NPcnHoG7Ck883KDRtRb9N65W1a72R9v9oKilCs9aYKEK6g/kbUGObvn8D3pv6fZw=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Equipment = _t, #"Start Time" = _t, #"End Time" = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"Equipment", type text}, {"Start Time", type datetime}, {"End Time", type datetime}}),
  #"Grouped Rows" = Table.Group(
    #"Changed Type",
    {"Equipment", "Start Time", "End Time"},
    {
      {"Count", each _},
      {"MinStart", each List.Min([Start Time]), type nullable datetime},
      {"MaxEnd", each List.Max([End Time]), type nullable datetime}
    },
    GroupKind.Local,
    (x, y) => Number.From(x[End Time] < y[Start Time])
  )
in
    #"Grouped Rows"
 
=========================================
 
Here the output should be 3 rows, the last row should be
20-1-2023 & 20-2-2023
as the other dates are coming in between.
 
I have another requirement where I need to calculate the uptime (above is downtime)
 
Data is,
 
Equipment Date Reading (Hrs)
XYZ 07-01-2022 24345
XYZ 12-03-2022 53464
 
Here the output should be 29119 (53464 - 24345)
 
but the output should be grouped based on the month
so, 29119 should be divided by number of days available between downtime dates
e.g. downtime was from 01-01-2022 to 06-01-2022 then 9-1-2022 to 17-01-2022 and finally from 20-1-2022 to 20-2-2022
so in the Jan we have only 4 days (7, 8, 18, 19, rest other day it was downtime)
Feb we have 8 days (as downtime till 20-2-2022)
March its 12 days
 
so total is 29119/24hrs * 24 days (4+8+12 days) = 50.55381944 per day
 
so final output should be,
 
Jan 4 202.2152778 (4* 50.55381944)
Feb 8 404.4305556
Mar 12 606.6458333
 
Your help is much appreciated
abdsk
New Member

Thank you so much @ImkeF, this is really next level; really appreciate the help.

ImkeF
Super User
Super User

Hello @abdsk ,
this can be achieved by the magical 5th parameter in the Table.Group-function.
If you paste the following M-code into the advanced editor of a blank query, you can follow the steps along:

 

let
  Source = Table.FromRows(
    Json.Document(
      Binary.Decompress(
        Binary.FromText(
          "i45W8k1MzsjMS1XSUTI00jcw1TcyMDJWMDCyMjAAIlRRQwMrY7BorA4ufZZQFaj6zKCm4dRnaI7NPiMD7PqMkV0E12egb2AGdQV2fQYmCBXGCH3GcFG4abGxAA==",
          BinaryEncoding.Base64
        ),
        Compression.Deflate
      )
    ),
    let
      _t = ((type nullable text) meta [Serialized.Text = true])
    in
      type table [Equipment = _t, #"Start Time" = _t, #"End Time" = _t]
  ),
  #"Changed Type" = Table.TransformColumnTypes(
    Source,
    {{"Start Time", type datetime}, {"End Time", type datetime}}
  ),
  #"Grouped Rows" = Table.Group(
    #"Changed Type",
    {"Equipment", "Start Time", "End Time"},
    {
      {"Count", each _},
      {"MinStart", each List.Min([Start Time]), type nullable datetime},
      {"MaxEnd", each List.Max([End Time]), type nullable datetime}
    },
    GroupKind.Local,
    (x, y) => Number.From(x[End Time] < y[Start Time])
  ),
  #"Inserted Time Subtraction" = Table.AddColumn(
    #"Grouped Rows",
    "Duration",
    each Duration.TotalHours([MaxEnd] - [MinStart])
  )
in
  #"Inserted Time Subtraction"

 

 

Imke Feldmann (The BIccountant)

If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!

How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries

Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!

Dec Fabric Community Survey

We want your feedback!

Your insights matter. That’s why we created a quick survey to learn about your experience finding answers to technical questions.

ArunFabCon

Microsoft Fabric Community Conference 2025

Arun Ulag shares exciting details about the Microsoft Fabric Conference 2025, which will be held in Las Vegas, NV.

December 2024

A Year in Review - December 2024

Find out what content was popular in the Fabric community during 2024.