Forum Discussion

AMONT's avatar
AMONT
Frequent Visitor
1 year ago
Solved

Filtering Consecutive Dates by a Measure

Apologies if this has been posted before, as I do see a lot of consecutive date topics here. 

 

I have groundwater well data that has been assigned a 1 or a 0, depending on if certain parameters are met ([Total Days with 3 Exceed Ref] on a give day ([DateOnly]), and while I can get the total days that received a 1, I need to know a few other measurements.

 

Q1) How many times did the groundwater well exceed the value over time?

Q2) How long did those times last?

Q3) The longest of those times?

 

In the example below, a well exceed a reference level 264 times total. One of those times was between 5/21/2024 and 6/16/2024. However, on 6/17/2024, the well level no longer met criteria. In this example, it would have met the criteria for 27 days. It exceeded again on 6/24/2024 and this lasted until 10/24/2024, so it met criteria again for 119 days. This occurs again in the timeframe as follows in the table. The ideal output would be a table like below, with the start date and end date, the total days that were consecutive, and how often that occurred. Example data is pasted at the bottom should you want to work with it.

 

STARTENDCOUNT OF DAYSINSTANCE
5/21/20246/16/2024271
6/24/202410/24/20241191
11/8/202412/4/2024271
12/30/20241/2/202541
1/2/20252/9/2025161
2/14/20253/1/2025161
3/6/20253/6/202511
3/10/20254/14/2025361
5/11/20255/28/2025181
  2649

 

 

RAW DATA: 

DateOnlyTotal Days With 3 Exceed Ref
5/20/2024 0:000
5/21/2024 0:001
5/22/2024 0:001
5/23/2024 0:001
5/24/2024 0:001
5/25/2024 0:001
5/26/2024 0:001
5/27/2024 0:001
5/28/2024 0:001
5/29/2024 0:001
5/30/2024 0:001
5/31/2024 0:001
6/1/2024 0:001
6/2/2024 0:001
6/3/2024 0:001
6/4/2024 0:001
6/5/2024 0:001
6/6/2024 0:001
6/7/2024 0:001
6/8/2024 0:001
6/9/2024 0:001
6/10/2024 0:001
6/11/2024 0:001
6/12/2024 0:001
6/13/2024 0:001
6/14/2024 0:001
6/15/2024 0:001
6/16/2024 0:001
6/17/2024 0:000
6/18/2024 0:000
6/19/2024 0:000
6/20/2024 0:000
6/21/2024 0:000
6/22/2024 0:000
6/23/2024 0:000
6/24/2024 0:001
6/25/2024 0:001
6/26/2024 0:001
6/27/2024 0:001
6/28/2024 0:001
6/29/2024 0:001
6/30/2024 0:001
7/1/2024 0:001
7/2/2024 0:001
7/3/2024 0:001
7/4/2024 0:001
7/5/2024 0:001
7/6/2024 0:001
7/7/2024 0:001
7/8/2024 0:001
7/9/2024 0:001
7/10/2024 0:001
7/11/2024 0:001
7/12/2024 0:001
7/13/2024 0:001
7/14/2024 0:001
7/15/2024 0:001
7/16/2024 0:001
7/17/2024 0:001
7/18/2024 0:001
7/19/2024 0:001
7/20/2024 0:001
7/21/2024 0:001
7/22/2024 0:001
7/23/2024 0:001
7/24/2024 0:001
7/25/2024 0:001
7/26/2024 0:001
7/27/2024 0:001
7/28/2024 0:001
7/29/2024 0:001
7/30/2024 0:001
7/31/2024 0:001
8/1/2024 0:001
8/2/2024 0:001
8/3/2024 0:001
8/4/2024 0:001
8/5/2024 0:001
8/6/2024 0:001
8/7/2024 0:001
8/8/2024 0:001
8/9/2024 0:001
8/10/2024 0:001
8/11/2024 0:001
8/12/2024 0:001
8/13/2024 0:001
8/14/2024 0:001
8/15/2024 0:001
8/16/2024 0:001
8/17/2024 0:001
8/18/2024 0:001
8/19/2024 0:001
8/20/2024 0:001
8/21/2024 0:001
8/22/2024 0:001
8/23/2024 0:001
8/24/2024 0:001
8/25/2024 0:001
8/26/2024 0:001
8/27/2024 0:001
8/28/2024 0:001
8/29/2024 0:001
8/30/2024 0:001
8/31/2024 0:001
9/1/2024 0:001
9/2/2024 0:001
9/3/2024 0:001
9/4/2024 0:001
9/5/2024 0:001
9/6/2024 0:001
9/7/2024 0:001
9/8/2024 0:001
9/9/2024 0:001
9/10/2024 0:001
9/11/2024 0:001
9/12/2024 0:001
9/13/2024 0:001
9/14/2024 0:001
9/15/2024 0:001
9/16/2024 0:001
9/17/2024 0:001
9/18/2024 0:001
9/19/2024 0:001
9/20/2024 0:001
9/21/2024 0:001
9/22/2024 0:001
9/23/2024 0:001
9/24/2024 0:001
9/25/2024 0:001
9/26/2024 0:001
9/27/2024 0:001
9/28/2024 0:001
9/29/2024 0:001
9/30/2024 0:001
10/1/2024 0:001
10/2/2024 0:001
10/3/2024 0:001
10/4/2024 0:001
10/5/2024 0:001
10/6/2024 0:001
10/7/2024 0:001
10/8/2024 0:001
10/9/2024 0:001
10/10/2024 0:001
10/11/2024 0:001
10/12/2024 0:001
10/13/2024 0:001
10/14/2024 0:001
10/15/2024 0:001
10/16/2024 0:001
10/17/2024 0:001
10/18/2024 0:001
10/19/2024 0:001
10/20/2024 0:001
10/21/2024 0:000
10/22/2024 0:000
10/23/2024 0:000
10/24/2024 0:000
10/25/2024 0:000
10/26/2024 0:000
10/27/2024 0:000
10/28/2024 0:000
10/29/2024 0:000
10/30/2024 0:000
10/31/2024 0:000
11/1/2024 0:000
11/2/2024 0:000
11/3/2024 0:000
11/4/2024 0:000
11/5/2024 0:000
11/6/2024 0:000
11/7/2024 0:000
11/8/2024 0:001
11/9/2024 0:001
11/10/2024 0:001
11/11/2024 0:001
11/12/2024 0:001
11/13/2024 0:001
11/14/2024 0:001
11/15/2024 0:001
11/16/2024 0:001
11/17/2024 0:001
11/18/2024 0:001
11/19/2024 0:001
11/20/2024 0:001
11/21/2024 0:001
11/22/2024 0:001
11/23/2024 0:001
11/24/2024 0:001
11/25/2024 0:001
11/26/2024 0:001
11/27/2024 0:001
11/28/2024 0:001
11/29/2024 0:001
11/30/2024 0:001
12/1/2024 0:001
12/2/2024 0:001
12/3/2024 0:001
12/4/2024 0:001
12/5/2024 0:000
12/6/2024 0:000
12/7/2024 0:000
12/8/2024 0:000
12/9/2024 0:000
12/10/2024 0:000
12/11/2024 0:000
12/12/2024 0:000
12/13/2024 0:000
12/14/2024 0:000
12/15/2024 0:000
12/16/2024 0:000
12/17/2024 0:000
12/18/2024 0:000
12/19/2024 0:000
12/20/2024 0:000
12/21/2024 0:000
12/22/2024 0:000
12/23/2024 0:000
12/24/2024 0:000
12/25/2024 0:000
12/26/2024 0:000
12/27/2024 0:000
12/28/2024 0:000
12/29/2024 0:000
12/30/2024 0:001
12/31/2024 0:001
1/1/2025 0:001
1/2/2025 0:001
1/3/2025 0:000
1/4/2025 0:000
1/5/2025 0:000
1/6/2025 0:000
1/7/2025 0:000
1/8/2025 0:000
1/9/2025 0:000
1/10/2025 0:000
1/11/2025 0:000
1/12/2025 0:000
1/13/2025 0:000
1/14/2025 0:000
1/15/2025 0:000
1/16/2025 0:000
1/17/2025 0:000
1/18/2025 0:000
1/19/2025 0:000
1/20/2025 0:000
1/21/2025 0:000
1/22/2025 0:000
1/23/2025 0:000
1/24/2025 0:000
1/25/2025 0:001
1/26/2025 0:001
1/27/2025 0:001
1/28/2025 0:001
1/29/2025 0:001
1/30/2025 0:001
1/31/2025 0:001
2/1/2025 0:001
2/2/2025 0:001
2/3/2025 0:001
2/4/2025 0:001
2/5/2025 0:001
2/6/2025 0:001
2/7/2025 0:001
2/8/2025 0:001
2/9/2025 0:001
2/10/2025 0:000
2/11/2025 0:000
2/12/2025 0:000
2/13/2025 0:000
2/14/2025 0:001
2/15/2025 0:001
2/16/2025 0:001
2/17/2025 0:001
2/18/2025 0:001
2/19/2025 0:001
2/20/2025 0:001
2/21/2025 0:001
2/22/2025 0:001
2/23/2025 0:001
2/24/2025 0:001
2/25/2025 0:001
2/26/2025 0:001
2/27/2025 0:001
2/28/2025 0:001
3/1/2025 0:001
3/2/2025 0:000
3/3/2025 0:000
3/4/2025 0:000
3/5/2025 0:000
3/6/2025 0:001
3/7/2025 0:000
3/8/2025 0:000
3/9/2025 0:000
3/10/2025 0:001
3/11/2025 0:001
3/12/2025 0:001
3/13/2025 0:001
3/14/2025 0:001
3/15/2025 0:001
3/16/2025 0:001
3/17/2025 0:001
3/18/2025 0:001
3/19/2025 0:001
3/20/2025 0:001
3/21/2025 0:001
3/22/2025 0:001
3/23/2025 0:001
3/24/2025 0:001
3/25/2025 0:001
3/26/2025 0:001
3/27/2025 0:001
3/28/2025 0:001
3/29/2025 0:001
3/30/2025 0:001
3/31/2025 0:001
4/1/2025 0:001
4/2/2025 0:001
4/3/2025 0:001
4/4/2025 0:001
4/5/2025 0:001
4/6/2025 0:001
4/7/2025 0:001
4/8/2025 0:001
4/9/2025 0:001
4/10/2025 0:001
4/11/2025 0:001
4/12/2025 0:001
4/13/2025 0:001
4/14/2025 0:001
4/15/2025 0:000
4/16/2025 0:000
4/17/2025 0:000
4/18/2025 0:000
4/19/2025 0:000
4/20/2025 0:000
4/21/2025 0:000
4/22/2025 0:000
4/23/2025 0:000
4/24/2025 0:000
4/25/2025 0:000
4/26/2025 0:000
4/27/2025 0:000
4/28/2025 0:000
4/29/2025 0:000
4/30/2025 0:000
5/1/2025 0:000
5/2/2025 0:000
5/3/2025 0:000
5/4/2025 0:000
5/5/2025 0:000
5/6/2025 0:000
5/7/2025 0:000
5/8/2025 0:000
5/9/2025 0:000
5/10/2025 0:000
5/11/2025 0:001
5/12/2025 0:001
5/13/2025 0:001
5/14/2025 0:001
5/15/2025 0:001
5/16/2025 0:001
5/17/2025 0:001
5/18/2025 0:001
5/19/2025 0:001
5/20/2025 0:001
5/21/2025 0:001
5/22/2025 0:001
5/23/2025 0:001
5/24/2025 0:001
5/25/2025 0:001
5/26/2025 0:001
5/27/2025 0:001
5/28/2025 0:001
5/29/2025 0:000

 

 

  • hello AMONT 

     

    Glad it is worked as intended.

     

    if you need to limit certain id, you can add more condition inside the filter.

    for example: 

    MAXX(
        FILTER(
            ALL('Table'),
            'Table'[DateOnly]<_Select&&
    'Table'[Well_ID]=_ID *** adding at top var _ID = SELECTEDVALUE('Table'[Well_ID])***
        ),
        'Table'[DateOnly]
    )

    or directly use should have same result : 

    MAXX(
        FILTER(
            ALL('Table'),
            'Table'[DateOnly]<_Select&&
    'Table'[Well_ID]=SELECTEDVALUE('Table'[Well_ID])
        ),
        'Table'[DateOnly]
    )

    This way, the measure only calculate for previous date AND for same ID.

    you can add more filter condition as you need as above.

     

    Hope this will help.
    Thank you.

11 Replies

  • Irwan's avatar
    Irwan
    Super User

    hello AMONT 

     

    i am not sure but the second value is written 6/24/24-10/24/24, but from your data it should be 6/24/24-10/20/24

     

    Regardless, please check if this accomodate your need.

     

    you can do this with PQ or DAX. 

    - using PQ

    let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("ddhLjhwhEATQq1iztkSTQAG+iuX7X8OFPz1y+aU0q9j0RNXrAPX37x+jxOv+i/7l9e31+vj68fr48fVXXP+J6984HDfH3fFwfDmejpfjzbi9HKvlVZyq+lXU/CoqfhX1vopqX0Wtr6LSV1Hnu4U633HS2fWq+1UXrG5YXbFO0Lvj5Xgzpt/r6fcdh+Pm2C3p97Lfy34v+73s97LfSaiTUCehTkKdhDoJdRLqJNRJqNNQp6FOQ52GOg11Guo01PmE+o5dsrpluCWHdnpop4d2Guo01Gmo01CnoU5DnRlUDu2i30W/i34X/S76XfS76HfR76LfZb/Lfpf9Lvtd9rvsd9nvst9lv8t+l/0u+132u+x32e+y32W/y36X/S77Xfa77HfT76bfTb+bfjf9bvrd9Lvpd9Pvtt9tv9t+t/1u+932u+132++2322/2363/W773fa77Xfb77bfbb/bfrf9bvu9X7BanvfuWC3vWC3vWC3vWC3vWC3vWC3vWC1Pnaxm0pNoT540JduTJ10J9+RJW9I9edKXeE+ua+7Jdc89uS66J+9JPpL8SnJd6U+uO/3Jdak/AHWrPzn71pLEfAq18CHUwmdQCx9BLXwCtfABVPOu5l0T3jXhXRPeNeFdE9414V0T3jXhXRPeNeFdE97V43zypC/n+eRJXw70yZO+nOiTJ3050gcg+4ZXOrzS4ZUOr3SYcZhxmHEUfo2j8FscT8afOb+u8WT8mfMLG0/Gn3nStSZl+UPEyZO6/Cni1z+f5Elfr3QkKx3JSkey0pGsdCQrHclKR7LSkTLmbfnPSI9nGkzbP+mfT/xN+790ML2YTqaL6Wb6G/X/cXUcjt2vumB1w+qK1R2rS1a3DLcMtwy3DLcMt4xhH5fj6Xg53lb2ciypQb9Bv/Hw+zftTNU7impHUesoKh1FncN+w37DfsN+4+n3/ZFuWF2xumN1yeqWoTcbT7/v2C8x/BbDLek37DfsN+y3kV4rejeN09k4nY3T2UivcTobp7NxOtuT3ruc21W9lPak9471UprpNdNrptdMr5leM71mes30muk102um10yvmV7L6HE6m6ezeTo7/XZOZ+d0dk5n53R2+u2czs7p7JzObr/dfrv9dvvt9tt99Hcf/d1Hf/fR3330dx/93Ud/99HfffR3H/396fcdu2W4ZbhluGVTy1FUcnBoB4d2cGgHh3bwjjo4tINDOzi0w2f8MNRhqMNQh6EOD+3w0A4P7fDQDg/teEJ9x27JoR0e2uGhHR7a4aEdHtrhoR2A+uMn", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [DateOnly = _t, #"Total Days With 3 Exceed Ref" = _t]),
    #"Extracted Text Before Delimiter" = Table.TransformColumns(Source, {{"DateOnly", each Text.BeforeDelimiter(_, " "), type text}}),
    #"Changed Type with Locale" = Table.TransformColumnTypes(#"Extracted Text Before Delimiter", {{"DateOnly", type date}}, "en-US"),
    #"Changed Type" = Table.TransformColumnTypes(#"Changed Type with Locale",{{"DateOnly", Int64.Type}}),
    #"Sorted Rows" = Table.Sort(#"Changed Type",{{"DateOnly", Order.Ascending}}),
    #"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "Index", 0, 1, Int64.Type),
    #"Added Index1" = Table.AddIndexColumn(#"Added Index", "Index.1", 1, 1, Int64.Type),
    #"Merged Queries" = Table.NestedJoin(#"Added Index1", {"Index"}, #"Added Index1", {"Index.1"}, "Added Index1", JoinKind.LeftOuter),
    #"Expanded Added Index1" = Table.ExpandTableColumn(#"Merged Queries", "Added Index1", {"Total Days With 3 Exceed Ref"}, {"Added Index1.Total Days With 3 Exceed Ref"}),
    #"Added Custom" = Table.AddColumn(#"Expanded Added Index1", "Custom", each if [Total Days With 3 Exceed Ref]<>[Added Index1.Total Days With 3 Exceed Ref] then 1 else 0),
    #"Filtered Rows" = Table.SelectRows(#"Added Custom", each ([Custom] = 1) and ([Added Index1.Total Days With 3 Exceed Ref] <> null)),
    #"Removed Other Columns" = Table.SelectColumns(#"Filtered Rows",{"DateOnly", "Total Days With 3 Exceed Ref"}),
    #"Added Custom1" = Table.AddColumn(#"Removed Other Columns", "Start", each if [Total Days With 3 Exceed Ref]="1" then [DateOnly] else null),
    #"Filled Down" = Table.FillDown(#"Added Custom1",{"Start"}),
    #"Added Custom2" = Table.AddColumn(#"Filled Down", "End", each if [Total Days With 3 Exceed Ref]="0" then [DateOnly]-1 else null),
    #"Filled Up" = Table.FillUp(#"Added Custom2",{"End"}),
    #"Removed Duplicates" = Table.Distinct(#"Filled Up", {"Start"}),
    #"Changed Type1" = Table.TransformColumnTypes(#"Removed Duplicates",{{"Start", type date}, {"End", type date}}),
    #"Removed Other Columns1" = Table.SelectColumns(#"Changed Type1",{"Start", "End"})
    in
    #"Removed Other Columns1"

     

    - using DA

    create a new calculated column to get next date value.

    Next Value =
    var _Date =
    MAXX(
        FILTER(
            'Table',
            'Table'[DateOnly]<EARLIER('Table'[DateOnly])
        ),
        'Table'[DateOnly]
    )
    Return
    MAXX(
        FILTER(
            'Table',
            'Table'[DateOnly]=_Date
        ),
        'Table'[Total Days With 3 Exceed Ref]
    )
    then create a new table to summarize the original table


    DAX =

    var _Sum =
    SUMMARIZE(
        FILTER(
            'Table',
            'Table'[Total Days With 3 Exceed Ref]<>'Table'[Next Value]
        ),
        'Table'[DateOnly],
        'Table'[Total Days With 3 Exceed Ref],
        'Table'[Next Value]
    )
    var _Next =
    ADDCOLUMNS(
        ADDCOLUMNS(
            _Sum,
            "Next Date1",
            MINX(
                FILTER(
                    _Sum,
                    'Table'[DateOnly]>=EARLIER('Table'[DateOnly])&&
                    'Table'[Next Value]=1
                ),
                'Table'[DateOnly]
            )
        ),
        "Datediff",
        DATEDIFF(
            [DateOnly],
            [Next Date1],
            DAY
        )
    )
    Return
    SELECTCOLUMNS(
        FILTER(
            _Next,
            [Datediff]>0
        ),
        "Start",[DateOnly],
        "End",[Next Date1]-1
    )
    You can tweak the code to your preferences
    Sorry if the code is quite messy but hope you can get the idea where to start.
     
    Hope this will help.
    Thank you.
    • AMONT's avatar
      AMONT
      Frequent Visitor

      Hi, thanks for taking a stab at this!

       

      For the 10/20 vs 10/24, that was me, just a typo, your values are correct. 

       

      I went the DAX route first, and I get an error for a circular dependency on the first step. In my dataset, the Total Days With 3 Exceed Ref is a measure, and I think that's what causing it. That measure is the product of three previous measures, which is why I think this has become such a headache for me: 

       

       

      The following measures are what has resulted in Total Days with 3 Exceed Ref:

      The first measure summarizes the column Exceeds Ref, which can have values ranging from 0 to 36, down to 0 to 3, as I have the hours filtered in my dataset.

       

      Total Exceeds Ref =
      ROUND(SUMX (
          VALUES ('All Stations'[Hour Start]),
          AVERAGE ('All Stations'[Exceeds Ref])
      ),0)
       
      Ref Count = IF([Total Exceeds Ref] = 3, 1, 0) --> Assigns a value so that only the Total Exceeds Ref that equal 3 are counted.
       
      Total Days With 3 Exceed Ref =
      SUMX(
      VALUES('All Stations'[DateOnly]),
      VAR DayCount =
      CALCULATE(
      [Ref Count],
      ALLEXCEPT('All Stations', 'All Stations'[DateOnly], 'All Stations'[Well_Number])
      )
      RETURN IF(DayCount = 1, 1, 0)
      )
       
      This measure is what allowed me to get the total days that a well has the value of 1 and be able to summarize in a visual.
      • Irwan's avatar
        Irwan
        Super User

        hello AMONT 

         

        here is the tweak if [Total Days With 3 Exceed Ref] as a measure.

        you can continue this below step from creating measure for [Total Days With 3 Exceed Ref].

        Right side of above screenshot is using [Total Days With 3 Exceed Ref] as a measure which has exact same result as using calculated column in left side.

        As measure works in filter context, so you need to have 'DateOnly' column as filter or else you will have blank because there is no filter.

         

         

        1. create a new measure for calculating end of conscutive date

        Measure Changed Value =
        var _Select = SELECTEDVALUE('Table'[DateOnly])
        var _Date =
        MAXX(
            FILTER(
                ALL('Table'),
                'Table'[DateOnly]<_Select
            ),
            'Table'[DateOnly]
        )
        var _Value =
        MAXX(
            FILTER(
                ALL('Table'),
                'Table'[DateOnly]=_Date
            ),
            [Measure Total Days With 3 Exceed Ref]
        )
        Return
        IF(
            [Measure Total Days With 3 Exceed Ref]<>_Value,
            1,
            0
        )
        2. create a new measure for calculating the first date of consecutive date
        Measure Min DateOnly =
        MINX(
            FILTER(
                ALL('Table'),
                'Table'[DateOnly]>=SELECTEDVALUE('Table'[DateOnly])&&
                [Measure Changed Value]=1&&
                [Measure Total Days With 3 Exceed Ref]=1
            ),
            'Table'[DateOnly]
        )
        3. create a new measure to define duplicate because we dont want to see many same value in between consecutive date
        Measure Remove Duplicate =
        IF(
            [Measure Min DateOnly]=SELECTEDVALUE('Table'[DateOnly]),
            1,
            0
        )
        4. Plot the mindate and nextdate of consecutive date then put the removeduplicate measure as visual filter

        5. create a new measure for calculating number of consecutive date.

        Measure Count =
        DATEDIFF(
            [Measure Min DateOnly],
            [Measure Next Date],
            DAY
        )
         
        Detail is in attached pbix.
         
        Hope this will help.
        Thank you.
  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi AMONT ,

    Thank you for reaching out to the Microsoft fabric community forum.

     

    Could you please confirm if the issue has been resolved. I wanted to check if you had the opportunity to review the information provided by Irwan .Please feel free to contact us if you have any further questions.

     

    Thank you.

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi AMONT ,

     

    I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions.

     

    Thank you.