Forum Discussion

pwrbiadm's avatar
pwrbiadm
Helper I
5 years ago
Solved

Removing cumulative frequency using power(m) query

Hi! I have the below table where the data gets generated every 1-2 seconds.

 

Column A - Datetime

Column B - Cumulative Counter

Column C - Calculated in Excel to remove cumulative frequency IF(B2>=B1,B2-B1,B2)

 

Please help me identify a way using power(m) query to calculate Column C in power bi.

Due to the large dataset (over 3 million records), the dax query keeps on processing but won't execute.

Thanks!

 

A B  C 
10/20/20 2:43                        5.00                        5.00
10/20/20 2:44                        5.00                            -  
10/20/20 2:45                        7.00                        2.00
10/20/20 2:46                     15.00                        8.00
10/20/20 2:48                     70.00                     55.00
10/20/20 2:49                     97.00                     27.00
10/20/20 2:50                   124.00                     27.00
10/20/20 2:51                   151.00                     27.00
10/20/20 2:52                            -                              -  
10/20/20 2:53                        1.00                        1.00
10/20/20 2:54                     13.00                     12.00
10/20/20 2:56                     67.00                     54.00
  • I was able to use the below query for a dax calculated column which gave me the desired results without taking a long time to process the large dataset. Had to create an index column first in power query.

     

    Column = var e = CALCULATE(MAX(Query4[B]),FILTER(Query4,Query4[Index]=EARLIER(Query4[Index])-1))
    var f = Query4[B]
    return IF(f=0,0,IF(f>=e,f-e,e))

3 Replies

  • AlB's avatar
    AlB
    Community Champion

    Hi pwrbiadm 

    This might run faster in DAX. However, place the following M code in a blank query to see the steps. If it's not fast enough we'll try another approach

     

     

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("lZFLDsAgCAWvYlibFlC0ehXj/a/RNl01fZiUsCIzhM8YJLzrnUF7ThQpOGEbM834FvJfwRZCRULxBIH9Dw+vjPDm4Q0NY4xx0YxocWgTROtFf4qrh8Au7kMkIdy9bnn2nyc=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [A = _t, B = _t]),
        #"Changed Type" = Table.TransformColumnTypes(Source,{{"A", type text}, {"B", type number}}),
        #"Replaced Value1" = Table.ReplaceValue(#"Changed Type",null,0,Replacer.ReplaceValue,{"B"}),
       
        col_ = List.RemoveLastN({0} & #"Replaced Value1"[B], 1),
        res_ = Table.FromColumns(Table.ToColumns(#"Replaced Value1") &  {col_}, {"A", "B","B_2"}),
        #"Changed Type1" = Table.TransformColumnTypes(res_,{{"B_2", type number}}),
        #"Added Custom" = Table.AddColumn(#"Changed Type1", "Custom", each [B]-[B_2], type number),
        #"Replaced Value" = Table.ReplaceValue(#"Added Custom",null,0,Replacer.ReplaceValue,{"B_2"})
    in
        #"Replaced Value"

     

     

    Please mark the question solved when done and consider giving a thumbs up if posts are helpful.

    Contact me privately for support with any larger-scale BI needs, tutoring, etc.

    Cheers 

     

  • Icey's avatar
    Icey
    Community Support

    Hi pwrbiadm ,

     

    Due to your large dataset, it is suggested to create a measure like so:

    Measure =
    VAR ThisRow_B =
        MAX ( [B] )
    VAR LastRow_B =
        CALCULATE (
            MAX ( 'Query1 (2)'[B] ),
            FILTER (
                ALLSELECTED ( 'Query1 (2)' ),
                'Query1 (2)'[Index]
                    = MAX ( 'Query1 (2)'[Index] ) - 1
            )
        )
    RETURN
        SWITCH ( TRUE (), ThisRow_B >= LastRow_B, ThisRow_B - LastRow_B, ThisRow_B )
    

     

    Best regards

    Icey

     

    If this post helps, then consider Accepting it as the solution to help other members find it faster.

  • I was able to use the below query for a dax calculated column which gave me the desired results without taking a long time to process the large dataset. Had to create an index column first in power query.

     

    Column = var e = CALCULATE(MAX(Query4[B]),FILTER(Query4,Query4[Index]=EARLIER(Query4[Index])-1))
    var f = Query4[B]
    return IF(f=0,0,IF(f>=e,f-e,e))