Forum Discussion

uif19085's avatar
uif19085
Icon for Helper III rankHelper III
3 years ago
Solved

Flag duplicates with DAX

Hello, i'm trying to find a way to flag first apprence of a duplicates across muliple columns using DAX, my table looks like this:
ID           DATE             FLAG
1       01/01/2011           1
1       01/01/2011           0
2      02/02/2011            1
2      02/02/2011            0
2      02/02/2011            0

I tried with this calculated column but didn't work: 

Column =
IF(CALCULATE(COUNTROWS('Weeks'),ALLEXCEPT('Weeks','Weeks'[id],'Weeks'[date]))>1,1,0)

 

Need Help
  • smpa01's avatar
    smpa01
    3 years ago

    uif19085  can you please test out the following meaure , you probably need a rowNum/Index Column

     

     

    dup =
VAR __val =
    CALCULATE (
        [current],
        OFFSET (
            -1,
            DISTINCT ( ALL ( 'Table' ) ),
            ORDERBY ( 'Table'[Index] ),
            KEEP,
            PARTITIONBY ( 'Table'[id] )
        )
    )
VAR currentId =
    MAX ( 'Table'[id] )
VAR currentIndex =
    MAX ( 'Table'[Index] )
VAR dateCount =
    CALCULATE (
        DISTINCTCOUNT ( 'Table'[date] ),
        FILTER (
            ALL ( 'Table' ),
            'Table'[id] = currentId
                && 'Table'[Index] <= currentIndex
        )
    )
VAR final =
    SWITCH (
        TRUE (),
        __val == BLANK (), 1,
        [current] - 1 = __val
            && dateCount = 1, 0,
        1
    )
RETURN
    final

     

     

     

    or like this

    Measure = 
VAR currDate =
    MAX ( 'Table'[date] )
VAR currDate2 = currDate + 0
VAR currentId =
    MAX ( 'Table'[id] )
VAR currentIndex =
    MAX ( 'Table'[Index] )
VAR minIndexbyID =
    CALCULATE (
        [currentIndex],
        INDEX (
            1,
            DISTINCT ( ALL ( 'Table' ) ),
            ORDERBY ( 'Table'[Index] ),
            KEEP,
            PARTITIONBY ( 'Table'[id] )
        )
    )
VAR filterTbl1 =
    FILTER (
        ALL ( 'Table' ),
        'Table'[id] = currentId
            && 'Table'[Index] >= minIndexbyID
            && currentIndex >= 'Table'[Index]
    )
VAR filterTbl2 =
    FILTER (
        ALL ( 'Table' ),
        'Table'[id] = currentId
            && 'Table'[Index] >= minIndexbyID
    )
VAR leftTbl1 =
    SELECTCOLUMNS ( filterTbl1, "dt", 'Table'[date] + 0 )
VAR leftTbl2 =
    SELECTCOLUMNS ( filterTbl2, "dt", 'Table'[date] + 0 )
VAR rightTbl =
    SELECTCOLUMNS ( { currDate }, "dt", [Value] + 0 )
VAR dates1 =
    NATURALINNERJOIN ( leftTbl1, rightTbl )
VAR dates2 =
    NATURALINNERJOIN ( leftTbl2, rightTbl )
VAR val1 =
    COUNTROWS ( dates1 )
VAR val2 =
    COUNTROWS ( dates2 )
VAR ternary =
    SWITCH (
        TRUE (),
        val1 = 1
            && val2 = 1, val1 & "-unique entry",
        val1 = 1
            && val2 <> 1, val1 & "-has duplicate entry",
        "0" & "-has duplicate entry"
    )
RETURN
    ternary

     

     

     

4 Replies

  • v-yueyunzh-msft's avatar
    v-yueyunzh-msft
    Icon for Community Support rankCommunity Support
    3 years ago

    Hi , uif19085 

    According to your description, you want to "Flag duplicates with DAX".

    First, to distinguish this difference, we need a column to distinguish the order of the data, and we need to add an index column to the Power Query.

    Then we can add a calculated column use this dax code:

    Column = var _t =FILTER('Table','Table'[ID]=EARLIER('Table'[ID]) && 'Table'[DATE]=EARLIER('Table'[DATE]) && 'Table'[Index]<= EARLIER('Table'[Index]))
return
IF(COUNTROWS(_t)=1,1,0)

    Then we can meet your need:

    Thank you for your time and sharing, and thank you for your support and understanding of PowerBI! 

     

    Best Regards,

    Aniya Zhang

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly

  • smpa01's avatar
    smpa01
    Icon for Community Champion rankCommunity Champion
    3 years ago

    uif19085  can you please test out the following meaure , you probably need a rowNum/Index Column

     

     

    dup =
VAR __val =
    CALCULATE (
        [current],
        OFFSET (
            -1,
            DISTINCT ( ALL ( 'Table' ) ),
            ORDERBY ( 'Table'[Index] ),
            KEEP,
            PARTITIONBY ( 'Table'[id] )
        )
    )
VAR currentId =
    MAX ( 'Table'[id] )
VAR currentIndex =
    MAX ( 'Table'[Index] )
VAR dateCount =
    CALCULATE (
        DISTINCTCOUNT ( 'Table'[date] ),
        FILTER (
            ALL ( 'Table' ),
            'Table'[id] = currentId
                && 'Table'[Index] <= currentIndex
        )
    )
VAR final =
    SWITCH (
        TRUE (),
        __val == BLANK (), 1,
        [current] - 1 = __val
            && dateCount = 1, 0,
        1
    )
RETURN
    final

     

     

     

    or like this

    Measure = 
VAR currDate =
    MAX ( 'Table'[date] )
VAR currDate2 = currDate + 0
VAR currentId =
    MAX ( 'Table'[id] )
VAR currentIndex =
    MAX ( 'Table'[Index] )
VAR minIndexbyID =
    CALCULATE (
        [currentIndex],
        INDEX (
            1,
            DISTINCT ( ALL ( 'Table' ) ),
            ORDERBY ( 'Table'[Index] ),
            KEEP,
            PARTITIONBY ( 'Table'[id] )
        )
    )
VAR filterTbl1 =
    FILTER (
        ALL ( 'Table' ),
        'Table'[id] = currentId
            && 'Table'[Index] >= minIndexbyID
            && currentIndex >= 'Table'[Index]
    )
VAR filterTbl2 =
    FILTER (
        ALL ( 'Table' ),
        'Table'[id] = currentId
            && 'Table'[Index] >= minIndexbyID
    )
VAR leftTbl1 =
    SELECTCOLUMNS ( filterTbl1, "dt", 'Table'[date] + 0 )
VAR leftTbl2 =
    SELECTCOLUMNS ( filterTbl2, "dt", 'Table'[date] + 0 )
VAR rightTbl =
    SELECTCOLUMNS ( { currDate }, "dt", [Value] + 0 )
VAR dates1 =
    NATURALINNERJOIN ( leftTbl1, rightTbl )
VAR dates2 =
    NATURALINNERJOIN ( leftTbl2, rightTbl )
VAR val1 =
    COUNTROWS ( dates1 )
VAR val2 =
    COUNTROWS ( dates2 )
VAR ternary =
    SWITCH (
        TRUE (),
        val1 = 1
            && val2 = 1, val1 & "-unique entry",
        val1 = 1
            && val2 <> 1, val1 & "-has duplicate entry",
        "0" & "-has duplicate entry"
    )
RETURN
    ternary

     

     

     

  • Ashish_Mathur's avatar
    Ashish_Mathur
    Icon for Super User rankSuper User
    3 years ago

    Hi,

    It may be easier to do this in the Query Editor.  Would you be interested in that solution?