Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The ultimate Microsoft Fabric, Power BI, Azure AI & SQL learning event! Join us in Las Vegas from March 26-28, 2024. Use code MSCUST for a $100 discount. Register Now

Reply
Kishore_SR
Frequent Visitor

Capture Columns based on Key words in Column Headers and also the values with from 1 to 9

Hi all,

 

I have pasted sample data here, this is only for reference 

ConsumerProductOverall_LikingTaste_LikingLiking of ProductLiking_Ratings
1Cookie9101
2Cookie8212
3Cookie7303
4Cookie641 4
5Cookie5505
6Cookie4615
7Cookie3704
8Cookie2813
9Cookie1902

 

Now I have Columns with key words "Liking" in it and values in Overall_Liking, Taste_Liking  are in the scale of 1 to 9, Liking of Product are in the scale of 0,1 and Liking_Ratings are in the scale of 1 to 5

 

What I want here is to capture the columns with Key Word "Liking" and values in the scale of 1 to 9 and remove the other columns, will it be possible to do this in Power Query!?

 

If so please help, Thanks in Advance!!

 

 

 

1 ACCEPTED SOLUTION
AntrikshSharma
Community Champion
Community Champion

@Kishore_SR Here are 2 methods:

with List.Accumulate:

let
    Source = Table.FromRows (
        Json.Document (
            Binary.Decompress (
                Binary.FromText (
                    "Tc89DsAgCAXgqzTOHVDxb+4xjGOHpoP334r0xTC8YPzQQO/Ou9Ndc77PLYcmWRekdZzdBctVEtASlKPlIol4HZXZcpYwXh+snqwnhLQuzpYZX/jNxXLEAKSti6vlgPH9nq1Z9lie/s3GBw==",
                    BinaryEncoding.Base64
                ),
                Compression.Deflate
            )
        ),
        let
            _t = ( ( type nullable text ) meta [ Serialized.Text = true ] )
        in
            type table [
                Consumer = _t,
                Product = _t,
                Overall_Liking = _t,
                Taste_Liking = _t,
                #"Liking of Product" = _t,
                Liking_Ratings = _t
            ]
    ),
    ChangedType = Table.TransformColumnTypes (
        Source,
        {
            { "Consumer", Int64.Type },
            { "Product", type text },
            { "Overall_Liking", Int64.Type },
            { "Taste_Liking", Int64.Type },
            { "Liking of Product", Int64.Type },
            { "Liking_Ratings", Int64.Type }
        }
    ),
    ColumnNames = List.Select (
        Table.ColumnNames ( ChangedType ),
        each Text.Contains ( Text.Lower ( _ ), "liking" )
    ),
    Transformation = List.Accumulate (
        ColumnNames,
        ChangedType,
        ( State, Current ) =>
            let
                Col = Table.Column ( State, Current ),
                Min = List.Min ( Col ),
                Max = List.Max ( Col ),
                Check =
                    if Min = 1 and Max = 9 then
                        State
                    else
                        Table.RemoveColumns ( State, Current )
            in
                Check
    )
in
    Transformation

 Method 2:

let
    Source = Table.FromRows (
        Json.Document (
            Binary.Decompress (
                Binary.FromText (
                    "Tc89DsAgCAXgqzTOHVDxb+4xjGOHpoP334r0xTC8YPzQQO/Ou9Ndc77PLYcmWRekdZzdBctVEtASlKPlIol4HZXZcpYwXh+snqwnhLQuzpYZX/jNxXLEAKSti6vlgPH9nq1Z9lie/s3GBw==",
                    BinaryEncoding.Base64
                ),
                Compression.Deflate
            )
        ),
        let
            _t = ( ( type nullable text ) meta [ Serialized.Text = true ] )
        in
            type table [
                Consumer = _t,
                Product = _t,
                Overall_Liking = _t,
                Taste_Liking = _t,
                #"Liking of Product" = _t,
                Liking_Ratings = _t
            ]
    ),
    ChangedType = Table.TransformColumnTypes (
        Source,
        {
            { "Consumer", Int64.Type },
            { "Product", type text },
            { "Overall_Liking", Int64.Type },
            { "Taste_Liking", Int64.Type },
            { "Liking of Product", Int64.Type },
            { "Liking_Ratings", Int64.Type }
        }
    ),
    LikingsColumns = List.Select (
        Table.ColumnNames ( ChangedType ),
        each Text.Contains ( Text.Lower ( _ ), "liking" )
    ),
    NonLikingsColumns = List.Select (
        Table.ColumnNames ( ChangedType ),
        each not Text.Contains ( Text.Lower ( _ ), "liking" )
    ),
    ColumnsToKeep = List.Select (
        LikingsColumns,
        ( ColName ) =>
            let
                ColList = Table.Column ( ChangedType, ColName ),
                Min     = List.Min ( ColList ),
                Max     = List.Max ( ColList ),
                Check   = if Min = 1 and Max = 9 then true else false
            in
                Check
    ),
    Result = Table.SelectColumns ( ChangedType, NonLikingsColumns & ColumnsToKeep )
in
    Result

 

View solution in original post

4 REPLIES 4
wdx223_Daniel
Super User
Super User

= Table.SelectColumns(Source,List.Select(Table.ColumnNames(PreviousStepName),each not Text.Contains(_,"Liking") or List.RemoveItems(Table.Column(PreviousStepName,_),{1..9})={}))

Kishore_SR
Frequent Visitor

Hi @AntrikshSharma 

 

Your Method 2 works great for me, Thank you for your help.

 

One concern for me is, instead of Max = 9 can we set it as greater than 5..!! because sometimes there might not be no 9 ratings given

 

Is there a work around for this!?

 

 

 

@Kishore_SR Then you can do 

Check = if Min = 1 and Max > 5 then true else false
AntrikshSharma
Community Champion
Community Champion

@Kishore_SR Here are 2 methods:

with List.Accumulate:

let
    Source = Table.FromRows (
        Json.Document (
            Binary.Decompress (
                Binary.FromText (
                    "Tc89DsAgCAXgqzTOHVDxb+4xjGOHpoP334r0xTC8YPzQQO/Ou9Ndc77PLYcmWRekdZzdBctVEtASlKPlIol4HZXZcpYwXh+snqwnhLQuzpYZX/jNxXLEAKSti6vlgPH9nq1Z9lie/s3GBw==",
                    BinaryEncoding.Base64
                ),
                Compression.Deflate
            )
        ),
        let
            _t = ( ( type nullable text ) meta [ Serialized.Text = true ] )
        in
            type table [
                Consumer = _t,
                Product = _t,
                Overall_Liking = _t,
                Taste_Liking = _t,
                #"Liking of Product" = _t,
                Liking_Ratings = _t
            ]
    ),
    ChangedType = Table.TransformColumnTypes (
        Source,
        {
            { "Consumer", Int64.Type },
            { "Product", type text },
            { "Overall_Liking", Int64.Type },
            { "Taste_Liking", Int64.Type },
            { "Liking of Product", Int64.Type },
            { "Liking_Ratings", Int64.Type }
        }
    ),
    ColumnNames = List.Select (
        Table.ColumnNames ( ChangedType ),
        each Text.Contains ( Text.Lower ( _ ), "liking" )
    ),
    Transformation = List.Accumulate (
        ColumnNames,
        ChangedType,
        ( State, Current ) =>
            let
                Col = Table.Column ( State, Current ),
                Min = List.Min ( Col ),
                Max = List.Max ( Col ),
                Check =
                    if Min = 1 and Max = 9 then
                        State
                    else
                        Table.RemoveColumns ( State, Current )
            in
                Check
    )
in
    Transformation

 Method 2:

let
    Source = Table.FromRows (
        Json.Document (
            Binary.Decompress (
                Binary.FromText (
                    "Tc89DsAgCAXgqzTOHVDxb+4xjGOHpoP334r0xTC8YPzQQO/Ou9Ndc77PLYcmWRekdZzdBctVEtASlKPlIol4HZXZcpYwXh+snqwnhLQuzpYZX/jNxXLEAKSti6vlgPH9nq1Z9lie/s3GBw==",
                    BinaryEncoding.Base64
                ),
                Compression.Deflate
            )
        ),
        let
            _t = ( ( type nullable text ) meta [ Serialized.Text = true ] )
        in
            type table [
                Consumer = _t,
                Product = _t,
                Overall_Liking = _t,
                Taste_Liking = _t,
                #"Liking of Product" = _t,
                Liking_Ratings = _t
            ]
    ),
    ChangedType = Table.TransformColumnTypes (
        Source,
        {
            { "Consumer", Int64.Type },
            { "Product", type text },
            { "Overall_Liking", Int64.Type },
            { "Taste_Liking", Int64.Type },
            { "Liking of Product", Int64.Type },
            { "Liking_Ratings", Int64.Type }
        }
    ),
    LikingsColumns = List.Select (
        Table.ColumnNames ( ChangedType ),
        each Text.Contains ( Text.Lower ( _ ), "liking" )
    ),
    NonLikingsColumns = List.Select (
        Table.ColumnNames ( ChangedType ),
        each not Text.Contains ( Text.Lower ( _ ), "liking" )
    ),
    ColumnsToKeep = List.Select (
        LikingsColumns,
        ( ColName ) =>
            let
                ColList = Table.Column ( ChangedType, ColName ),
                Min     = List.Min ( ColList ),
                Max     = List.Max ( ColList ),
                Check   = if Min = 1 and Max = 9 then true else false
            in
                Check
    ),
    Result = Table.SelectColumns ( ChangedType, NonLikingsColumns & ColumnsToKeep )
in
    Result

 

Helpful resources

Announcements
Fabric Community Conference

Microsoft Fabric Community Conference

Join us at our first-ever Microsoft Fabric Community Conference, March 26-28, 2024 in Las Vegas with 100+ sessions by community experts and Microsoft engineering.

February 2024 Update Carousel

Power BI Monthly Update - February 2024

Check out the February 2024 Power BI update to learn about new features.

Fabric Career Hub

Microsoft Fabric Career Hub

Explore career paths and learn resources in Fabric.

Fabric Partner Community

Microsoft Fabric Partner Community

Engage with the Fabric engineering team, hear of product updates, business opportunities, and resources in the Fabric Partner Community.

Top Solution Authors
Top Kudoed Authors