Forum Discussion

Nad_PBI_21's avatar
Nad_PBI_21
Frequent Visitor
2 years ago
Solved

Categorisation through keywords slow

Hi, I'm looking for a more efficient method to categorize dates based on a reference table that contains keywords. In the query below, I search for keywords (Booking and Partner) twice and then mer...
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi Nad_PBI_21 ,

    I can give you some suggestions:

    1. Minimize the use of 'Table.Buffer'. While buffering can improve performance by reducing the number of times a source is accessed, overusing it, especially with large tables, can lead to high memory consumption. Consider buffering only the smallest necessary tables or the ones that are used multiple times in subsequent transformations.
    2. Instead of performing the keyword search within an `each` function for every row, you could try to transform your reference table into a list of keywords and their corresponding categories once, and then use this list to categorize your main table. This way, you perform the heavy lifting of creating the keyword-category list just once.
    3. Use 'Table.Join' instead of 'Table.NestedJoin' when possible. If you don't need nested tables and only require a few columns from the reference table, a regular join might be more efficient.
    4.  Ensure that the columns used for joins and searches have the appropriate data types. Text operations can be slow on numeric or date fields if they are not properly typed.

    Best Regards,
    Dino Tao
    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.