Forum Discussion

Mike282's avatar
Mike282
Icon for Helper III rankHelper III
5 years ago

Better way than Sort, table.buffer and then group

Hi all,

 

So I've got a problem. I've got a really really large table of invoice items. It's basically a list of line item products grouped together by a invoice ID. So kinda like:

 

|Invoice ID   | Product     | Purchase Date|  Customer ID |

01                 Product 1     24/11/2020       01

01                 Product 2     24/11/2020       01

01                 Product 3     24/11/2020       01

02                 Product 1     20/11/2020       01

02                 Product 2     20/11/2020       01

02                 Product 3     20/11/2020       01

 

You can see each product line item is grouped together by a invoice ID. So for Invoice 01 a person bought product 1, 2 and 3 on the 24th of November 2020 and previously bought the same products on the 20th of November 2020.

 

Now in M query, I want to only have the most recent purchase so the one bought on the 24th. I also want to concatenate all the products into one column and have it separated by " | ". I also want when the products get concatenated to have it concantenated in the same order of products so that I can put it in a visual. So the concatenated product bundle column will be like this for example: "Product 1 | Product 2 | Product 3".

 

So I first sort the product column by descending order. I then have to buffer the table to ensure it maintains the sort. I then do a group.

 

So I group by customer ID, Invoice ID and Purchase Date first and do a Text.Combine to the product column so that it concatenates the value. This buffer helps when the products gets concatenated to maintain the same order of products when grouped.

 

I then want to sort the result again by the purchase date column by descending order to get the most recent items to the top. I then have to buffer the table again because I want to maintain again the sort to ensure most recent are at the top during my next step (not sure if I do need to perform the buffer again. Can someone confirm). I then perform another grouping and this time only group by customer ID, Invoice Id and use List.First to get the most recent concatenated purchase.

 

The problem is the buffering. I understand the reason why buffering needs to be done to maintain the sort but I've got such a large data set that it's lagging and using up all my memory in Power BI. Sometimes it's also failing as it times out.

 

My question is whether there's a better way than buffering the results to ensure when you group something it still maintains the order from the previous step?

1 Reply

  • AllisonKennedy's avatar
    AllisonKennedy
    Icon for Community Champion rankCommunity Champion

    What is the end goal, you could try using DAX instead and something like RANKX and CONCATENATEX