Forum Discussion
Better way than Sort, table.buffer and then group
Hi all,
So I've got a problem. I've got a really really large table of invoice items. It's basically a list of line item products grouped together by a invoice ID. So kinda like:
|Invoice ID | Product | Purchase Date| Customer ID |
01 Product 1 24/11/2020 01
01 Product 2 24/11/2020 01
01 Product 3 24/11/2020 01
02 Product 1 20/11/2020 01
02 Product 2 20/11/2020 01
02 Product 3 20/11/2020 01
You can see each product line item is grouped together by a invoice ID. So for Invoice 01 a person bought product 1, 2 and 3 on the 24th of November 2020 and previously bought the same products on the 20th of November 2020.
Now in M query, I want to only have the most recent purchase so the one bought on the 24th. I also want to concatenate all the products into one column and have it separated by " | ". I also want when the products get concatenated to have it concantenated in the same order of products so that I can put it in a visual. So the concatenated product bundle column will be like this for example: "Product 1 | Product 2 | Product 3".
So I first sort the product column by descending order. I then have to buffer the table to ensure it maintains the sort. I then do a group.
So I group by customer ID, Invoice ID and Purchase Date first and do a Text.Combine to the product column so that it concatenates the value. This buffer helps when the products gets concatenated to maintain the same order of products when grouped.
I then want to sort the result again by the purchase date column by descending order to get the most recent items to the top. I then have to buffer the table again because I want to maintain again the sort to ensure most recent are at the top during my next step (not sure if I do need to perform the buffer again. Can someone confirm). I then perform another grouping and this time only group by customer ID, Invoice Id and use List.First to get the most recent concatenated purchase.
The problem is the buffering. I understand the reason why buffering needs to be done to maintain the sort but I've got such a large data set that it's lagging and using up all my memory in Power BI. Sometimes it's also failing as it times out.
My question is whether there's a better way than buffering the results to ensure when you group something it still maintains the order from the previous step?
1 Reply
- AllisonKennedy
Community Champion
What is the end goal, you could try using DAX instead and something like RANKX and CONCATENATEX