Forum Discussion
MattMcMillin_71
1 year agoHelper I
Troubles with grouping in Power Query
Hi, I have c.20 million rows of data with aobut 15 columns per row/transaction but want to limit the output into the Desktop by using the group by function in Power Query. This is meaning I am wai...
- 1 year ago
You have 2 options then I reckon. If it's a fabric pipeline, Spark based pipeline or an Azure Data Factory pipeline, you can add transformation steps to do the aggregations and then the Power Query engine won't need to spend time doing that effort.
Alternatively, the better option in my opinion, would be to make a view based on that table, within the SQL Database that you're querying. You can do things like grouping and roll up within SQL statements. If you're not the best at SQL then maybe use ChatGPT to try and convert your m code into SQL code, which you can use to make the view.
BabyYoda
1 year agoFrequent Visitor
Can you do the grouping at the source so that you aren't bringing in so many rows? What type of data source is it?