Inconsistent GroupBy behavior that ignores certain rows and duplicates others

[champioz]

Hi,

I'm trying to do a simple two-column groupby with a single aggregation, and noticed I had a few entries with incorrect output. So I filtered to a single ID both before and after grouping to compare their results

Here is that one ID's appearance with no grouping. I am grouping by the ID and StageName columns and taking the MIN of CreatedDate:

Here is what happens when I filter to just that ID and then GroupBy (correct):

Here is what happens when I GroupBy the entire dataset and then filter to just this ID (incorrect, missing Closed Lost and dupes Probe):

As you can see in the steps, this GroupBy and filtering are the only operations I am performing. How is this error possible?

Edit: The plot thickens. I reduced the size of the dataset by taking out a chunk of records I was going to remove afterwards, and now the GroupBy performs consistently correctly on the previous erroneous entries. Is there a row limit on grouping accuracy in power query? I went from about 100,000 to about 50,000.

[v-junyant-msft]

Hi @champioz ,

I tested it on a small dataset but was not able to replicate the situation you are facing and there is no documentation that mentions that there is a limit to the number of rows that can be processed in Power Query. This may be a temporary display error, please test with a different ID to see if you face the same issue, or click refresh preview or a different machine to see if the issue is resolved.

Best Regards,
Dino Tao
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[champioz]

Hmm, I'm not sure about a display error as the errant output was persistent upon applying power query results in the PBI table view. There were only about 25 out of 42,000 unique IDs affected that I could find but you can see how it would make me uneasy about the result accuracy. If it would be helpful I can try to replicate this behavior with a batch of fake data and try it on multiple machines, though at present I don't really have any need for further help as reducing the dataset seems to have worked.