Forum Discussion
Incremental refresh and Group By in M/PowerQuery
Hi Anonymous,
Why do you want to remove these date fields?
AFAIK, 'incremental refresh' is based on date query parameters and corresponding date fields. If these date fields removed, incremental refresh may not be triggered.
Incremental refresh in Power BI#filter-large-datasets-in-power-bi-desktop
Regards,
Xiaoxin Sheng
So I want to remove the date fields because I want to aggregate users by Country. If I leave in the date field, it will do unique userID counts per day, but that means a returning user would be counted twice in the final table. By removing the date field and aggregating to Country, the unique counts are correct.
I could also just load the entire source table, but depending on the client, data (not in this table, but in others) does run into the billions of rows territory.
And yes, so I understand that incremental refresh is based on a date field. I just think that if the user removes the date field in powerquery, maybe the incremental refresh switch (when you right click on a table) should be greyed out instead of active. It also still mentions the date column that I have removed, which seems to indicate that this would still work, see screenshot:
Hi Anonymous,
Maybe you can try to create a new query to refer from the current query table and remove date field and grouping in new table, then cancel the loading of original query table. After these, the date fields should exist in the original table that not displayed.
Since the official document also not explain this cleanly, you can consider contacting to power bi team to know more about incremental refresh.Regards,
Xiaxoin Sheng
Aw, seemed like a nice workaround, but when I do this, the loaded table doesnt let you set incremental refresh on it, because it doesn't contain the filtering steps on rangestart and end that incremental refresh requires.
I think for now I will just assume that Incremental refresh and group by statements that remove the datetime fields do not play nice together. It makes sense.
Jaap