Date filter breaks ALL

Given a simple table with a category, a date and a value column, e.g. :

I then write a measure to sum the value:

Sum of Value = SUM( test[value] ) 

And a measure to ignore the filtering of the dataset on the category:

Sum of All Categories = CALCULATE( [Sum of Value], ALL( test[category] ) ) 

If I now add just category to the report level filter, making changes to the selection has no effect on the value of "Sum of All Categories", but when I add the date column to the report level filters aswell, and set it to "Relative date filtering - is in the last 100 years", or just select some arbitrary dates, then all of a sudden changing the selection on the category filter begins to affect the value of the measure.

This only seems to be affecting dates, and if I use ALL( test ) to clear all filters, it also seems to work, but I only want to remove the filter on the category column not the date column.

Has anyone seen this behaviour before? This seems to work on other data types but as soon as you use a date it seems to break.