ArcGIS legend colors not syncing up with slicer

[inbw]

Hello All,

I'm making a choropleth map where for each district I have some data available on a population that is divided by age. If I take the whole population (so aged 0 to 100) it gives me this :

However, the problem is that I want to be able to use my slicer to select only some parts of the population, for example the 40-60, and that's where it messes up because the legend/color gradient is not updated, so I get this :

Is there a way to make it update automatically based on the slicer that I use ?

Thanks !