Slicer visual set to 'between' doesn't respect custom sorting

I have a slicer with a field with the hours of the day with a custom sorting applied that makes the set start at 7, wrap around 23, and end at 6. When the slicer is viewed as a list, the sorting works perfectly.

However, when I change the slicer to 'between' it goes back to 0-23.

Is there a setting or a workaround for this?