Improve Select All Slicer Performance

There are instances where we need to filter down the data set on one variable, like contract number (say it's an enterprise contract for a number of locations with different customer numbers) because we want to use this filter to find all the related customer numbers. With the limited customer number list, I want to be able to "Select All" customer numbers now showing and then unselect the contract number because I want to see all business for these customer numbers, whether each item was sold under contract or not. However, when I click to erase the contract number, the "Select All" function remains engaged and now selects every single customer number in the dataset. Why doesn't it keep the prior selection? We have users that are having to click one by one the customer numbers - there could be 60 under one contract - to perform this same function. The attribute slicer with brush function turned on works well for "brush" selecting, but that slicer does not sync across a pbix file. This is a really important time-saving feature.