how to get unfiltered values

I'm typing this out on a mobile so hopefully reads OK (the text area is really small!)

It's certainly an interesting proposal. I believe that if you're using the filter API, you'll get the filter request your visual has made, but not necessarily what's in the filter pane as other contexts may apply (e.g. report, page or other visuals). There's currently no specific API for the filter pane content.

Note that it's possible for Power BI to determine whether your visual's filter is even needed for other visuals on the same page (for instance if you supply dates or ranges that aren't in any filter context in any other visual) - in these cases Power BI typically won't even filter as there's no computation required once it's figured things out. If there are other slicers or filter contexts affecting your visual's dataset there is no way to find this out currently, as visuals are isolated from the main window in addition to other visuals. The one exception to this is if your visual is using sync slicer capabilities and its tied to another instance of the same visual in another page. In these cases, only the jsonFilter is shared and no actual data, except for the data view Power BI computes and passes into the visual as normal.

I found this put the hard way after trying to develop a very complex slicer for a client - the developer visual host is (understandably) concerned with passing in the the bare minimum amount of data your visual should be able to see, which is basically what Power BI decides to give it based on the visual's capabilities.