Can't remove filter from visual

Hello everyone, first post here

I created a graph that shows the value of a KPI for every shift (A, B, C) for the last 7 days. It works as expected in its own page. However I tried to use it as a tooltip and show it on a table that has DATE, SHIFT, KPI and VALUE as columns. Both DATE and KPI come from the fact table (I think it's called that?) while SHIFT comes from its own dimension table and VALUE is a measure that grabs the value from the fact table (I'm toying with labels with different colors so I have to use a couple of specific measures around that value)

However, when I move the mouse over one row of the table, the graph tooltip will only display the KPI for the last 7 days for the selected shift of that row. I've tried many different things and combinations of removefilters, all, etc but I cannot remove the shift filter. It's very possible I'm using them wrong. Take this as a simplified version of the measure I'm using as my VALUE column.