Matrix is showing unexpected result..

I had 3 tables. Each table contains one xxxx-ID column, which acts as key. Table 1 only contains unique values of xxxx-ID, while table 2 and 3 contain multiple occurences xxxx-IDs (duplicates). Table 2 for instance has a column called 'action' with about 20 different actions. An xxxx-ID can have more than one action, but an action always has one of the same xxxx-ID. 

As I need to do some data validation I wanted to check how many of the xxxx-IDs from table 1 are also in table 2, for each 'action'. So I created a matrix visual. However, since I the results I got didn't make much sense, I added a new table (table 4) which contains a single column that has a single instance of all xxxx-ID in all three tables (so only unique xxxx-IDs).

Then, I created a new matrix visual to count the number of xxxx-ID per 'action' as shown below:

As you can see, the numbers still don't make sense. For each 'action' it shows the count of all the xxxx-ID's in table 4. While some actions occur with just a couple of dozen xxxx-ID's.

My relationships looks like this. Basically I have table 4 connected to the three other tables by key xxxx-ID (marked yellow):