Add or remove measure from table

There must be a silly simple solution to my problem but I have not found it yet.  Lets say I have a list of departments and for each I am showing 2 measures, actuals and budgets. But at some point during the year a forecast will be made and I want my users to be able to choose whether they want to add this to the visual, which will then show 3 measures, actuals, budgets and forecasts. 

I have seen solutions that work based on a measure table and switching between chosen values, but these produce blanks or zeros, I would like the measure to be removed from the visual, so no empty column.