Dynamic Prior Year Balance with Year Slicer

Hi everyone, 

I've been trying to figure out how to do this but couldn't find a good solution yet.

I have a matrix visualization showing the following:

I currently have a simple data column set up specifically for 'Year' and it will change to 2021 & Prior once it hits 2024. 

The issue is that this isn't truly dynamic as it obviously doesn't work with date slicers on the dashboard.

For example I have a slicer for 'Number of Years' to show on the dashboard which is set to 3 as default.

Ideally I would like to be able to change the slicer to any number such as 5 and then the matrix will show: 2019 & Prior, 2020, 2021, 2022, and 2023. 

The only way I can think of is to create a seperate table with a single row using measures for the Prior Year totals but I was hoping that there would be a more elegant solution.

Thank you!