Rolling average with adjustable time frame

I'm having issues creating a measure to calculate the average over the past X days for a specific account. In the initial Data I have multiple rows per day for several accounts with a specific value recorded for each account, each day. The data is over a year and I need to come up with an average amount looking back X days, then compare the current days value to the average amount over the adjustable period. to calculate the average amount I have a query that almost works, it just looks at every day and creates an average instead of averaging over the days in the spreadsheet (which excludes weekends and holidays). How would I be able to fix this issue? I need to create a matrix which will show every value and then count the amount of times the specific day breaks the specified average value.

AverageAmountOverXDays =

VAR SelectedDays = SELECTEDVALUE(DaysP arameter[Days Parameter])

RETURN

CALCULATE(

AVERAGE('Table'[Amount]),

DATESINPERIOD('Date'[Date], MAX('Date' [Date]), -SelectedDays, DAY)

)

I also have a reference calendar that contains only dates from the data and a count to rank them. I couldn't figure out how to take advantage of it though.