Field Parameter Sorting

When using a Field Parameter, I have a parameter table using Year and Fiscal Qtr. The Year field is an Whole Number data type. Fiscal Qtr is Text and the value is like "2022 Q1". If I use the field parameter in a graph, when I have Year selected, it sorts on Year as expected. When I select Fiscal Qtr, now the graph changes to sorting by the measure within the graph instead of by the Fiscal Qtr field in my field parameter. I want the graphs to sort by this field parameter regardless of what field I have selected.