Difference with previous Date's Value

Try

Previous Value =
VAR BaseTable =
    CALCULATETABLE (
        SUMMARIZE ( Sheet1, Sheet1[Date], Sheet1[Category] ),
        ALLSELECTED ( Sheet1 )
    )
VAR PrevRow =
    OFFSET (
        -1,
        BaseTable,
        ORDERBY ( Sheet1[Date], ASC ),
        PARTITIONBY ( Sheet1[Category] )
    )
VAR Result =
    CALCULATE ( [Avg_Value], PrevRow )
RETURN
    Result

Firstly i created a measure for the average value 

AverageValue = 
AVERAGE(TestTable[Value])

Then a new measure for the previous row value based on max date in each category

PrevAverageValue = 
VAR CurrentCategory = MAX('TestTable'[Category])  -- Get the current row's category
VAR CurrentDate = MAX('TestTable'[Date])  -- Get the current row's date
VAR PrevDate = 
    CALCULATE(
        MAX('TestTable'[Date]),
        FILTER(
            ALL('TestTable'),  -- Ignore any filters (e.g., from the matrix row context)
            'TestTable'[Category] = CurrentCategory && 
            'TestTable'[Date] < CurrentDate  -- Get the previous date for the same category
        )
    )  -- Find the previous date for the same category
RETURN
    CALCULATE(
        [AverageValue],  -- Get the value for the previous date
        'TestTable'[Date] = PrevDate && 'TestTable'[Category] = CurrentCategory
    )

Now you can create a new measure to display your variance.

Variance = 
VAR _Numerator = [AverageValue]
VAR _Denominator = [PrevAverageValue]

RETURN 
_Numerator - _Denominator // Abs value
// DIVIDE((_Numerator -_Denominator), _Denominator, 0) // Variance %