datediff formula with multiple criteria not pulling proper data

I am trying to create a column that pulls downtime minutes between 2 dates excluding hours or non-ooperation 9PM to 5AM.  If the downtime minutes fall between the exclude hours they should probably be set to 0.  This is what I have so far on the formula but much of the results are incorrect.

WorkingDowntimeMinutes =
IF (
    (
        EventLog[startdatetime] >= HOUR ( 5 )
            || EventLog[StartTime] <= HOUR ( 21 )
    )
        && (
            EventLog[stopdatetime] >= HOUR ( 5 )
                || EventLog[stopdatetime] <= HOUR ( 21 )
        ),
    DATEDIFF ( EventLog[startdatetime], EventLog[stopdatetime], MINUTE )
)

Any ideas where I am going wrong and how to add an else for the 0?

Thanks,

Mike