On-Prem Report Server Custom Authentication for Dummies?

I think that is because what you are asking for does not exist. The custom authentication feature was built to allow ISVs who have existing software with it's own authentication to integrated that into Report Server so that they could use it as a platform for the reporting for the application. It was not built as a way to bypass unhelp IT admins. 

In your situation it would seem that either moving the server to Domain B or setting up a trust relationship are the correct approaches.

Or if Domains A & B each have their own sets of users and they need their own sets of different reports over different data sets and they need to be kept separate for legitimate security reasons, then you might just need to setup a new report server in Domain B.