Power BI Report Server Architecture Question

Yes, SQL Server Enterprise License with Software Assurance cover all these products (SQL Server, Power BI Report Server and Analysis Services).

But, if you have 4 core licences (2x 2core pack) and if your servers are virtual, it is not enough for two server configuration, you need 4core for server A and and 4core for server B (at least). There two solutions, buy more licences. Or place licence on physical server with 4core procesor, where you can run any number virtual servers, with any number of virtual cores.

Two server configuration (with mixed SQL and SSAS) is also not good practice, is better three seperated (virtual) servers, one for MSSQL, second for SSAS, and third for Power BI Report Server (at least).

Utilization of MSSQL (for Report Server catalog) is low, there absolutelly enough use SQL Server Standard without SA.