Generally, if you have a new site, I find it best to design the subnets for that site as independent ones - then it can function independently of Site A if required.
As you say both routers have routing enabled, it seems even more obvious that SIte B's local hosts have their default gateway locally, with routing to get them off-site.
However, often a link between sites used by *servers* requires layer2 connectivity for the server subnet (VMotion, etc...).
In any case, "latency issues on the server VLAN"?
- Sounds like a Server Team trying to pass off their problems as being a "network problem".
--> Work with them to pinpoint the exact location of the issue: what you will be trying to do is prove exactly which server/switch interfaces are on the path of the "latency issues" and demonstrate that each relevant switchport is always running with lots and lots of spare capacity.