Hi mg14,
Yes ACL needs to ne deployed on the Routing Switch which owns the SVIs (Gateways) of involved VLANs. Supposing you're dealing with three VLANs (say VLAN id x, id y and id z tied to, respectively, Network Segment x, y and z with, respectively, SVI IP address x, y and z) then you should (it's not essential) build three ACLs, one for each VLAN, each ACL should be applied in the incoming direction as seen from the SVI (so each ACL would deal with incoming traffic leaving the VLAN - source - with any other possible destination, internally connected or not, as possible target).
At least this is the way I build mines and I found building them quite simple once understood the flow direction's concept and following this approach.
So - in my opinion - when you write "
If I have for example, VLAN 5, 10.0.5.0/23. I don't want anything to be able to get to the systems on that VLAN (computers, printers, etc.), except for our servers on VLAN 10, 172.16.20.0/24(DHCP,DNS,AD, etc..)." you have to translate it into this sentence like: "Systems belonging to VLAN 5 shouldn't be able to communicate with anything outside VLAN 5 - including any external network like the Internet, I add (isn't it?) - they can only communicate with - starting communication to or responding to incoming messages coming from - VLAN 10 systems like DHCP Server, DNS Server...and so on." and the approach resolving this requirement will the one that help you to build the ACL protecting the VLAN 5 (or any other VLANs).
Repeat & Rinse for VLAN 10 (and any other involved VLAN you want to protect) always remembering you should build your ACL skeleton seeing traffic incoming to the SVI of the particular VLAN you're building the ACL to.
This just to start with.