Route 2012 RDS Farm user to specific server

Server 2012

I have a server farm consisting of a broker, three rds servers etc

I want to have my domain groups/users control which users are allowed access to specific servers.

Example: a xxx_admin group will be routed to server 1; xxx_sales group will be routed to server 2 etc.

I'd like to do this all with domain user/group configuration.

Thanks for the help...still pretty new to this.