This is a 2012 R2 RDS server on Xenserver 6.5 Hypervisor with plenty of resources not being stressed, 20% CPU 30% RAM. Domain Controller is on the same Hypervisor with plenty of resources just loafing.
With 8 or 9 users connected via Window 8.1 Pro workstations joined to the domain, using native Remote Desktop, Excel, Word, PDF open in 1 or 2 seconds. Just add one or 2 more users and it takes 20 seconds plus keystrokes are delayed when typing. Log off one or 2 users and it goes back to 1 or 2 seconds, no keystroke delays.
Users needs are minimal, web browser, Office documents, PDF viewing.
For testing I put up a fresh RDS server on dedicated hardware and I have the same issue on it with the same number of users. Putting 10 users on one and 2 on the other the RDS server that has the lower number of users does not have the issue
There is a max of 12 users that will log in. I have done load balancing between the 2 RDS servers and never have an issue with less than 10 users per server. It's crazy to have 2 RDS servers to support 12 users.
ProcMon and PerfMon did not highlight any issues.
Any suggestions? Thanks.