Are you 100% sure you're editing the zones files in the right directory, not using bind-chroot for example? I'm not aware of BIND doing any caching of local zones (other than loading them into RAM maybe).

@EdGrimm absolutely - what I meant, perhaps unclearly, by "require some changes" (the PAM configuration), also you would need to specifically block the user in your sshd config (explicit user or group) as I mentioned at the top. Frankly I just took it as an academic question (hence the comment not an attempt at answer) because ultimately, as you also say, using sudo would be a much better (secure, flexible, auditable, etc) way of working.

+1 for getaddrinfo() but actually this may just not be possible on a home network depending on DNS and DHCP configuration - if you're using ISP DNS servers directly then the router (which knows hostname to IP mapping) won't provide an answer.

You can add users explicitly to be denied in the sshd config and it's perfectly possible to have a user without a password (username:: in /etc/shadow - can be set various ways). Changing to the user with su would require some changes though as by default su will prompt for a password and an empty one will fail. See the top answer on unix.stackexchange.com/questions/113754/… - however I'd be much more tempted for this use case to use sudo -u username to execute commands as the user rather than trying to circumvent security on su.

Well that certainly points to it being an SSH error (personally I'd still try the ping and a big file exchange to be 100% certain). SSH-specific slowdowns come from a variety of reasons, most commonly some issue with the configuration - have a look at serverfault.com/questions/35453/what-can-cause-slow-ssh-sess‌​ion

First check: is it an SSH issue or a general network issue - try pinging one host to another (look for packet loss and latency times), try flood pinging (ping -f), also try the same to the router from each of the hosts. Another good check for this is start a non-SSH server (web or similar) and transfer a file down, see what sort of transfer speed you get (does it drop, go up, come down, etc). This will hopefully tell you if it's a network (wireless interference, something on the network causing problems, router overload, etc) or SSH (various options) issue to investigate further.