My raspberry pi randomly started refusing connections. After a reboot it allows connections, nothing has been changed in months with the config.

Here is the ssh -vvv output

"OpenSSH_5.9p1 Debian-5ubuntu1.3, OpenSSL 1.0.1f 6 Jan 2014
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to 192.168.1.21 [192.168.1.21] port 22.
debug1: Connection established.
debug3: Incorrect RSA1 identifier
debug3: Could not load "/home/matthew/.ssh/id_rsa" as a RSA1 public key
debug1: identity file /home/matthew/.ssh/id_rsa type 1
debug1: Checking blacklist file /usr/share/ssh/blacklist.RSA-2048
debug1: Checking blacklist file /etc/ssh/blacklist.RSA-2048
debug1: identity file /home/matthew/.ssh/id_rsa-cert type -1
debug1: identity file /home/matthew/.ssh/id_dsa type -1
debug1: identity file /home/matthew/.ssh/id_dsa-cert type -1
debug1: identity file /home/matthew/.ssh/id_ecdsa type -1
debug1: identity file /home/matthew/.ssh/id_ecdsa-cert type -1ssh_exchange_identification: Connection closed by remote host"