I am attempting to connect over the internet to a service that is listening on a port on a remote machine behind a router, through an ssh tunnel. I have root privileges on local machine, remote machine and admin access to the router. The remote machine's firewall denies all incoming connections except on its ssh port, let's say it's 2222. Port 2222 is forwarded by the router and the local machine's ssh client is also using this port. I am able to connect from my local machine over the internet to the remote machine with ssh <my_user>@<router public ip>, giving me a shell.
So far so good. To set up the tunnel, I run
ssh -D 5555 <my_user>@<router public ip> and I get a shell on the remote machine as expected. In the output of sudo ss -tap | grep ssh on the local machine I can see the ssh client listening on localhost port 5555, acting as SOCKS5 server, and the established connection for the tunnel:
LISTEN 0 0 127.0.0.1:5555 0.0.0.0:* users:(("ssh", pid=...)) ESTAB 0 0 <local machine private ip>:40632 <router public ip>:2222 users:(("ssh", pid=...)) 40632 is the random outgoing port used by the ssh client. I tell my application client to use the SOCKS5 server at localhost:5555 and attempt to connect via the tunnel to the application server through its gui. This fails with an input/output error in the gui, in the terminal I get
channel 3: open failed: connect failed: Connection refused and in the remote machine's auth.log I get
sshd[<pid>]: error: connect_to <router public ip> port 5555 failed The 'channel 3' message, having read a lot of other sources about this, is often due to there being no service listening on port 5555 on the remote machine. I suppose then that sshd on the remote machine cannot pass the now-unencrypted packets on. In my case, the application service is listening on remote port 5555: I can see this both in the output of ss and I am also able to connect to that port using nc on the remote machine, producing an established internal connection in the ss output.
I think the relevant error message is the one coming from the remote machine's sshd. This tells me that sshd is trying to connect to the router's public ip address port 5555, which is not forwarded and therefore fails to connect (I have verified this by allowing this port forward by the router, but this is not a solution for me). My impression is that the tunnel is working up to the part where remote's sshd should pass packets to the target port on remote.
Shouldn't sshd on the remote machine try to connect to the remote machine's localhost:5555 rather than use the public ip address? What might I have not configured correctly here?
Having to forward 5555 through the router undermines the tunnel, so I'm sure I have set something up incorrectly. One more thing to mention is that when I do everything over the LAN, the tunnel works. As I said above, a simple ssh connection over the internet also works. I have also checked that the remote machine's firewall is not blocking anything, and it is not (no blocks in ufw.log).
If I have missed any crucial details needed, please let me know. Any ideas on this would be much appreciated! Thank-you.