I installed libvirt out of the box on Ubuntu 16.04, it creates a virbr0 virtual bridge, at 192.168.122.0/24, a dnsmasq instance to serve that range and sets up basic NAT masquerading.
I created a tap interface beneath that for my virtualization software to use. The software itself is able to access the internet and remainder of the network perfectly, so masquerading is working properly. I can also access the virtualization software ssh port from the host, using the IP address claimed by that software. So I think the routing tables "out of the box" are generally correct.
What I want to do is port forward an inbound ssh connection from outside to this software, and the host system in general. Now the host system of course has ssh, and I need that to work. So I'm trying to use another port, such as port 2022.
I've created a DNAT rule in iptables to direct inbound traffic on port 2022 to the IP address of the software at port 22. I get destination unreachable when I attempt to ssh in from the outside (ex. computer A below). It looks like the address translation is happening properly, but immediately after traversing to the "FORWARD" step in iptables, I start getting outbound unreachable ICMP packets.
I'm stumped on how to proceed. I've tried some SNAT rules, but I either botched it or that's not the problem. Does anyone have ideas on this?
UPDATE: Wireshark on tap0 does not show having received the forwarded packet. Whatever is returning the packet is not related to the virtualization software.
Here's a picture of my network:
---------- | Internet | ---------- | | -------- | Router | 10.211.255.1 -------- | | ____________|_______________ | | ---------- --------- | | | enp0s5 | 10.211.255.4 | Computer | 10.211.255.2 | Host | 192.168.122.1 | A | | virbr0 | ---------- --------- | | LAN NAT 192.168.122.0/24 | ---------- | tap0 | | Guest OS | 192.168.122.118 | | ---------- Here are my IP tables rules:
Chain INPUT (policy ACCEPT) target prot opt source destination LOG all -- anywhere anywhere LOG level debug prefix "INTRACE: " ACCEPT udp -- anywhere anywhere udp dpt:domain ACCEPT tcp -- anywhere anywhere tcp dpt:domain ACCEPT udp -- anywhere anywhere udp dpt:bootps ACCEPT tcp -- anywhere anywhere tcp dpt:bootps Chain FORWARD (policy ACCEPT) target prot opt source destination LOG all -- anywhere anywhere LOG level debug prefix "FORTRACE: " ACCEPT all -- anywhere 192.168.122.0/24 ctstate RELATED,ESTABLISHED ACCEPT all -- 192.168.122.0/24 anywhere ACCEPT all -- anywhere anywhere REJECT all -- anywhere anywhere reject-with icmp-port-unreachable REJECT all -- anywhere anywhere reject-with icmp-port-unreachable Chain OUTPUT (policy ACCEPT) target prot opt source destination LOG all -- anywhere anywhere LOG level debug prefix "OUTTRACE: " ACCEPT udp -- anywhere anywhere udp dpt:bootpc NAT tables: Chain PREROUTING (policy ACCEPT) target prot opt source destination LOG tcp -- anywhere anywhere tcp dpt:2022 LOG level debug prefix "NATPTRACE: " DNAT tcp -- anywhere anywhere tcp dpt:2022 to:192.168.122.118:22 Chain INPUT (policy ACCEPT) target prot opt source destination LOG all -- anywhere anywhere LOG level debug prefix "NATITRACE: " Chain OUTPUT (policy ACCEPT) target prot opt source destination LOG all -- anywhere anywhere LOG level debug prefix "NATOTRACE: " Chain POSTROUTING (policy ACCEPT) target prot opt source destination LOG all -- anywhere anywhere LOG level debug prefix "NATQTRACE: " SNAT tcp -- anywhere anywhere tcp spt:ssh to::2022 RETURN all -- 192.168.122.0/24 base-address.mcast.net/24 RETURN all -- 192.168.122.0/24 broadcasthost.localdomain MASQUERADE tcp -- 192.168.122.0/24 !192.168.122.0/24 masq ports: 1024-65535 MASQUERADE udp -- 192.168.122.0/24 !192.168.122.0/24 masq ports: 1024-65535 MASQUERADE all -- 192.168.122.0/24 !192.168.122.0/24 Finally this is an example of what I see in syslog with the above rules:
Oct 20 09:49:15 ubuntu kernel: [85149.812291] RAWPTRACE: IN=enp0s5 OUT= MAC=00:1c:42:3a:00:df:00:1c:42:00:00:08:08:00 SRC=10.211.55.2 DST=10.211.55.4 LEN=64 TOS=0x00 PREC=0x00 TTL=64 ID=40132 DF PROTO=TCP SPT=53245 DPT=2022 WINDOW=65535 RES=0x00 SYN URGP=0 Oct 20 09:49:15 ubuntu kernel: [85149.812318] MGPTRACE: IN=enp0s5 OUT= MAC=00:1c:42:3a:00:df:00:1c:42:00:00:08:08:00 SRC=10.211.55.2 DST=10.211.55.4 LEN=64 TOS=0x00 PREC=0x00 TTL=64 ID=40132 DF PROTO=TCP SPT=53245 DPT=2022 WINDOW=65535 RES=0x00 SYN URGP=0 Oct 20 09:49:15 ubuntu kernel: [85149.812326] NATPTRACE: IN=enp0s5 OUT= MAC=00:1c:42:3a:00:df:00:1c:42:00:00:08:08:00 SRC=10.211.55.2 DST=10.211.55.4 LEN=64 TOS=0x00 PREC=0x00 TTL=64 ID=40132 DF PROTO=TCP SPT=53245 DPT=2022 WINDOW=65535 RES=0x00 SYN URGP=0 Oct 20 09:49:15 ubuntu kernel: [85149.812336] MGFTRACE: IN=enp0s5 OUT=virbr0 MAC=00:1c:42:3a:00:df:00:1c:42:00:00:08:08:00 SRC=10.211.55.2 DST=192.168.122.118 LEN=64 TOS=0x00 PREC=0x00 TTL=63 ID=40132 DF PROTO=TCP SPT=53245 DPT=22 WINDOW=65535 RES=0x00 SYN URGP=0 Oct 20 09:49:15 ubuntu kernel: [85149.812340] FORTRACE: IN=enp0s5 OUT=virbr0 MAC=00:1c:42:3a:00:df:00:1c:42:00:00:08:08:00 SRC=10.211.55.2 DST=192.168.122.118 LEN=64 TOS=0x00 PREC=0x00 TTL=63 ID=40132 DF PROTO=TCP SPT=53245 DPT=22 WINDOW=65535 RES=0x00 SYN URGP=0 Oct 20 09:49:15 ubuntu kernel: [85149.812354] RAWOTRACE: IN= OUT=enp0s5 SRC=10.211.55.4 DST=10.211.55.2 LEN=92 TOS=0x00 PREC=0xC0 TTL=64 ID=50608 PROTO=ICMP TYPE=3 CODE=3 [SRC=10.211.55.2 DST=192.168.122.118 LEN=64 TOS=0x00 PREC=0x00 TTL=63 ID=40132 DF PROTO=TCP SPT=53245 DPT=22 WINDOW=65535 RES=0x00 SYN URGP=0 ] Oct 20 09:49:15 ubuntu kernel: [85149.812357] MGOTRACE: IN= OUT=enp0s5 SRC=10.211.55.4 DST=10.211.55.2 LEN=92 TOS=0x00 PREC=0xC0 TTL=64 ID=50608 PROTO=ICMP TYPE=3 CODE=3 [SRC=10.211.55.2 DST=192.168.122.118 LEN=64 TOS=0x00 PREC=0x00 TTL=63 ID=40132 DF PROTO=TCP SPT=53245 DPT=22 WINDOW=65535 RES=0x00 SYN URGP=0 ] Oct 20 09:49:15 ubuntu kernel: [85149.812361] OUTTRACE: IN= OUT=enp0s5 SRC=10.211.55.4 DST=10.211.55.2 LEN=92 TOS=0x00 PREC=0xC0 TTL=64 ID=50608 PROTO=ICMP TYPE=3 CODE=3 [SRC=10.211.55.2 DST=192.168.122.118 LEN=64 TOS=0x00 PREC=0x00 TTL=63 ID=40132 DF PROTO=TCP SPT=53245 DPT=22 WINDOW=65535 RES=0x00 SYN URGP=0 ] Oct 20 09:49:15 ubuntu kernel: [85149.812364] MGQTRACE: IN= OUT=enp0s5 SRC=10.211.55.4 DST=10.211.55.2 LEN=92 TOS=0x00 PREC=0xC0 TTL=64 ID=50608 PROTO=ICMP TYPE=3 CODE=3 [SRC=10.211.55.2 DST=192.168.122.118 LEN=64 TOS=0x00 PREC=0x00 TTL=63 ID=40132 DF PROTO=TCP SPT=53245 DPT=22 WINDOW=65535 RES=0x00 SYN URGP=0 ] 
