4

A strange issue with split DNS that's been annoying me for ages, DHCP dns points to my adguard (primary) and my home router (secondary). Both have DNS rewrites for my local home domain servers to the LAN IP.

This works perfectly for all devices except my Ubuntu laptop. This randomly decides to find the external DNS entries for those services, which points to the external interface of my router and fails (I need this for LetsEncrypt). If I statically set my DNS to the right DNS server it does the same thing, flush the cache and an nslookup works the first time you run it, but the 2nd it's switched back.

Digging into it has left me in loops and rabbit holes so figured I'd ask if anyone else can help me make sense.

If I edit /etc/resolv.conf from 127.0.0.53 to my dns server this is fine... until I'm on another wi-fi network.

nmcli DNS configuration: servers: 192.168.1.85 192.168.1.1 domains: localdomain interface: wlp195s0 nslookup ha.test.co.uk Server: 127.0.0.53 Address: 127.0.0.53#53 Non-authoritative answer: ha.test.co.uk canonical name = fake.test.co.uk. Name: fake.test.co.uk Address: 199.199.199.199 nslookup ha.test.co.uk 192.168.1.85 Server: 192.168.1.85 Address: 192.168.1.85#53 Non-authoritative answer: Name: ha.test.co.uk Address: 192.168.1.85

Running resolvectl status

Link 2 (wlp195s0) Current Scopes: DNS Protocols: +DefaultRoute -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported Current DNS Server: 192.168.1.1 DNS Servers: 192.168.1.85 192.168.1.1 DNS Domain: localdomain Default Route: yes

So it seems that for whatever reason my machine has decided to use the router DNS service which is my backup.

nslookup ha.test.co.uk 192.168.1.1

Server: 192.168.1.1 Address: 192.168.1.1#53 Name: ha.test.co.uk Address: 192.168.1.85 ha.test.co.uk canonical name = fake.test.co.uk.

The router has both the internal and external records (this is by default, I've got internal rewrites on there for local LAN), and Ubuntu us using the secondary DNS server, then from this using the secondary DNS entries... confusing!

I've read around this as much as I can, looked at disabling the loopback resolver, but this broke DNS totally.

Anyone any ideas?

Improve this question
1
  • Have edited my rrouter DHCP server to only give out one DNS server and this seems to be working. My concern is I want it as a fall back in case this is down. Commented Jun 28 at 17:32

2 Answers 2

Reset to default
7

The presence of nameserver 127.0.0.53 in /etc/resolv.conf suggests the actual domain name resolving is done by systemd-resolved, and the 127.0.0.53 is just a backwards-compatibility proxy for software that doesn't use the standard hostname resolution functions of glibc, but instead reads the /etc/resolv.conf directly.

If the hosts: line in /etc/nsswitch.conf contains the new keyword resolve instead of or before the classic dns keyword, then glibc (and all programs using its hostname resolution services) will skip /etc/resolv.conf and talk directly to systemd-resolved instead, if it is running.

If systemd-resolved is in use (see systemctl status systemd-resolved.service), then the actual effective DNS server settings can be viewed with resolvectl status.

If your network interfaces are configured by NetworkManager (as suggested by your use of nmcli), then to use custom DNS servers and override the DHCP-based configuration, you will need:

nmcli connection modify <connection name> ipv4.ignore-auto-dns yes nmcli connection modify <connection name> ipv4.dns 192.168.1.85,192.168.1.1

For wireless connections, the <connection name> usually is the same as the wireless network name (also known as SSID). For wired connections, it can be whatever you specified or whatever your NetworkManager front-end of choice defaulted to; see nmcli connection show for a list of connections. The <connection name> you're looking for will be in the NAME column.

But note that any DNS servers configured together like this are assumed to have the same data: with the classic glibc resolver you might have gotten away with it, but systemd-resolved may send queries to both servers in parallel, and then happily take the answer from whichever DNS server returns it first. If one of your DNS servers has AdGuard and the other doesn't, you are not going to want this.

Also note that modern web browsers in particular may sometimes default to using DNS-over-TLS or DNS-over-HTTP (DoT or DoH, respectively). If the browser uses some big DoT/DoH provider like Cloudflare or Google, it may entirely bypass your local DNS servers, unless you tell your web browser to not use DoT/DoH.

Improve this answer
1
  • Thanks for the detailed reply, if I understand correctly from more testing the parallel query piece seems to be what's happening here and my router responded quicker so it ubuntu switched to that DNS server. The router was also sending back both the internal and external records in it's reply, so I can see how my machine then got the external IP. I've edited my DHCP to only give out the adguard DNS for now and will test how this goes. I can then look at my unifi router and see what the options there are as I'd like that as a fall-back DNS should adguard be down. Thanks! Commented Jun 29 at 9:40
2

... DHCP dns points to my adguard (primary) and my home router (secondary). ... Ubuntu us using the secondary DNS server, ...

Despite what the names "primary" and "secondary" may seem to imply, there is no implied priority in DNS servers. Clients are free to use any one of the specified DNS servers, or alternate between them, or even use them simultaneously. [ 1 2 3 ]

Windows appears to preferentially use the first DNS server (although even that doesn't always seem to be true [ 1 2 ]). Ubuntu 25.04 apparently behaves differently, which is perfectly allowed.

The only way to guarantee proper DNS operation is for all DNS servers to provide full functionality.

..., then from this using the secondary DNS entries... confusing!

The same applies for multiple A-records in a DNS record. Clients are free to use any of the returned addresses in any order, and are actually encouraged to not just use the first one as a way of doing primitive load balancing!

If your DNS server is returning both internal AND external IPs for the same DNS query, then your configuration sounds broken to me.

This randomly decides to find the external DNS entries for those services, which points to the external interface of my router and fails

For services hosted on the router itself, I think this should work. Your internal clients should be able to access services through your external IP address. At least, it works on my router, but perhaps some routers disallow it. (For port forwards, I'm less convinced this should work)

(I need this for LetsEncrypt).

I don't think you do. Let's Encrypt needs to resolve your hostname to your IP address, but it will query the domain's DNS servers, which is not your home DNS server. The only clients querying your home DNS server are those on the internal network, so just returning the internal IP is fine.

The above doesn't apply if you host the authoritative names servers for the domain itself on your router (using SOA and NS records), but I doubt you are doing that. And if you are: trying to host the authoritative name servers for your domain AND a local caching DNS forwarder on the same software configuration sounds like asking for trouble to me; try to split them.

Improve this answer
1
  • This is a brilliant write up, thank you for taking the time. Makes absolute sense what's happening. I do come from the windows world so hence my thinking. Guess we've been spoilt! I can see what's happening, it's my router sending both the static DNS records but also the external ones as well. Ubuntu is picking up and picking the 'wrong' one. Unifi are releasing cnames in version 9.3 so hopefully this will solve things. Currently have only my adguard DNS set, but would like a 'backip' if this fails for any reason. Commented Jul 2 at 18:13

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.