1

I can access my server via curl and browsers

but I can't access it via wget. It is stuck at

Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt' Resolving *** Connecting to ***:443...

wget works with other similar servers though.

I've tried wget --debug but it doesn't really show a whole lot more; the last line where it is stuck is still just Connecting to .... The IP that wget resolves to is correct.

If I switch my network to the hotspot of my smartphone, I can use wget without problems. Restarting the router didn't help.

It fails on two manjaro installations, wget version 1.21.3. But works on a live xubuntu usb stick, wget version 1.21. I can also use wget from within other servers that are running ubuntu.

The https module of node.js behaves in the same way as wget for me.

Any idea where I can continue to look for problems?

Here is the output of wget2:

wget2 https://foo.***.com -vd 02.132709.170 Local URI encoding = 'UTF-8' 02.132709.170 Input URI encoding = 'UTF-8' 02.132709.170 Fetched HSTS data from '/home/vs-mango/.local/share/wget/.wget-hsts' 02.132709.170 Fetched HPKP data from '/home/vs-mango/.local/share/wget/.wget-hpkp' 02.132709.170 Fetched OCSP hosts from '/home/vs-mango/.local/share/wget/.wget-ocsp_hosts' 02.132709.170 Fetched OCSP fingerprints from '/home/vs-mango/.local/share/wget/.wget-ocsp' 02.132709.170 set_exit_status(0) 02.132709.170 *url = 02.132709.170 *3 https://foo.***.com 02.132709.170 local filename = 'index.html' 02.132709.170 host_add_job: job fname index.html 02.132709.170 host_add_job: 0x5626fc4850c0 https://foo.***.com 02.132709.170 host_add_job: qsize 1 host-qsize=1 02.132709.170 queue_size: qsize=1 02.132709.170 queue_size: qsize=1 02.132709.170 queue_size: qsize=1 02.132709.170 [0] action=1 pending=0 host=0x0 02.132709.170 dequeue job https://foo.***.com 02.132709.170 resolving foo.***.com:443... 02.132709.174 has ***ipv6:443 02.132709.174 has ***ipv4:443 02.132709.174 trying ***ipv6:443... 02.132709.174 GnuTLS init 02.132709.189 GnuTLS system certificate store is empty 02.132709.189 Certificates loaded: 153 02.132709.189 GnuTLS init done 02.132709.189 TLS False Start requested 02.132709.189 ALPN offering h2 02.132709.189 ALPN offering http/1.1

ssl modes:

openssl s_client  ✔  2m 9s  140269842409280:error:0200206F:system library:connect:Connection refused:crypto/bio/b_sock2.c:110: 140269842409280:error:2008A067:BIO routines:BIO_connect:connect error:crypto/bio/b_sock2.c:111: 140269842409280:error:0200206F:system library:connect:Connection refused:crypto/bio/b_sock2.c:110: 140269842409280:error:2008A067:BIO routines:BIO_connect:connect error:crypto/bio/b_sock2.c:111: connect:errno=111

Summary of the one particular case where it breaks:

  • I'm in my wifi (hotspot works)
  • on manjaro (ubuntu works)
  • using wget/node-https/wget2 (curl and browsers work)
Improve this question
6
  • in wireshark if I filter for ip.addr==***, something will pop up with curl, but not with wget Commented May 2, 2022 at 10:12
  • 1
    There's Wget2 - a successor of Wget which uses threads and so is much more faster than Wget. The development usually happens on Wget2. Can you try with Wget2? Commented May 2, 2022 at 10:13
  • wget2 installation via the AUR fails, can't try. I have updated the post a bit. It works on an ubuntu live usb, it is somehow related to manjaro. Commented May 2, 2022 at 11:16
  • 1
    Guess: certificate or SSL mode issues, which depends on what library wget/curl/the https module use. Debug with openssl s_client to see which SSL modes are supported. Commented May 2, 2022 at 11:28
  • got wget2 to work, see post for log Commented May 2, 2022 at 11:29

3 Answers 3

Reset to default
4

Description

The problem seems to be that wget and wget2 (and other tools maybe as well) try the resolved IP addresses one after the other. In your case, the first resolved IP 'hangs' at the connection phase (e.g. due to firewall / NAT settings) until it times out. The default for wget and wget2 is to wait forever.

Work-around

To "fix" this, make use of --connect-timeout=N (N is in seconds, fractions allowed, e.g. 1.5). You may want to put this option into ~/.wgetrc (a single line connect-timeout = N) or into ~/.config/wget2rc or XDG_CONFIG_HOME/wget2rc.

So this is not a general issue with IPv6 vs IPv4, any blocked IP can cause this issue. In your case, you can also investigate why this happens at all and fix it on a different layer.

Why does curl not hang ?

Either curl prefers IPv4 over IPv6 or curl connects to all IPs simultaneously and uses the first (possibly the fastest) connection for communication.
You can read about the second option here.
If this sounds good to you, feel free to open a feature request on Gitlab.

Improve this answer
2
  • thanks, that works! The option is --connect-timeout though, not "connection" Commented May 23, 2022 at 8:16
  • thanks, amended :-) Commented May 25, 2022 at 5:56
1

Using wget https://foo.bar.com -4 it works

Via the smartphones hotspot ipv6 doesn't seem to work at all and it tries ipv4, therefore the request to the server worked.

It seems our other servers don't have ipv6 configured, which is why those worked.

Via our wifi it tried to use ipv6 and failed.

The various online ipv6 test tools say my computers ipv6 connectivity is working correctly, so I'm assuming our server configuration for ipv6 is broken.

Improve this answer
2
0

The reason why curl works and wget doesn't can be found in this comparison made by the creator of curl. It is because it implements Happy Eyeballs, one algorithm created for avoiding clients to wait for failing IPv6 path and protocol.

There were different opinions about this approach. The advantage is that improve the end user experience and the drawback is that hides the network problems, delaying the support of IPv6.

Just to put my example. When I try with a IPv4-only site like s3.us-east-1.amazonaws.com with curl the answer is fast:

curl -v https://s3.us-east-1.amazonaws.com/ * Trying 64:ff9b::34d8:22a0:443... * Trying 52.216.29.166:443... * Connected to s3.us-east-1.amazonaws.com (52.216.29.166) port 443 (#0) ... < Content-Length: 0 < * Connection #0 to host s3.us-east-1.amazonaws.com left intact real 0m0,855s user 0m0,095s sys 0m0,008s

note that curl did try with both addresses answered by the DNS, the IPv6 and IPv4 one. In my case I can not reach those IPv6 addresses, they correspond to broken AAAA records.

On the other hand wget answer took more time because it tried first all the IPv6 addresses returned by the DNS:

$ time wget --debug https://s3.us-east-1.amazonaws.com/ ... Resolving s3.us-east-1.amazonaws.com (s3.us-east-1.amazonaws.com)... 64:ff9b::34d8:2460, 64:ff9b::34d8:ddc8, 64:ff9b::34d8:fba6, ... Caching s3.us-east-1.amazonaws.com => 64:ff9b::34d8:2460 64:ff9b::34d8:ddc8 64:ff9b::34d8:fba6 64:ff9b::36e7:8550 64:ff9b::34d8:d170 64:ff9b::34d9:8578 64:ff9b::34d8:3d40 64:ff9b::36e7:aa98 52.216.209.112 52.216.251.166 54.231.170.152 54.231.133.80 52.216.221.200 52.216.36.96 52.217.133.120 52.216.61.64 Connecting to s3.us-east-1.amazonaws.com (s3.us-east-1.amazonaws.com)|64:ff9b::34d8:2460|:443... Closed fd 3 failed: Connection timed out. Connecting to s3.us-east-1.amazonaws.com (s3.us-east-1.amazonaws.com)|64:ff9b::34d8:ddc8|:443... Closed fd 3 failed: Connection timed out. Connecting to s3.us-east-1.amazonaws.com (s3.us-east-1.amazonaws.com)|64:ff9b::34d8:fba6|:443... Closed fd 3 failed: Connection timed out. Connecting to s3.us-east-1.amazonaws.com (s3.us-east-1.amazonaws.com)|64:ff9b::36e7:8550|:443... Closed fd 3 failed: Connection timed out. Connecting to s3.us-east-1.amazonaws.com (s3.us-east-1.amazonaws.com)|64:ff9b::34d8:d170|:443... Closed fd 3 failed: Connection timed out. Connecting to s3.us-east-1.amazonaws.com (s3.us-east-1.amazonaws.com)|64:ff9b::34d9:8578|:443... Closed fd 3 failed: Connection timed out. Connecting to s3.us-east-1.amazonaws.com (s3.us-east-1.amazonaws.com)|64:ff9b::34d8:3d40|:443... Closed fd 3 failed: Connection timed out. Connecting to s3.us-east-1.amazonaws.com (s3.us-east-1.amazonaws.com)|64:ff9b::36e7:aa98|:443... Closed fd 3 failed: Connection timed out. Connecting to s3.us-east-1.amazonaws.com (s3.us-east-1.amazonaws.com)|52.216.209.112|:443... connected. Created socket 3. Releasing 0x0000560df7694100 (new refcount 1). ... real 17m29,487s user 0m0,022s sys 0m0,013s

Said that, both tools have the -4 and -6 options to force the use of IPv4 or IPv6 respectively.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.