8

On a server wget-1.16 takes 8 minutes to complete:

$ wget http://http.debian.net/debian/dists/stable/Release -O -
--2017-06-12 23:44:40--  http://http.debian.net/debian/dists/stable/Release         [4693/5569]
Resolving http.debian.net (http.debian.net)... 2001:4f8:1:c::15, 2605:bc80:3010:b00:0:deb:166:202, 2001:610:1908:b000::148:14, ...                                                            
Connecting to http.debian.net (http.debian.net)|2001:4f8:1:c::15|:80... failed: Connection timed out.                 
Connecting to http.debian.net (http.debian.net)|2605:bc80:3010:b00:0:deb:166:202|:80... failed: Connection timed out. 
Connecting to http.debian.net (http.debian.net)|2001:610:1908:b000::148:14|:80... failed: Connection timed out.       
Connecting to http.debian.net (http.debian.net)|140.211.166.202|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://cdn-fastly.deb.debian.org/debian/dists/stable/Release [following]
--2017-06-12 23:51:02--  http://cdn-fastly.deb.debian.org/debian/dists/stable/Release
Resolving cdn-fastly.deb.debian.org (cdn-fastly.deb.debian.org)... 2a04:4e42:3::204, 151.101.12.204                   
Connecting to cdn-fastly.deb.debian.org (cdn-fastly.deb.debian.org)|2a04:4e42:3::204|:80... failed: Connection timed out.
Connecting to cdn-fastly.deb.debian.org (cdn-fastly.deb.debian.org)|151.101.12.204|:80... connected.
...

Because it is trying to connect using IPv6 address. curl-7.38.0 on the same machine responds instantly. Because it uses IPv4 address. Do they resolve domain differently? How do they do it? How can I make wget use IPv4 address?

UPD

$ ip a
...
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether d8:cb:8a:37:cf:57 brd ff:ff:ff:ff:ff:ff
    inet 188.40.99.4/26 brd 188.40.99.63 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 2a01:4f8:100:738b::2/64 scope global 
       valid_lft forever preferred_lft forever 
    inet6 fe80::dacb:8aff:fe37:cf57/64 scope link                                                     valid_lft forever preferred_lft forever 

$ ip route
default via 188.40.99.1 dev eth0 
10.0.0.0/24 dev br0  proto kernel  scope link  src 10.0.0.1 
188.40.99.0/26 via 188.40.99.1 dev eth0 
188.40.99.0/26 dev eth0  proto kernel  scope link  src 188.40.99.4
x-yuri
  • 3,223
  • 10
  • 39
  • 62
  • 4
    Use `--inet4-only` perhaps? – DopeGhoti Jun 12 '17 at 21:30
  • Or maybe `--bind-address=` in case your server has multiple IPv4 addresses. – Deathgrip Jun 12 '17 at 21:53
  • Actually, the bigger picture is, the issue manifested itself when creating lxc container with `lxc-create`. So I can't possibly add extra switches to `wget`. But more importantly, why `curl` succeeds fast, but `wget` doesn't? Doesn't it all come down to some system call they both should use? – x-yuri Jun 12 '17 at 22:00
  • Your system seems to think that it has IPv6 connectivity without actually having it. Can you show your interface addresses and routing table? – Sander Steffann Jun 12 '17 at 22:43
  • @SanderSteffann Please, take a look at the question. Is that it? – x-yuri Jun 12 '17 at 23:47
  • 1
    You have a global IPv6 address. You only show the IPv4 routing table, so I can't see if you have a default IPv6 route, but I'm pretty sure you do. You can check with `ip -6 route`. The problem is that something isn't working. The fast failover of curl masks the problem. The real problem is that your IPv6 configuration isn't working. You might need to talk to your network admin or Idaho to find our what exactly is wrong. – Sander Steffann Jun 13 '17 at 03:53
  • I second Sander, I came to the comments to write exactly the same thing. You might have IPv6 badly defined at your infra-structure, or blocked at firewall level. The accepted solution only masks the problem. – Rui F Ribeiro Jun 13 '17 at 09:03

1 Answers1

13

curl and wget do not use different mechanisms for resolving domains (they're using getaddrinfo()). However, curl implements a fast fallback algorithm to improve the user experience in cases where IPv6 connectivity is less than good.

This algorithm is described in detail in RFC 6555 (Happy Eyeballs): https://www.rfc-editor.org/rfc/rfc6555

According to curl/lib/connect.h this timeout is set to 200ms: https://github.com/curl/curl/blob/a8e523f086c12e7bb9acb18d1ac84d92dde0605b/lib/connect.h#L43

Both curl and wget support -4/-6 options which will force the connection to either IPv4 or IPv6 respectively.

Community
  • 1
user234931
  • 684
  • 4
  • 8