bad httplib latency due to IPv6 use

Discussion in 'Python' started by Ulrich Eckhardt, Oct 17, 2012.

  1. Hi!

    I noticed yesterday that a single HTTP request to localhost takes
    roughly 1s, regardless of the actually served data, which is way too
    long. After some digging, I found that the problem lies in
    socket.create_connection(), which first tries the IPv6 ::1 and only then
    tries the IPv4 127.0.0.1. The first one times out after 1s, causing the
    long latency.

    What I'm wondering is this:
    1. The server only serves on IPv4, changing this to IPv6 would probably
    help. However, I wouldn't consider this a bug, or?
    2. I don't even have any IPv6 addresses configured and I'm not using
    IPv6 in any way, so why does it try those at all?
    3. Of course I can optimize the code for IPv4, but then I would be
    pessimizing IPv6 and vice versa...

    Any other suggestions?

    Uli


    Notes:
    * Using 127.0.0.1 as host works without the delay.
    * I'm using Python 2.7 on win7/64bit
     
    Ulrich Eckhardt, Oct 17, 2012
    #1
    1. Advertising

  2. Ulrich Eckhardt

    Hans Mulder Guest

    On 17/10/12 09:55:13, Ulrich Eckhardt wrote:
    > Hi!
    >
    > I noticed yesterday that a single HTTP request to localhost takes
    > roughly 1s, regardless of the actually served data, which is way too
    > long. After some digging, I found that the problem lies in
    > socket.create_connection(), which first tries the IPv6 ::1 and only then
    > tries the IPv4 127.0.0.1. The first one times out after 1s, causing the
    > long latency.
    >
    > What I'm wondering is this:
    > 1. The server only serves on IPv4, changing this to IPv6 would probably
    > help. However, I wouldn't consider this a bug, or?


    I'd say it's a bug in your TCP/IP stack. An IP shouldn't take that long
    to figure out that it is not configured to connect to IPv6 addresses.

    > 2. I don't even have any IPv6 addresses configured and I'm not using
    > IPv6 in any way, so why does it try those at all?


    I have no experience with win7/64, but on earlier versions of Windows,
    there's a file named "hosts", somewhere in a system directory. When
    looking up an IP address, this file is consulted first. Removing the
    ::1 from the entry for localhost might be a sensible work-around.

    > 3. Of course I can optimize the code for IPv4, but then I would be
    > pessimizing IPv6 and vice versa...


    I'd avoid doing that, if at all possible.

    > Any other suggestions?


    Try "ping localhost" on the command line. It that has the same problem,
    than the problem is not in Python, but in your IP stack. It might be in
    a FAQ list for some windows-specific forum.

    > Notes:
    > * Using 127.0.0.1 as host works without the delay.


    That make sense: it's the attempt to connect to ::1 that is the problem.

    > * I'm using Python 2.7 on win7/64bit



    Hope his helps,

    -- HansM
     
    Hans Mulder, Oct 17, 2012
    #2
    1. Advertising

  3. On Wed, Oct 17, 2012 at 8:37 PM, Hans Mulder <> wrote:
    > I have no experience with win7/64, but on earlier versions of Windows,
    > there's a file named "hosts", somewhere in a system directory. When
    > looking up an IP address, this file is consulted first. Removing the
    > ::1 from the entry for localhost might be a sensible work-around.


    It ought to be c:\windows\system32\drivers\etc\hosts if your Widows is
    installed to default location. If localhost isn't there at all, you
    can add a line for it.

    ChrisA
     
    Chris Angelico, Oct 17, 2012
    #3
  4. Ulrich Eckhardt

    Laszlo Nagy Guest


    >> What I'm wondering is this:
    >> 1. The server only serves on IPv4, changing this to IPv6 would probably
    >> help. However, I wouldn't consider this a bug, or?

    > I'd say it's a bug in your TCP/IP stack. An IP shouldn't take that long
    > to figure out that it is not configured to connect to IPv6 addresses.

    It might also be, that he has a firewall installed that is blocking
    access to ::1. In that case, it takes much more time to figure out that
    you cannot connect. Because in that case, it is not a "connection
    refused" problem, but a "trying to connect to a closed/not responding
    port" problem.
     
    Laszlo Nagy, Oct 17, 2012
    #4
  5. Ulrich Eckhardt

    Roy Smith Guest

    In article <>,
    Ulrich Eckhardt <> wrote:

    > Hi!
    >
    > I noticed yesterday that a single HTTP request to localhost takes
    > roughly 1s, regardless of the actually served data, which is way too
    > long. After some digging, I found that the problem lies in
    > socket.create_connection(), which first tries the IPv6 ::1 and only then
    > tries the IPv4 127.0.0.1. The first one times out after 1s, causing the
    > long latency.
    >
    > What I'm wondering is this:
    > 1. The server only serves on IPv4, changing this to IPv6 would probably
    > help. However, I wouldn't consider this a bug, or?


    This is most likely a configuration problem on your server. You don't
    say what kind of system you're using, so I can only guess at the exact
    answer, but here's some suggestions.

    Look at your /etc/hosts file. What entries do you have for localhost?
    For example, on my OSX laptop, I've got:

    127.0.0.1 localhost
    ::1 localhost
    fe80::1%lo0 localhost

    which gives three IP addresses (one IPv4, two IPv6) for "localhost".
    This works because the box is properly configured for IPv6 (i.e. all
    services of interest listen on both protocols). On the other hand, this
    is on a linux box I have:

    127.0.0.1 localhost
    ::1 ip6-localhost ip6-loopback

    Now, localhost only gives me the IPv4 address. If I want to connect to
    localhost with IPv6, I have to explicitly say "ip6-localhost".

    My guess is that you have some variation of the first example, and what
    you need is some variation on the second.

    In the alternative, you need to fix up your HTTP server to listen on
    both protocols. For sure, in the long term, that's the right solution,
    because Real Soon Now, we're going to run out of IPv4 addresses and
    everybody is going to switch to IPv6. Of course, that's been true for
    about the past 15 years, but now it's really, really true.
     
    Roy Smith, Oct 17, 2012
    #5
  6. Some updates on the issue:

    The etc/hosts file contains the following lines:

    # localhost name resolution is handled within DNS itself.
    # 127.0.0.1 localhost
    # ::1 localhost

    As I understand it, those effectively mean that localhost is not
    resolved via this hosts file but "within DNS itself", whatever that
    exactly means.


    Concerning the question whether ping works, the result is that "ping
    localhost" works and that it uses the IPv6 (sic!) address. I also tried
    "ping ::1" and "ping 127.0.0.1" and both work. Weird, as ipconfig
    doesn't list any IPv6 addresses.


    Concerning the question whether a firewall blocks and unnecessarily
    delays connection attempts to ::1, I haven't determined that yet. I'll
    ask our admins here to verify whether that is the case.


    Lastly, I tried the same using Python 3.2.3/64bit (the other was
    actually the 32-bit version), and the same issues are there. In summary,
    I guess that it's a problem with the IP configuration not one in
    Python's or my code.

    Sorry for the noise...

    Uli
     
    Ulrich Eckhardt, Oct 17, 2012
    #6
  7. On Wed, Oct 17, 2012 at 11:40 PM, Ulrich Eckhardt
    <> wrote:
    > Concerning the question whether a firewall blocks and unnecessarily delays
    > connection attempts to ::1, I haven't determined that yet. I'll ask our
    > admins here to verify whether that is the case.


    It would only be a software firewall on the local machine that could
    be doing that; anything on any other machine can't. If you have no
    particular reason to do otherwise, I would recommend removing all
    firewalling from localhost - you won't lose much security (two
    programs running on your computer are allowed to communicate freely in
    plenty of other ways anyway), and it'll make all sorts of things
    easier/simpler.

    ChrisA
     
    Chris Angelico, Oct 17, 2012
    #7
  8. Ulrich Eckhardt

    Roy Smith Guest

    In article <>,
    Ulrich Eckhardt <> wrote:

    > Some updates on the issue:
    >
    > The etc/hosts file contains the following lines:
    >
    > # localhost name resolution is handled within DNS itself.
    > # 127.0.0.1 localhost
    > # ::1 localhost
    >
    > As I understand it, those effectively mean that localhost is not
    > resolved via this hosts file but "within DNS itself", whatever that
    > exactly means.


    The path from hostname to IP address is a long and tangled one. It
    starts with a call to getaddrinfo(). From there, the details depend on
    a wide variety of decisions made by whatever idiots designed, installed,
    and configured your operating system.

    One likely path is to check in /etc/nsswitch.conf to see what data
    sources the resolver should consult. On the box I'm using at the
    moment, it says:

    hosts: files dns

    which is pretty typical. That means, first look in [some set of static
    files, which usually, but not always, means /etc/hosts], and if you
    don't find it there, ask DNS. Other possibilities include NIS, NISPLUS,
    and maybe some other perverse things.

    And, finally, you get to DNS, so now you have to look in (probably)
    /etc/resolv.conf. I see on my linux box, even that has grown more
    tendrils; there's a whole /etc/resolvconf *directory* full of more
    config files which describe additional ways somebody could have
    misconfigured this mess.

    Personally, if I were you (and assuming you don't have some corporate IT
    nazis to deal with), I would just put localhost in /etc/hosts and be
    done with it.
     
    Roy Smith, Oct 18, 2012
    #8
  9. On Thu, Oct 18, 2012 at 11:28 AM, Roy Smith <> wrote:
    > One likely path is to check in /etc/nsswitch.conf to see what data
    > sources the resolver should consult. On the box I'm using at the
    > moment, it says:
    >
    > hosts: files dns


    This is true on Linux, and presumably on various other Unices, but the
    OP is on Windows. I think it's hard-coded on Windows to just use the
    hosts file and then DNS.

    ChrisA
     
    Chris Angelico, Oct 18, 2012
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    5
    Views:
    421
  2. Michael Ekstrand
    Replies:
    0
    Views:
    343
    Michael Ekstrand
    Aug 8, 2005
  3. Fresh
    Replies:
    2
    Views:
    663
    Bo Persson
    Apr 22, 2008
  4. rantingrick
    Replies:
    44
    Views:
    1,304
    Peter Pearson
    Jul 13, 2010
  5. Jesse van den Kieboom

    Ipv6 on a ipv4/ipv6 hostname

    Jesse van den Kieboom, Jun 5, 2005, in forum: Ruby
    Replies:
    1
    Views:
    396
    Dibya Prakash
    Jun 5, 2005
Loading...

Share This Page