Strange socket problem

Discussion in 'Python' started by huy, Jun 15, 2005.

  1. huy

    huy Guest

    Hi,

    I'm using cherrypy to provide a user interface for users to start a
    linux server program eg. os.system("nohup myserver.py &"). The problem
    is that if I stop cherrypy server and restart, I get the "Address
    Already In Use" problem until I stop myserver.py. Can someone shed some
    light on why this happens ? Why would the socket be held up in the other
    process ?

    Thanks

    Huy
     
    huy, Jun 15, 2005
    #1
    1. Advertising

  2. huy

    Jeff Epler Guest

    When using os.system(), files that are open in the parent are available
    in the child, as you can see here in Linux' listing of the files open by
    the child program:

    [jepler@sofa jepler]$ python -c 'f = open("/tmp/test", "w"); print f.fileno(); import os; os.system("ls -l /proc/self/fd")'
    3
    total 5
    lrwx------ 1 jepler jepler 64 Jun 15 07:25 0 -> /dev/pts/2
    lrwx------ 1 jepler jepler 64 Jun 15 07:25 1 -> /dev/pts/2
    lrwx------ 1 jepler jepler 64 Jun 15 07:25 2 -> /dev/pts/2
    l-wx------ 1 jepler jepler 64 Jun 15 07:25 3 -> /tmp/test
    lr-x------ 1 jepler jepler 64 Jun 15 07:25 4 -> /proc/3108/fd

    You may be able to set the FD_CLOEXEC flag on the files that should not
    be passed to children, something like this:
    old = fcntl.fcntl(fd, fcntl.F_GETFD)
    fcntl.fcntl(fd, fcntl.F_SETFD, old | fcntl.FD_CLOEXEC)
    Refer to a real Unix reference for more information on what FD_CLOEXEC
    does.

    You may want to replace the use of os.system() with fork + close files
    + exec. Then "myserver.py" will no longer have the listening socket
    descriptor of your cherrypy server.

    Python 2.4's 'subprocess' module may work better in this respect than
    os.system. It seems to include support for requesting that fds be
    closed in the child (the close_fds parameter to subprocess.Popen)

    Jeff

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.2.6 (GNU/Linux)

    iD8DBQFCsB/gJd01MZaTXX0RArdnAJwN9iek350cqXtYDdUzzSomz6dQIgCdFaC5
    Ebd2XyIMsS+QgoLQB5jyPMU=
    =9XcR
    -----END PGP SIGNATURE-----
     
    Jeff Epler, Jun 15, 2005
    #2
    1. Advertising

  3. huy

    Magnus Lycka Guest

    huy wrote:
    > Hi,
    >
    > I'm using cherrypy to provide a user interface for users to start a
    > linux server program eg. os.system("nohup myserver.py &"). The problem
    > is that if I stop cherrypy server and restart, I get the "Address
    > Already In Use" problem until I stop myserver.py. Can someone shed some
    > light on why this happens ? Why would the socket be held up in the other
    > process ?


    As I understand it, if a socket is closed by the server, the OS
    will typically hold on to the socket for a while, in case the
    client wants some packages resent etc. This might be a part of
    your problem.

    Andrew Dalke explained it much better to me when I ran into this
    some time ago. Quoted below:

    > Magnus Lyckå wrote:
    >> Why doesn't my socket
    >> get released by the OS when I exit via my handle_error?

    >
    > Hi Magnus,
    >
    > I wrote about this at
    > http://www.dalkescientific.com/writings/diary/archive/2005/04/21/using_xmlrpc.html
    >
    > The reason for it is described at
    > http://hea-www.harvard.edu/~fine/Tech/addrinuse.html
    >
    > You can set the class variable "allow_reuse_address = True" in
    > your derived ExitableSocketServer to get the behaviour you
    > expect, at the expense of some problems mentioned in the
    > above URL.
    >
    > Andrew
    >
     
    Magnus Lycka, Jun 15, 2005
    #3
  4. huy

    Guest

    Gurus,

    I am still doing my baby steps in the wonderful world of python (so
    far, so good).
    However, I am quite familiar with sockets. There is a socket option
    called
    SO_REUSEADDR that your server should call to fix this problem.
     
    , Jun 15, 2005
    #4
  5. huy

    huy Guest

    Hi Jeff,

    Thanks for your help. Although I haven't confirmed this, I think you
    just hit my nail on the head. I thought os.system was like a totally
    separate process though, i.e nothing is shared. not the usual fork()
    call within the program.

    Regards,

    Huy

    Jeff Epler wrote:
    > When using os.system(), files that are open in the parent are available
    > in the child, as you can see here in Linux' listing of the files open by
    > the child program:
    >
    > [jepler@sofa jepler]$ python -c 'f = open("/tmp/test", "w"); print f.fileno(); import os; os.system("ls -l /proc/self/fd")'
    > 3
    > total 5
    > lrwx------ 1 jepler jepler 64 Jun 15 07:25 0 -> /dev/pts/2
    > lrwx------ 1 jepler jepler 64 Jun 15 07:25 1 -> /dev/pts/2
    > lrwx------ 1 jepler jepler 64 Jun 15 07:25 2 -> /dev/pts/2
    > l-wx------ 1 jepler jepler 64 Jun 15 07:25 3 -> /tmp/test
    > lr-x------ 1 jepler jepler 64 Jun 15 07:25 4 -> /proc/3108/fd
    >
    > You may be able to set the FD_CLOEXEC flag on the files that should not
    > be passed to children, something like this:
    > old = fcntl.fcntl(fd, fcntl.F_GETFD)
    > fcntl.fcntl(fd, fcntl.F_SETFD, old | fcntl.FD_CLOEXEC)
    > Refer to a real Unix reference for more information on what FD_CLOEXEC
    > does.
    >
    > You may want to replace the use of os.system() with fork + close files
    > + exec. Then "myserver.py" will no longer have the listening socket
    > descriptor of your cherrypy server.
    >
    > Python 2.4's 'subprocess' module may work better in this respect than
    > os.system. It seems to include support for requesting that fds be
    > closed in the child (the close_fds parameter to subprocess.Popen)
    >
    > Jeff
     
    huy, Jun 15, 2005
    #5
  6. huy

    huy Guest

    Hi Jeff,

    Thanks for your help. Although I haven't confirmed this, I think you
    just hit my nail on the head. I thought os.system was like a totally
    separate process though, i.e nothing is shared. not the usual fork()
    call within the program.

    Regards,

    Huy

    Jeff Epler wrote:
    > When using os.system(), files that are open in the parent are available
    > in the child, as you can see here in Linux' listing of the files open by
    > the child program:
    >
    > [jepler@sofa jepler]$ python -c 'f = open("/tmp/test", "w"); print f.fileno(); import os; os.system("ls -l /proc/self/fd")'
    > 3
    > total 5
    > lrwx------ 1 jepler jepler 64 Jun 15 07:25 0 -> /dev/pts/2
    > lrwx------ 1 jepler jepler 64 Jun 15 07:25 1 -> /dev/pts/2
    > lrwx------ 1 jepler jepler 64 Jun 15 07:25 2 -> /dev/pts/2
    > l-wx------ 1 jepler jepler 64 Jun 15 07:25 3 -> /tmp/test
    > lr-x------ 1 jepler jepler 64 Jun 15 07:25 4 -> /proc/3108/fd
    >
    > You may be able to set the FD_CLOEXEC flag on the files that should not
    > be passed to children, something like this:
    > old = fcntl.fcntl(fd, fcntl.F_GETFD)
    > fcntl.fcntl(fd, fcntl.F_SETFD, old | fcntl.FD_CLOEXEC)
    > Refer to a real Unix reference for more information on what FD_CLOEXEC
    > does.
    >
    > You may want to replace the use of os.system() with fork + close files
    > + exec. Then "myserver.py" will no longer have the listening socket
    > descriptor of your cherrypy server.
    >
    > Python 2.4's 'subprocess' module may work better in this respect than
    > os.system. It seems to include support for requesting that fds be
    > closed in the child (the close_fds parameter to subprocess.Popen)
    >
    > Jeff
     
    huy, Jun 15, 2005
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Laszlo Nagy
    Replies:
    1
    Views:
    5,091
    Mark Wooding
    Jan 27, 2009
  2. Jean-Paul Calderone
    Replies:
    0
    Views:
    1,027
    Jean-Paul Calderone
    Jan 27, 2009
  3. Laszlo Nagy
    Replies:
    0
    Views:
    595
    Laszlo Nagy
    Feb 1, 2009
  4. Steve Holden
    Replies:
    0
    Views:
    711
    Steve Holden
    Feb 1, 2009
  5. Steve Holden
    Replies:
    1
    Views:
    756
Loading...

Share This Page