socket's strange behavior with subprocesses

Discussion in 'Python' started by Jane Austine, Nov 12, 2003.

  1. Jane Austine

    Jane Austine Guest

    Running Python 2.3 on Win XP

    It seems like socket is working interdependently with subprocesses of
    the process which created socket.

    ------------------------------------
    #the server side
    >>> import socket
    >>> s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
    >>> s.bind(('localhost',9000))
    >>> s.listen(5)
    >>> z=s.accept()
    >>> import os
    >>> f=os.popen('notepad.exe') #notepad appears on the screen
    >>> z[0]

    <socket._socketobject object at 0x0096C390>
    >>> z[0].recv(6)

    'foobar'
    >>> z[0].send('hello world')

    11

    #the client side
    >>> s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
    >>> s.connect(('localhost',9000))
    >>> s.send('foobar')

    4
    >>> print s.recv(1024)

    hello world

    ------------------------------
    Now when the client requests to recv 1024 bytes, and since there is no
    more to read from the socket it blocks.

    #client side
    >>> s.recv(1024) #it hangs


    and the server side tries to close the socket:

    #server side
    >>> z[0].close()
    >>> #yes, it seems to have worked.


    Alas, the client side doesn't wake up! It doesn't wake up unless the
    notepad is exited first; only after that, 'Connection reset by peer'
    is raised. What does the socket has to do with subprocesses?
     
    Jane Austine, Nov 12, 2003
    #1
    1. Advertising

  2. Jane Austine wrote:

    > and the server side tries to close the socket:
    >
    > #server side
    >
    >>>>z[0].close()
    >>>>#yes, it seems to have worked.

    >
    >
    > Alas, the client side doesn't wake up! It doesn't wake up unless the
    > notepad is exited first; only after that, 'Connection reset by peer'
    > is raised. What does the socket has to do with subprocesses?


    Nothing, I guess... try to shutdown the socket explicitly before closing it:


    z[0].shutdown(2)
    z[0].close()

    does that work?

    --Irmen
     
    Irmen de Jong, Nov 12, 2003
    #2
    1. Advertising

  3. Jane Austine

    Colin Brown Guest

    "Jane Austine" <> wrote in message
    news:...
    > Running Python 2.3 on Win XP
    >
    > It seems like socket is working interdependently with subprocesses of
    > the process which created socket.
    > and the server side tries to close the socket:

    ......
    > Alas, the client side doesn't wake up! It doesn't wake up unless the
    > notepad is exited first; only after that, 'Connection reset by peer'
    > is raised. What does the socket has to do with subprocesses?


    Hi Jane

    I think it may be tied up with the way subprocesses are made under Windows.
    Below is a copy of part of a post I made some months back when inexplicable
    file-locking was causing me problems. It took me weeks to track down the
    reason.

    The Python default subprocess inherits open handles from the parent.
    Your options are:
    1. Open the subprocess before the socket.
    2. Create the subprocess using win32 API primitives (this is what I had to
    do).
    Let me know if want details.

    Colin Brown
    PyNZ

    ----------------------------------------------------------------------------
    ------
    I have been struggling to solve why an occasional "Permission Denied" error
    popped up from the following code fragment in one of my Win2K program
    threads:

    ....
    input_file = preprocess(raw_file)
    os.system('third_party input_file output_file > error_file')
    if os.path.exists(saved_file):
    os.remove(saved_file)
    os.rename(input_file,saved_file)
    ....

    The error occurs on the os.rename(). The third_party executable was closing
    input_file before terminating and anyway the subshell process has finished
    before the os.rename is called! Most baffling.

    With the aid of handle.exe from www.sysinternals.com I finally resolved the
    problem. To see the problem at first hand try the following script:

    import time, thread, os

    def rm(file):
    print 'delete x.x'
    os.remove(file)

    def wait():
    if os.name == 'nt':
    os.system('pause')
    elif os.name == 'posix':
    os.system('sleep 3')
    print 'end wait'

    print 'create x.x'
    f = open('x.x','w')
    thread.start_new_thread(wait,())
    time.sleep(1)
    print '\nclose x.x'
    f.close()
    rm('x.x')

    Although this works fine on Linux, I get a Permission Denied error on
    Windows. I surmise that an os.system call uses a C fork to create the
    subshell - an exact duplicate of the current process environment including
    OPEN FILE HANDLES. Because the os.system call has not returned before the
    os.remove is called a "Permission Denied" error occurs. Quite simple really.

    Now going back to my original problem, I had other threads doing os.system
    calls. When one of these calls happens during the preprocess file write of
    my above thread AND takes longer to complete than the os.system call in the
    above thread then the file will still be open and the os.rename will return
    the Permission Denied error.
     
    Colin Brown, Nov 12, 2003
    #3
  4. Jane Austine

    Jane Austine Guest

    "Colin Brown" <> wrote in message news:<3fb28c56$>...
    > "Jane Austine" <> wrote in message
    > news:...
    > > Running Python 2.3 on Win XP
    > >
    > > It seems like socket is working interdependently with subprocesses of
    > > the process which created socket.
    > > and the server side tries to close the socket:

    > .....
    > > Alas, the client side doesn't wake up! It doesn't wake up unless the
    > > notepad is exited first; only after that, 'Connection reset by peer'
    > > is raised. What does the socket has to do with subprocesses?

    >
    > Hi Jane
    >
    > I think it may be tied up with the way subprocesses are made under Windows.
    > Below is a copy of part of a post I made some months back when inexplicable
    > file-locking was causing me problems. It took me weeks to track down the
    > reason.
    >
    > The Python default subprocess inherits open handles from the parent.
    > Your options are:
    > 1. Open the subprocess before the socket.
    > 2. Create the subprocess using win32 API primitives (this is what I had to
    > do).
    > Let me know if want details.
    >
    > Colin Brown
    > PyNZ
    >


    Thank you very much, first of all.

    I tried win32 API primitives for creating subprocesses: win32all's
    CreateProcess. I used the Process class in winprocess.py in the
    "demos" directory. However, it didn't work with sockets perfectly.

    Say, a socket server python script is launched via CreateProcess. It
    then launches sub-processes. And I kill(via TerminateProcess) the
    socket server process. The subprocesses remain alive(as expected). I
    try to connect to the 'dead server port'. I expect almost immediate
    "connect refused" error, but it doesn't come up until the subprocesses
    are all gone and client hangs forever.

    Jane
     
    Jane Austine, Nov 13, 2003
    #4
  5. Jane Austine

    Colin Brown Guest

    "Jane Austine" <> wrote in message
    news:...
    > "Colin Brown" <> wrote in message

    news:<3fb28c56$>...
    > > "Jane Austine" <> wrote in message
    > > news:...

    ....
    > I tried win32 API primitives for creating subprocesses: win32all's
    > CreateProcess. I used the Process class in winprocess.py in the
    > "demos" directory. However, it didn't work with sockets perfectly.
    >
    > Say, a socket server python script is launched via CreateProcess. It
    > then launches sub-processes. And I kill(via TerminateProcess) the
    > socket server process. The subprocesses remain alive(as expected). I
    > try to connect to the 'dead server port'. I expect almost immediate
    > "connect refused" error, but it doesn't come up until the subprocesses
    > are all gone and client hangs forever.
    >
    > Jane


    If I am interpreting what you are saying here correctly you have:

    Main_process
    => [create_process]
    => Sub_process1
    socket_server_connection
    subprocess2 (of subprocess1) started

    Sub_process1 terminated, but socket connection held until subprocess2
    terminated.

    This is what I would expect based on my findings. Subprocess2 has inherited
    the socket handle when it was created (assuming you used os.system,
    os.spawn* or os.popen*). You would have to use the correct incantation of
    create_process to launch subprocess2.

    I have attached the code I used in place of os.system.

    Colin

    --[Win32.py]---------------------------------------------------------------
    # perform equivalent of os.system without open file handles

    import win32process,win32event

    def system(cmd):
    handles = win32process.CreateProcess \
    (None,cmd,None,None,0,0,None,None,win32process.STARTUPINFO())
    status = win32event.WaitForSingleObject(handles[0],win32event.INFINITE)
    if status == win32event.WAIT_ABANDONED:
    raise 'win32.system WAIT_ABANDONED'
    elif status == win32event.WAIT_FAILED:
    raise 'win32.system WAIT_FAILED'
    elif status == win32event.WAIT_IO_COMPLETION:
    raise 'win32.system WAIT_IO_COMPLETION'
    elif status == win32event.WAIT_OBJECT_0:
    pass
    elif status == win32event.WAIT_TIMEOUT:
    raise 'win32.system WAIT_TIMEOUT'
    else:
    raise 'win32.system - unknown event status = '+str(status)
     
    Colin Brown, Nov 13, 2003
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Laszlo Nagy
    Replies:
    1
    Views:
    5,104
    Mark Wooding
    Jan 27, 2009
  2. Jean-Paul Calderone
    Replies:
    0
    Views:
    1,028
    Jean-Paul Calderone
    Jan 27, 2009
  3. Laszlo Nagy
    Replies:
    0
    Views:
    598
    Laszlo Nagy
    Feb 1, 2009
  4. Steve Holden
    Replies:
    0
    Views:
    712
    Steve Holden
    Feb 1, 2009
  5. Steve Holden
    Replies:
    1
    Views:
    756
Loading...

Share This Page