Python 2.6, multiprocessing module and BSD

Discussion in 'Python' started by YouCanCallMeAl@gmail.com, Oct 21, 2008.

  1. Guest

    It seems that the multiprocessing module in 2.6 is broken for *BSD;
    I've seen issue 3770 regarding this. I'm curious if there are more
    details on this issue since the posts in 3770 were a bit unclear. For
    example, one post claimed that the problem was that sem_open isn't
    implemented in *BSD, but it is available on FreeBSD 7 (I checked). I'd
    be willing to help get this working if someone could point me in the
    right direction.
    , Oct 21, 2008
    #1
    1. Advertising

  2. On Oct 21, 2008, at 6:45 PM, wrote:

    > It seems that the multiprocessing module in 2.6 is broken for *BSD;
    > I've seen issue 3770 regarding this. I'm curious if there are more
    > details on this issue since the posts in 3770 were a bit unclear. For
    > example, one post claimed that the problem was that sem_open isn't
    > implemented in *BSD, but it is available on FreeBSD 7 (I checked). I'd
    > be willing to help get this working if someone could point me in the
    > right direction.



    Hi Al,
    I don't know anything about the multiprocessing module, but I have
    some recent experience with the semaphores on FreeBSD 6 & 7 as a
    result of implementing my posix_ipc module.

    Since you mentioned sem_open, I assume you're talking about POSIX
    semaphores, rather than System V semaphores, yes? On FreeBSD 6 & 7
    sem_open exists and works, but support for said semaphores are listed
    as "very experimental". I experienced a problem that looked like a bug
    to me, and a rather big one at that. There's more detail here (scroll
    down to the part about FreeBSD 6/7) including a link to the bug report
    I filed against the FreeBSD kernel:

    http://semanchuk.com/philip/posix_ipc/#platforms


    HTH
    Philip
    Philip Semanchuk, Oct 22, 2008
    #2
    1. Advertising

  3. Jesse Noller Guest

    On Tue, Oct 21, 2008 at 6:45 PM, <> wrote:
    > It seems that the multiprocessing module in 2.6 is broken for *BSD;
    > I've seen issue 3770 regarding this. I'm curious if there are more
    > details on this issue since the posts in 3770 were a bit unclear. For
    > example, one post claimed that the problem was that sem_open isn't
    > implemented in *BSD, but it is available on FreeBSD 7 (I checked). I'd
    > be willing to help get this working if someone could point me in the
    > right direction.
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >


    The BSD issue was raised late in the cycle for 2.6. The problem is
    that FBSD's support is "very experimental" as Phillip points out - and
    OpenBSD doesn't even have them.

    Due to the lateness of the issue and a finite amount of time I have to
    work on things, I chose to disable support for this on the various
    *BSDs until I can cook up a stable patch or have one provided by
    someone more familiar with the inner workings of Free-BSD. OpenBSD
    support is a non-starter.

    Ideally, I would like to get this fixed and put on the 2.6 maint
    branch ASAP, but I haven't had a chance to circle back to it.

    Also note Nick's comment in that bug: "Unfortunately, our OpenBSD and
    FreeBSD buildbots are so unreliable that they don't get much attention
    when they go red"

    Stable reliable buildbots and a few more volunteers more familiar with
    BSDs might be a great and welcome addition to python-dev.

    As for getting this working - I would love a patch. You are going to
    want to start with python-trunk and look in setup.py. You are going to
    want to adjust the flags the package uses:

    elif platform in ('freebsd5', 'freebsd6', 'freebsd7', 'freebsd8'):
    # FreeBSD's P1003.1b semaphore support is very experimental
    # and has many known problems. (as of June 2008)
    macros = dict( # FreeBSD
    HAVE_SEM_OPEN=0,
    HAVE_SEM_TIMEDWAIT=0,
    HAVE_FD_TRANSFER=1,
    )
    libraries = []

    You will also need to look at: Lib/multiprocessing/synchronize.py to
    disable the import error - Modules/_multiprocessing/multiprocessing.h
    will need to be updated for the proper ifdefs for the bsd(s) as well.
    Finally, the core of the semaphore usage is in
    Modules/_multiprocessing/semaphore.c

    I apologize we/I could not get this in for 2.6

    -jesse
    Jesse Noller, Oct 22, 2008
    #3
  4. Guest

    On Oct 21, 8:08 pm, Philip Semanchuk <> wrote:
    > On Oct 21, 2008, at 6:45 PM, wrote:
    >
    > > It seems that the multiprocessing module in 2.6 is broken for *BSD;
    > > I've seen issue 3770 regarding this. I'm curious if there are more
    > > details on this issue since the posts in 3770 were a bit unclear. For
    > > example, one post claimed that the problem was that sem_open isn't
    > > implemented in *BSD, but it is available on FreeBSD 7 (I checked). I'd
    > > be willing to help get this working if someone could point me in the
    > > right direction.

    >
    > Hi Al,
    > I don't know anything about the multiprocessing module, but I have  
    > some recent experience with the semaphores on FreeBSD 6 & 7 as a  
    > result of implementing my posix_ipc module.
    >
    > Since you mentioned sem_open, I assume you're talking about POSIX  
    > semaphores, rather than System V semaphores, yes? On FreeBSD 6 & 7  
    > sem_open exists and works, but support for said semaphores are listed  
    > as "very experimental". I experienced a problem that looked like a bug  
    > to me, and a rather big one at that. There's more detail here (scroll  
    > down to the part about FreeBSD 6/7) including a link to the bug report  
    > I filed against the FreeBSD kernel:
    >
    > http://semanchuk.com/philip/posix_ipc/#platforms
    >
    > HTH
    > Philip


    Thanks for the background on FreeBSD semaphores. I read through the
    link; it should help.
    , Oct 22, 2008
    #4
  5. Guest

    On Oct 22, 8:11 am, "Jesse Noller" <> wrote:
    > On Tue, Oct 21, 2008 at 6:45 PM,  <> wrote:
    > > It seems that the multiprocessing module in 2.6 is broken for *BSD;
    > > I've seen issue 3770 regarding this. I'm curious if there are more
    > > details on this issue since the posts in 3770 were a bit unclear. For
    > > example, one post claimed that the problem was that sem_open isn't
    > > implemented in *BSD, but it is available on FreeBSD 7 (I checked). I'd
    > > be willing to help get this working if someone could point me in the
    > > right direction.
    > > --
    > >http://mail.python.org/mailman/listinfo/python-list

    >
    > The BSD issue was raised late in the cycle for 2.6. The problem is
    > that FBSD's support is "very experimental" as Phillip points out - and
    > OpenBSD doesn't even have them.
    >
    > Due to the lateness of the issue and a finite amount of time I have to
    > work on things, I chose to disable support for this on the various
    > *BSDs until I can cook up a stable patch or have one provided by
    > someone more familiar with the inner workings of Free-BSD. OpenBSD
    > support is a non-starter.
    >
    > Ideally, I would like to get this fixed and put on the 2.6 maint
    > branch ASAP, but I haven't had a chance to circle back to it.
    >
    > Also note Nick's comment in that bug: "Unfortunately, our OpenBSD and
    > FreeBSD buildbots are so unreliable that they don't get much attention
    > when they go red"
    >
    > Stable reliable buildbots and a few more volunteers more familiar with
    > BSDs might be a great and welcome addition to python-dev.
    >
    > As for getting this working - I would love a patch. You are going to
    > want to start with python-trunk and look in setup.py. You are going to
    > want to adjust the flags the package uses:
    >
    >         elif platform in ('freebsd5', 'freebsd6', 'freebsd7', 'freebsd8'):
    >             # FreeBSD's P1003.1b semaphore support is very experimental
    >             # and has many known problems. (as of June 2008)
    >             macros = dict(                  # FreeBSD
    >                 HAVE_SEM_OPEN=0,
    >                 HAVE_SEM_TIMEDWAIT=0,
    >                 HAVE_FD_TRANSFER=1,
    >                 )
    >             libraries = []
    >
    > You will also need to look at: Lib/multiprocessing/synchronize.py to
    > disable the import error - Modules/_multiprocessing/multiprocessing.h
    > will need to be updated for the proper ifdefs for the bsd(s) as well.
    > Finally, the core of the semaphore usage is in
    > Modules/_multiprocessing/semaphore.c
    >
    > I apologize we/I could not get this in for 2.6
    >
    > -jesse


    This is exactly the sort of response I was hoping for. Thanks for the
    additional background on the problem. I'll take a look at the code and
    see if I can figure out a patch. I'll also read up on the buildbot
    issue ( I think I saw a link about that)...I might have a stable co-
    located FreeBSD box that could be used.
    -Alan
    , Oct 22, 2008
    #5
  6. Jesse Noller Guest

    On Wed, Oct 22, 2008 at 10:31 AM, <> wrote:
    > On Oct 22, 8:11 am, "Jesse Noller" <> wrote:
    >> On Tue, Oct 21, 2008 at 6:45 PM, <> wrote:
    >> > It seems that the multiprocessing module in 2.6 is broken for *BSD;
    >> > I've seen issue 3770 regarding this. I'm curious if there are more
    >> > details on this issue since the posts in 3770 were a bit unclear. For
    >> > example, one post claimed that the problem was that sem_open isn't
    >> > implemented in *BSD, but it is available on FreeBSD 7 (I checked). I'd
    >> > be willing to help get this working if someone could point me in the
    >> > right direction.
    >> > --
    >> >http://mail.python.org/mailman/listinfo/python-list

    >>
    >> The BSD issue was raised late in the cycle for 2.6. The problem is
    >> that FBSD's support is "very experimental" as Phillip points out - and
    >> OpenBSD doesn't even have them.
    >>
    >> Due to the lateness of the issue and a finite amount of time I have to
    >> work on things, I chose to disable support for this on the various
    >> *BSDs until I can cook up a stable patch or have one provided by
    >> someone more familiar with the inner workings of Free-BSD. OpenBSD
    >> support is a non-starter.
    >>
    >> Ideally, I would like to get this fixed and put on the 2.6 maint
    >> branch ASAP, but I haven't had a chance to circle back to it.
    >>
    >> Also note Nick's comment in that bug: "Unfortunately, our OpenBSD and
    >> FreeBSD buildbots are so unreliable that they don't get much attention
    >> when they go red"
    >>
    >> Stable reliable buildbots and a few more volunteers more familiar with
    >> BSDs might be a great and welcome addition to python-dev.
    >>
    >> As for getting this working - I would love a patch. You are going to
    >> want to start with python-trunk and look in setup.py. You are going to
    >> want to adjust the flags the package uses:
    >>
    >> elif platform in ('freebsd5', 'freebsd6', 'freebsd7', 'freebsd8'):
    >> # FreeBSD's P1003.1b semaphore support is very experimental
    >> # and has many known problems. (as of June 2008)
    >> macros = dict( # FreeBSD
    >> HAVE_SEM_OPEN=0,
    >> HAVE_SEM_TIMEDWAIT=0,
    >> HAVE_FD_TRANSFER=1,
    >> )
    >> libraries = []
    >>
    >> You will also need to look at: Lib/multiprocessing/synchronize.py to
    >> disable the import error - Modules/_multiprocessing/multiprocessing.h
    >> will need to be updated for the proper ifdefs for the bsd(s) as well.
    >> Finally, the core of the semaphore usage is in
    >> Modules/_multiprocessing/semaphore.c
    >>
    >> I apologize we/I could not get this in for 2.6
    >>
    >> -jesse

    >
    > This is exactly the sort of response I was hoping for. Thanks for the
    > additional background on the problem. I'll take a look at the code and
    > see if I can figure out a patch. I'll also read up on the buildbot
    > issue ( I think I saw a link about that)...I might have a stable co-
    > located FreeBSD box that could be used.
    > -Alan
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >


    I know I really appreciate the additional set of eyes on this, so thanks Alan!
    Jesse Noller, Oct 22, 2008
    #6
  7. On Oct 22, 2008, at 10:11 AM, Jesse Noller wrote:

    > On Tue, Oct 21, 2008 at 6:45 PM, <> wrote:
    >> It seems that the multiprocessing module in 2.6 is broken for *BSD;
    >> I've seen issue 3770 regarding this. I'm curious if there are more
    >> details on this issue since the posts in 3770 were a bit unclear. For
    >> example, one post claimed that the problem was that sem_open isn't
    >> implemented in *BSD, but it is available on FreeBSD 7 (I checked).
    >> I'd
    >> be willing to help get this working if someone could point me in the
    >> right direction.
    >> --
    >> http://mail.python.org/mailman/listinfo/python-list
    >>

    >
    > The BSD issue was raised late in the cycle for 2.6. The problem is
    > that FBSD's support is "very experimental" as Phillip points out - and
    > OpenBSD doesn't even have them.
    >
    > Due to the lateness of the issue and a finite amount of time I have to
    > work on things, I chose to disable support for this on the various
    > *BSDs until I can cook up a stable patch or have one provided by
    > someone more familiar with the inner workings of Free-BSD. OpenBSD
    > support is a non-starter.


    Hi Jesse,
    I wasn't aware of the multiprocessing module. It looks slick! Well done.

    I don't know if you clicked on the link I gave for my posix_ipc
    module, but it looks like we're duplicating effort to some degree. My
    module makes POSIX semaphore & shared memory primitives available to
    Python programs. Obviously, what you've done is much more sophisticated.

    One oversight I noticed the multiprocessing module docs is that a
    semaphore's acquire() method shouldn't have a timeout on OS X as
    sem_timedwait() isn't supported on that platform. (You note OS X's
    lack of support for sem_getvalue() elsewhere.)

    A question - how do you handle the difference in error messages on
    various platforms? For instance, sem_trywait() raises error 35,
    "Resource temporarily unavailable" under OS X but error 11 under
    Ubuntu. Right now I'm just passing these up to the (Python) caller as
    OSErrors. This makes it really hard for the Python programmer to write
    cross-platform code.

    The only solution I can think of (which I haven't coded yet) is to
    compile & run a series of small C programs during setup.py that test
    things like sem_trywait() to see what errors occur, and provide those
    constants to my main .c module so that it can detect those errors
    exactly and wrap them into a specific, custom error for the Python
    caller.

    Any thoughts on this?

    Cheers
    Philip
    Philip Semanchuk, Oct 22, 2008
    #7
  8. Jesse Noller Guest

    On Wed, Oct 22, 2008 at 11:06 AM, Philip Semanchuk <> wrote:
    >> The BSD issue was raised late in the cycle for 2.6. The problem is
    >> that FBSD's support is "very experimental" as Phillip points out - and
    >> OpenBSD doesn't even have them.
    >>
    >> Due to the lateness of the issue and a finite amount of time I have to
    >> work on things, I chose to disable support for this on the various
    >> *BSDs until I can cook up a stable patch or have one provided by
    >> someone more familiar with the inner workings of Free-BSD. OpenBSD
    >> support is a non-starter.

    >
    > Hi Jesse,
    > I wasn't aware of the multiprocessing module. It looks slick! Well done.
    >


    The credit goes to R. Oudkerk, the original author of the pyprocessing
    library - I'm simply a rabid user who managed to wrangle it into
    Python-Core. See: http://www.python.org/dev/peps/pep-0371/

    > I don't know if you clicked on the link I gave for my posix_ipc module, but
    > it looks like we're duplicating effort to some degree. My module makes POSIX
    > semaphore & shared memory primitives available to Python programs.
    > Obviously, what you've done is much more sophisticated.
    >


    I actually saw your stuff cross the 'tubes - it looks darned nice as a
    lower-level interface. What the MP package is meant to be is obviously
    much more high level (and "thread like"), MP goes out of it's way to
    hide the gritty internals of the semaphore management/etc - posix_ipc
    is much more low level than that.

    > One oversight I noticed the multiprocessing module docs is that a
    > semaphore's acquire() method shouldn't have a timeout on OS X as
    > sem_timedwait() isn't supported on that platform. (You note OS X's lack of
    > support for sem_getvalue() elsewhere.)


    Please file a ticket or update http://bugs.python.org/issue4012 so I
    don't loose it, my memory is increasingly lossy. Good catch.

    >
    > A question - how do you handle the difference in error messages on various
    > platforms? For instance, sem_trywait() raises error 35, "Resource
    > temporarily unavailable" under OS X but error 11 under Ubuntu. Right now I'm
    > just passing these up to the (Python) caller as OSErrors. This makes it
    > really hard for the Python programmer to write cross-platform code.
    >


    If you look at the code, we're pretty much raising OSError - it's
    possible we could enhance this in later versions, but given MP is
    supposed to be a cross-platform as possible and protect the user from
    the seedy underbelly of semaphores/pipes/etc - when an OSError does
    occur, it's generally a bug in our code, not the users.

    > The only solution I can think of (which I haven't coded yet) is to compile &
    > run a series of small C programs during setup.py that test things like
    > sem_trywait() to see what errors occur, and provide those constants to my
    > main .c module so that it can detect those errors exactly and wrap them into
    > a specific, custom error for the Python caller.
    >
    > Any thoughts on this?
    >


    That's actually (while feeling hacky) a possibly sensible idea, the
    problem is is that you'd need to maintain documentation to tell users
    the exceptions for their platform.

    -jesse
    Jesse Noller, Oct 22, 2008
    #8
  9. On Oct 22, 2008, at 11:37 AM, Jesse Noller wrote:

    > On Wed, Oct 22, 2008 at 11:06 AM, Philip Semanchuk <
    > > wrote:
    >>> One oversight I noticed the multiprocessing module docs is that a

    >> semaphore's acquire() method shouldn't have a timeout on OS X as
    >> sem_timedwait() isn't supported on that platform. (You note OS X's
    >> lack of
    >> support for sem_getvalue() elsewhere.)

    >
    > Please file a ticket or update http://bugs.python.org/issue4012 so I
    > don't loose it, my memory is increasingly lossy. Good catch.


    I updated issue4012.

    >> A question - how do you handle the difference in error messages on
    >> various
    >> platforms? For instance, sem_trywait() raises error 35, "Resource
    >> temporarily unavailable" under OS X but error 11 under Ubuntu.
    >> Right now I'm
    >> just passing these up to the (Python) caller as OSErrors. This
    >> makes it
    >> really hard for the Python programmer to write cross-platform code.
    >>

    >
    > If you look at the code, we're pretty much raising OSError - it's
    > possible we could enhance this in later versions, but given MP is
    > supposed to be a cross-platform as possible and protect the user from
    > the seedy underbelly of semaphores/pipes/etc - when an OSError does
    > occur, it's generally a bug in our code, not the users.


    Gotcha. I had a look at the code, and you're testing for errno ==
    EAGAIN when sem_trywait() fails. This is correct for OS X and Ubuntu
    (the platforms I mentioned above) and probably most other Unices. I
    just don't have confidence that it will be true across all platforms,
    esp. ones to which I don't have access like AIX and big-iron systems
    that support POSIX. Maybe I am just taking defensive programming too
    far.

    >> The only solution I can think of (which I haven't coded yet) is to
    >> compile &
    >> run a series of small C programs during setup.py that test things
    >> like
    >> sem_trywait() to see what errors occur, and provide those constants
    >> to my
    >> main .c module so that it can detect those errors exactly and wrap
    >> them into
    >> a specific, custom error for the Python caller.
    >>
    >> Any thoughts on this?
    >>

    >
    > That's actually (while feeling hacky) a possibly sensible idea, the
    > problem is is that you'd need to maintain documentation to tell users
    > the exceptions for their platform.


    If I pass all errors up as OSError, yes, that will be true. But that's
    actually the situation I was trying to avoid.

    By generating the error at install time, I can see exactly what
    platform X returns in that situation (e.g. 11). Then I can create a
    #define something like this:
    #define ERRNO_WHEN_CALLING_SEM_TRYWAIT_ON_A_LOCKED_SEMAPHORE 11

    My main .c module can then test for errno ==
    ERRNO_WHEN_CALLING_SEM_TRYWAIT_ON_A_LOCKED_SEMAPHORE and wrap that in
    a custom error, like posix_ipc.SemaphoreBusyError.


    Again, maybe I'm just taking defensive programming too far. I've been
    bitten by other cross-platform inconsistencies at the C API level and
    I'm trying to get the jump on them here.

    Cheers
    Philip
    Philip Semanchuk, Oct 22, 2008
    #9
  10. MRAB Guest

    On Oct 22, 5:14 pm, Philip Semanchuk <> wrote:
    > On Oct 22, 2008, at 11:37 AM, Jesse Noller wrote:
    >
    > > On Wed, Oct 22, 2008 at 11:06 AM, Philip Semanchuk <
    > > > wrote:
    > >>> One oversight I noticed the multiprocessing module docs is that a
    > >> semaphore's acquire() method shouldn't have a timeout on OS X as
    > >> sem_timedwait() isn't supported on that platform. (You note OS X's  
    > >> lack of
    > >> support for sem_getvalue() elsewhere.)

    >
    > > Please file a ticket or updatehttp://bugs.python.org/issue4012so I
    > > don't loose it, my memory is increasingly lossy. Good catch.

    >
    > I updated issue4012.
    >
    > >> A question - how do you handle the difference in error messages on  
    > >> various
    > >> platforms? For instance, sem_trywait() raises error 35, "Resource
    > >> temporarily unavailable" under OS X but error 11 under Ubuntu.  
    > >> Right now I'm
    > >> just passing these up to the (Python) caller as OSErrors. This  
    > >> makes it
    > >> really hard for the Python programmer to write cross-platform code.

    >
    > > If you look at the code, we're pretty much raising OSError - it's
    > > possible we could enhance this in later versions, but given MP is
    > > supposed to be a cross-platform as possible and protect the user from
    > > the seedy underbelly of semaphores/pipes/etc - when an OSError does
    > > occur, it's generally a bug in our code, not the users.

    >
    > Gotcha. I had a look at the code, and you're testing for errno ==  
    > EAGAIN when sem_trywait() fails. This is correct for OS X and Ubuntu  
    > (the platforms I mentioned above) and probably most other Unices. I  
    > just don't have confidence that it will be true across all platforms,  
    > esp. ones to which I don't have access like AIX and big-iron systems  
    > that support POSIX. Maybe I am just taking defensive programming too  
    > far.
    >
    >
    >
    > >> The only solution I can think of (which I haven't coded yet) is to  
    > >> compile &
    > >> run a series of small C programs during setup.py that test things  
    > >> like
    > >> sem_trywait() to see what errors occur, and provide those constants  
    > >> to my
    > >> main .c module so that it can detect those errors exactly and wrap  
    > >> them into
    > >> a specific, custom error for the Python caller.

    >
    > >> Any thoughts on this?

    >
    > > That's actually (while feeling hacky) a possibly sensible idea, the
    > > problem is is that you'd need to maintain documentation to tell users
    > > the exceptions for their platform.

    >
    > If I pass all errors up as OSError, yes, that will be true. But that's  
    > actually the situation I was trying to avoid.
    >
    > By generating the error at install time, I can see exactly what  
    > platform X returns in that situation (e.g. 11). Then I can create a  
    > #define something like this:
    > #define ERRNO_WHEN_CALLING_SEM_TRYWAIT_ON_A_LOCKED_SEMAPHORE  11
    >
    > My main .c module can then test for errno ==  
    > ERRNO_WHEN_CALLING_SEM_TRYWAIT_ON_A_LOCKED_SEMAPHORE and wrap that in  
    > a custom error, like posix_ipc.SemaphoreBusyError.
    >
    > Again, maybe I'm just taking defensive programming too far. I've been  
    > bitten by other cross-platform inconsistencies at the C API level and  
    > I'm trying to get the jump on them here.
    >

    Re the error codes, the OS-specific code can be mapped to an OS-
    agnostic(?) code. On Windows there's a Windows-specific subclass of
    OSError, WindowsError, containing the Windows error code winerror as
    well as errno which is inherited from OSError.
    MRAB, Oct 23, 2008
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Chameleon

    BSD ImageResize library in J2ME

    Chameleon, Mar 25, 2006, in forum: Java
    Replies:
    3
    Views:
    734
    Roedy Green
    Mar 25, 2006
  2. Elf M. Sternberg

    Python signal delivery under BSD 4.4

    Elf M. Sternberg, Aug 12, 2003, in forum: Python
    Replies:
    8
    Views:
    347
    Donn Cave
    Aug 16, 2003
  3. j_mckitrick
    Replies:
    0
    Views:
    315
    j_mckitrick
    May 14, 2004
  4. Replies:
    7
    Views:
    312
    Rod Pemberton
    Oct 8, 2006
  5. PurpleServerMonkey

    Distributing Python Apps on Linux\BSD

    PurpleServerMonkey, Mar 21, 2008, in forum: Python
    Replies:
    3
    Views:
    437
    Diez B. Roggisch
    Mar 22, 2008
Loading...

Share This Page