Summary of threading for experienced non-Python programmers?

[skip]

I'm having trouble explaining the benefits and tradeoffs of threads to my
coworkers and countering their misconceptions about Python's threading model
and facilities. They all come from C++ and are used to thinking of
multithreading as a way to harness multiple CPU cores for compute-bound
processing. I also encountered, "Python doesn't really do threads" today.
*sigh*

I don't need pointers to the relevant documentation sections. A bit of
googling didn't turn up much about threading beyond tutorial introductions.
I'm looking for something which will explain the rationale for using threads
in Python to someone who is experienced with multithreading in other
languages (like C/C++). Maybe a compare-and-contrast sort of document?

Thanks,

Skip

 

[Aahz]

I'm having trouble explaining the benefits and tradeoffs of threads to my
coworkers and countering their misconceptions about Python's threading model
and facilities. They all come from C++ and are used to thinking of
multithreading as a way to harness multiple CPU cores for compute-bound
processing. I also encountered, "Python doesn't really do threads" today.
*sigh*

I don't need pointers to the relevant documentation sections. A bit of
googling didn't turn up much about threading beyond tutorial introductions.
I'm looking for something which will explain the rationale for using threads
in Python to someone who is experienced with multithreading in other
languages (like C/C++). Maybe a compare-and-contrast sort of document?

Maybe this will help?

http://www.pyzine.com/Issue001/Section_Articles/article_ThreadingGlobalInterpreter.html

 

[Hrvoje Niksic]

I'm having trouble explaining the benefits and tradeoffs of threads
to my coworkers and countering their misconceptions about Python's
threading model and facilities. They all come from C++ and are used
to thinking of multithreading as a way to harness multiple CPU cores
for compute-bound processing. I also encountered, "Python doesn't
really do threads" today. *sigh*

Compute-bound processing pretty much excludes a Python-only solution,
so any performance considerations should take into account C
extensions. Extensions are free to allow other threads to run during
CPU-extensive portions of their work, and many of them in fact do so.
As long as the extensions are correctly written, you can write your
"glue code" in Python and harness the multiple cores using threads,
exactly as expected by a C++ programmer.

The other use for threads is the case when dealing with blocking APIs
that don't support polling. These typically include database APIs and
some network APIs (such as the portable host name lookup), but also
basic file input/output, if you take into account network file
systems. (In theory, file input/output should also be available as
asynchronous code, but async IO is low-level and not available in
Python.) While threads shouldn't be considered a replacement for
event-driven programming, they are certainly useful in such
situations.

 

[Diez B. Roggisch]

systems. (In theory, file input/output should also be available as

asynchronous code, but async IO is low-level and not available in
Python.) While threads shouldn't be considered a replacement for

I suggest you tell that the twisted-guys. And the ones from the built-in
asyncore-module.

They will be surprised to hear that their years worth of working code
will evaporate in a rosa cloud.

Diez

 

[sturlamolden]

I'm having trouble explaining the benefits and tradeoffs of threads to my
coworkers and countering their misconceptions about Python's threading model
and facilities.  

Python's threading module is modelled on Java's thread model. There
are some minor differences, though. Whereas Python has special lock
objects, Java can lock (synchronize) on any object.

They all come from C++ and are used to thinking of
multithreading as a way to harness multiple CPU cores for compute-bound
processing.  I also encountered, "Python doesn't really do threads" today.
*sigh*

You can't use threads for that in CPython, due to the GIL (global
interpreter lock). The GIL resembles the BKL in earlier versions of
the Linux kernel. Due to the GIL, multiple threads cannot be
simultaneously in the Python interpreter. This e.g. means that I
cannot implement a parallel QuickSort in pure Python and get
performance gain from multiple CPUs.

Although common misbeliefs, this DOES NOT mean:

* Python threads are not useful.
* Python programs cannot utilize multiple CPUs or multi-core CPUs.

Here is the explanations:

The GIL can be released by extension modules, which are native
libraries of compiled C, C++ or Fortran. This is the key to the
usefulness of Python threads. For example:

* Python file and socket objects are extension modules that release
the GIL. The GIL is released when a thread is waiting for i/o to
complete. This e.g. allows you to write multi-threaded server apps in
Python.

* NumPy is an extension library that releases the GIL before
commencing on a time-consuming CPU-bound computations. If you have N
CPUs, NumPy allows you to do N FFTs or SVDs in parallel.

* ctypes is an extension module that allows Python code to call
DLLs. ctypes releases the GIL before the foregin call on cdecl
functions (but not stdcall functions!), allowing you to call many
cdecl DLL functions in parallel.

* Extension libraries that spawns multiple threads will utilize
mutiple CPUs, even if the GIL are not released.

There is also other reasons why threads are useful, that does not
depend on extension modules releasing the GIL. One example:
Multithreading is the key to responsive user interfaces, as only one
thread should process events. An event-handler should spawn a thread
before commencing on a time-consuming task.

IronPython and Jython are implemented without a GIL. (They run on
the .NET and Java VMs, respectively.)

Finally, remeber that pure Python often runs 200 times slower than
pure C on algoritmic code! If you need to do lengthy computational
tasks, a pure Python may not be what you want. With two dual-core
CPUs, nearly perfect load scheduling, a no-GIL implementation of
Python, a pure Python would still be more than 50 times solwer than a
single-threaded C solution. Hence, you would gain a lot more from
profiling, identifying the worst bottlenecks, and translating those
parts to C.

 

[Donn Cave]

systems. (In theory, file input/output should also be available as
asynchronous code, but async IO is low-level and not available in
Python.) While threads shouldn't be considered a replacement for

I suggest you tell that the twisted-guys. And the ones from the built-in
asyncore-module.

They will be surprised to hear that their years worth of working code
will evaporate in a rosa cloud.

Diez[/QUOTE]

I appreciate the droll sense of humor, but do you mean to
assert that asyncore.py supports asynchronous disk file I/O?

What that means to me is, you queue a disk read, and there's
an event flag or something that you can wait for before you
come back to find the data in your buffer. (That's how I
remember it from the old days, when it mattered a little,
though not enough that I ever remember actually doing it,
and 20 years later I guess the incentive is even less.)

I see MacOS supports an F_RDADVISE that might give you a head
start on reading into the system buffer, but that's 3rd rate
asynchrony because there's no way to know when the data is
ready, and 3rd rate I/O because afterwards you still have the
copying to do. I don't see even this much in asyncore.py, but
I just gave it a glance.

thanks,
Donn Cave, (e-mail address removed)

 

[Hrvoje Niksic]

I suggest you tell that the twisted-guys. And the ones from the
built-in asyncore-module.

Note that I said "*file* input/output". Twisted and asyncore are
about asynchronous socket programming that polls over nonblocking file
descriptors such as found in socket programming, not about wrapping
aio(3) and the equivalent Windows APIs.

 

[Paul Rubin]

Note that I said "*file* input/output". Twisted and asyncore are
about asynchronous socket programming that polls over nonblocking file
descriptors such as found in socket programming, not about wrapping
aio(3) and the equivalent Windows APIs.

aio is also used for sockets, while twisted and asyncore use select or
something similar. That is asynchronous but in a different sense of
the word. See also: http://www.kegel.com/c10k.html

 

[Paul Rubin]

aio is also used for sockets, while twisted and asyncore use select or
something similar. That is asynchronous but in a different sense of
the word. See also: http://www.kegel.com/c10k.html

Hmm that actually says that in Linux 2.6.0#-test2 sockets weren't
supported. I can't tell the situation from the man page on the system
I'm using right now. I may have misremembered but I thought I talked
with someone a while back who was using aio in a high concurrency VOIP
system and that it was beating other approaches. I don't know what
kernels were involved etc. Anyway using this stuff in Python would be
overkill.

 

[Diez B. Roggisch]

Note that I said "*file* input/output". Twisted and asyncore are
about asynchronous socket programming that polls over nonblocking file
descriptors such as found in socket programming, not about wrapping
aio(3) and the equivalent Windows APIs.

I admit glossing over the file-input - however this is available on
posix-systems using the select-module. So if I were in nitpicking-mood,
your assertion still would be false - albeit having to consider
platoform dependencies certainly is unfortunate and could be considered
"missing".

I'm pretty sure though that tiwsted & asynchore don't poll, but instead
use the select-module. Which at least for unix (and AFAIK for Windows as
well) is asynchronous - you get notified if data arrives or has been
transmitted, and otherwise your process sleeps.


Diez

 

[Diez B. Roggisch]

I appreciate the droll sense of humor, but do you mean to

assert that asyncore.py supports asynchronous disk file I/O?

As I said in an answer to the OP, I somewhat glossed over the "file" and
just read IO. And under Posix, python *does* support asynchronous IO
using the select-module. If you can faciliate that using the asynchore
module I can't say, but the question was if python as whole supported
async IO out of the box - asyncore & twisted I mention as references of
implementations using that.

What that means to me is, you queue a disk read, and there's
an event flag or something that you can wait for before you
come back to find the data in your buffer. (That's how I
remember it from the old days, when it mattered a little,
though not enough that I ever remember actually doing it,
and 20 years later I guess the incentive is even less.)

Which is exactly what select allows you to do. You pass a set of
file-descriptors and are notified if data arrives or has been
successfully transmitted.

Diez

 

[Diez B. Roggisch]

aio is also used for sockets, while twisted and asyncore use select or
something similar. That is asynchronous but in a different sense of
the word. See also: http://www.kegel.com/c10k.html

In which sense is that different? AFAIK select lets you avoid polling
and provides notifications (possibly with timeouts) for IO-events. So
where exactly is the difference? I read TFA, and it does mention that
select/poll have potential for optimization, but not something that
disqualified them (or rather select) as being not async.

Diez

 

[Paul Rubin]

As I said in an answer to the OP, I somewhat glossed over the "file"
and just read IO. And under Posix, python *does* support asynchronous
IO using the select-module.

Select does nonblocking i/o but I think Donn is interpreting
"asynchronous i/o" as having a narrower meaning. See:

http://www.ibm.com/developerworks/linux/library/l-async/

for some info.

 

[Paul Rubin]

In which sense is that different? AFAIK select lets you avoid polling
and provides notifications (possibly with timeouts) for IO-events. So
where exactly is the difference? I read TFA, and it does mention that
select/poll have potential for optimization, but not something that
disqualified them (or rather select) as being not async.

Select blocks until the data is ready, while with AIO the i/o happens
completely in the background and your process gets an interrupt when
the i/o completes. Also, with select based i/o, usually the kernel
reads data from the external device into a system buffer, and then you
do a read system call that copies the data from the system buffer to
your buffer. AIO can be set up so the i/o happens directly into your
buffer, avoiding the extra copying. You can also initiate a bunch of
different i/o events with a single system call, avoiding some context
switches.

http://www.ibm.com/developerworks/linux/library/l-async/

describes select as "asynchronous, blocking" as opposed to AIO which
is asynchronous and nonblocking. Other descriptions I've seen reserve
"asynchronous i/o" for AIO-like schemes. It's just a terminological
thing.

The PDP-10 operating systems even let you define your own page fault
handlers, so you could do i/o with something like mmap and not get
delayed in the cases where the page you wanted wasn't in memory. I
guess that's even more asynchronous than AIO.

 

[Hrvoje Niksic]

I admit glossing over the file-input - however this is available on
posix-systems using the select-module.

I believe you stil misunderstand. The select module doesn't provide
an inteface to aio(3). It provides an interface to select() and
poll() system calls, which don't provide asynchronous access to
regular files.

So if I were in nitpicking-mood, your assertion still would be false

I invite constructive nitpicking, but you are completely missing the
point. You are confusing aio(3) with select and poll.

I'm pretty sure though that tiwsted & asynchore don't poll, but
instead use the select-module. Which at least for unix (and AFAIK
for Windows as well) is asynchronous - you get notified if data
arrives or has been transmitted, and otherwise your process sleeps.

Unfortunately, this is not the case for files at all, even on Unix.
(On Windows, select doesn't work on files at all, it only accepts
sockets.)

 

[Hrvoje Niksic]

I believe you stil misunderstand. The select module doesn't provide
an inteface to aio(3). It provides an interface to select() and
poll() system calls, which don't provide asynchronous access to
regular files.

It occurred to me that I didn't provide an example of what I mean by
select not providing asynchronous access to regular files. Using this
code:
3

# /mnt/custom is an SMB or NFS mount. At this point, kill -STOP the
# SMB server.

select.select([fd], [], [], 10) # test whether fd is readable ([3], [], []) # exits immediately, assume
os.read(3, 1024)

.... hangs until smb server continues ...

The thing is, select() *always* returns immediately on regular files,
even if they are in fact not readable or writable. In this case,
select() claimed the file descriptor to be readable when in fact it
wasn't. The same is the case with poll, but not with aio.

In most cases this isn't a problem, but it does mean that an
application that reads from a network-share-located file in a
select/poll-driven event loop will stall until the file server
responds. Threads, on the other hand, don't have this problem. A
program that reads the file in a separate thread will not block even
if the file is on a temporarily non-responding NFS server.

 

[hdante]

I'm having trouble explaining the benefits and tradeoffs of threads to my
coworkers and countering their misconceptions about Python's threading model
and facilities. They all come from C++ and are used to thinking of
multithreading as a way to harness multiple CPU cores for compute-bound
processing. I also encountered, "Python doesn't really do threads" today.
*sigh*

I don't need pointers to the relevant documentation sections. A bit of
googling didn't turn up much about threading beyond tutorial introductions.
I'm looking for something which will explain the rationale for using threads
in Python to someone who is experienced with multithreading in other
languages (like C/C++). Maybe a compare-and-contrast sort of document?

Thanks,

Skip

I think you are having trouble because threads suck. They are the
worst way of dealing with concurrency that exists (unfortunately they
are the fastest also) .

Threads are bad because, by construction, code may be subject to non-
trivial hazards. There are other ways of dealing with concurrency that
doesn't have this problem, like event-driven programming. I think
threads are so bad, that when I left my job at a company that
developed for PDAs, I told my team: "whenever you have an argument
about how to make multitasking, you may count my vote against using
threads".

If you really need to use threads, then use design patterns that make
threads behave like message passing systems (I believe there were the
"monitor model" or the "actor model" or something).

If you need threads because you need speed, then use a recent JIT-
compiled language, like Java or C#. In general, you may use C or C++
also, but there are a few obscure problems with them because they
weren't defined with threads in mind.

If you just would like to use "multitasking" as a programming
paradigm, try Twisted, or switch to stackless python (then write an
blog about your findings).

 

[Donn Cave]

Select blocks until the data is ready, while with AIO the i/o happens
completely in the background and your process gets an interrupt when
the i/o completes. Also, with select based i/o, usually the kernel
reads data from the external device into a system buffer, and then you
do a read system call that copies the data from the system buffer to
your buffer. AIO can be set up so the i/o happens directly into your
buffer, avoiding the extra copying. You can also initiate a bunch of
different i/o events with a single system call, avoiding some context
switches.

http://www.ibm.com/developerworks/linux/library/l-async/

describes select as "asynchronous, blocking" as opposed to AIO which
is asynchronous and nonblocking. Other descriptions I've seen reserve
"asynchronous i/o" for AIO-like schemes. It's just a terminological
thing.

kqueue(2) on MacOS X mentions an EVFILT_AIO option. It isn't
supported on that platform, but maybe that's a vestige of some
other platform that does support "asynchronous, blocking" with
aio -- as VAX/VMS did (and presumably still does), with event
flags.

Donn Cave, (e-mail address removed)

 

[John Nagle]

Unfortunately, this is not the case for files at all, even on Unix.
(On Windows, select doesn't work on files at all, it only accepts
sockets.)

"select" doesn't work on Windows pipes, either. I had to go to
a multithreaded program to work around that.

John Nagle

 

[Diez B. Roggisch]

I believe you stil misunderstand. The select module doesn't provide

an inteface to aio(3). It provides an interface to select() and
poll() system calls, which don't provide asynchronous access to
regular files.

I never claimed it provided access to aio. In the thread with Paul
Rubin, it was clarified that there is the distinction made between
blocking and non-blocknig async calls. Which you didn't mention.

Don't get me wrong: I appreciate the mentioning of aio (didn't know
about it beforehand) and don't dispute it's superiority. But that
doesn't change that one is not forced to use threading in python (which
can become very expensive) to deal with mulitple I/O streams asynchronously.

I invite constructive nitpicking, but you are completely missing the
point. You are confusing aio(3) with select and poll.

I didn't confuse anything. You didn't _mention_ aio, and I didn't even
know it...

Unfortunately, this is not the case for files at all, even on Unix.
(On Windows, select doesn't work on files at all, it only accepts
sockets.)

AFAIK select is used in cases where e.g. several processes are piped
together to prevent blocking. Your usecase with a deliberately distorted
file-server is sure relevant for certain usecases - but I don't think it
qualifies as "not at all, even on Unix".

To reiterate what my post was actually after: I understood it as if you
didn't acknowledge the asynchronous capabilities of python which are
used by widely known & successfully adopted built-in modules or
3rd-party-extensions - without resorting to C, and without forcing
threads upon the users.

It did *not* say that it supports every existing, more powerful and
generally better asynchronous mechanism supported by any OS out there.
Even though it would certainly be nice if it did


Diez