MultiThreading and 'fork'

[Naveen Reddy]

Hello all,
From my Perl program I can spawn a parent and a child process using
the 'fork'.
Is this synonymous to 'multi-threading' and are the 'threads' same as
'processes'?

I'm trying to understand why theres a module for threads, as in, 'use
threads', when fork lets you accomplish the mutiltasking?

Thanks for your time,
Reddy

 

[Ben Morrow]

Hello all,
From my Perl program I can spawn a parent and a child process using
the 'fork'.
Is this synonymous to 'multi-threading' and are the 'threads' same as
'processes'?

No. Have you read perlthrtut?

I'm trying to understand why theres a module for threads, as in, 'use
threads', when fork lets you accomplish the mutiltasking?

Threads are lighter than processes (creating many threads consumes
fewer resources than creating many processes), and it is possible to
share variables among threads. In theory all data is shared, but
perl's ithreads copy all the perl data structures when you create a
new thread, except those that are marked shared.

I would say that there is a good case for considering ithreads to be
pretty much redundant with fork() and threads::shared with Sys::Mmap
or some such, but those concepts are not portable outside unix whereas
ithreads are.

Ben

 

[Chris Mattern]

Hello all,
From my Perl program I can spawn a parent and a child process using
the 'fork'.

Well, you can spawn a child process. The parent was there to begin
with.

Is this synonymous to 'multi-threading' and are the 'threads' same as
'processes'?

No. Threads share the same address space.

I'm trying to understand why theres a module for threads, as in, 'use
threads', when fork lets you accomplish the mutiltasking?

Because fork() has much higher overhead and processes have more
difficulty passing information back and forth.

Chris Mattern

 

[Walter Roberson]

:Naveen Reddy wrote:
:> I'm trying to understand why theres a module for threads, as in, 'use
:> threads', when fork lets you accomplish the mutiltasking?

:Because fork() has much higher overhead and processes have more
:difficulty passing information back and forth.

fork() on modern Unix systems does not necessarily
have more overhead than perl 5.8.2 ithreads .

Many modern unix servers have hardware-supported "copy-on-write"
semantics -- all they have to to is sweep through the existing process
page descriptor table, set the COW bit, and copy the resulting page
descriptor table into the new process.

The current implimentation (5.8.2) of ithreads involves process-level
copying of all existing perl variables that are not marked as shared.
If one has much data at all, that procedure is going to take longer
than the process creation overhead.

In a multi-threaded process I was recently working on, it took
about (roughly) 2 seconds per ithread spawned, whereas fork()'s
take a fraction of a second each.

 

[Ilya Zakharevich]

[A complimentary Cc of this posting was sent to
Chris Mattern

Because fork() has much higher overhead and processes have more
difficulty passing information back and forth.

Starting from v5.8, perl (by default) does not support threads any
more. v5.9 has no thread support at all.

What v5.8 supports is "emulation of fork() on Win32 systems". This
emulation layer was hacked to implement creation of a "thread +
cloned-interpreter" pair. Creation of such a pair (via the
fork()-emulation layer!) is (in my measurements) 2 orders of magnitude
slower that creation of a new child process.

Hope this helps,
Ilya

 

[Walter Roberson]

:Starting from v5.8, perl (by default) does not support threads any
:more. v5.9 has no thread support at all.

Perhaps that needs to be qualified with "ActiveState"? Or are
you referring to 5005 threads as compared to ithreads ??

 

[Ben Morrow]

:Starting from v5.8, perl (by default) does not support threads any
:more. v5.9 has no thread support at all.

Perhaps that needs to be qualified with "ActiveState"? Or are
you referring to 5005 threads as compared to ithreads ??

Of course he is. Given Ilya's previous posts of the evils of fork(),
it is unsurprising he dislikes ithreads. I have to admit, it seems
rather daft to attempt to reimplement in perl something which the OS
does perfectly well; except on those OSen which don't. I have serious
doubts that ithread creation will ever become less expensive than
process creation on a decent unix, simply because they are essentially
the same operation.

Ben

 

[Ilya Zakharevich]

[A complimentary Cc of this posting was sent to
Ben Morrow

Of course he is. Given Ilya's previous posts of the evils of fork(),
it is unsurprising he dislikes ithreads.

Myself, I do not see any connection between these two acts of madness.
Introducing fork() *was* an act of madness in '70; however, today,
when (on Unix) most problems associated with this madness are either
already resolved, or at least well understood, most it deserves is
just a shrug. [Of course, outside of Unix it provides enormous
difficulties to developers, but for the purpose of simiplicity, I
would like to keep the discussion of ithreads focused on one
architecture only.]

The fact that "fork() emulation on Win32" is related to fork() has
little to do with the problem of ithreads. Consider the code to
perform emulation as a black box. It is just a subsystem of Perl
which does "something".

Now somebody got a bright idea: "running this subsystem on Unix will
give us an interpreter running in a new thread; code reuse is a good
thing; let us just use this subsystem to create new threads."

"BTW, a lot of people had problems with running the old model of
threads. We care; let us remove the support for the old model, so
people do not have these problems."

Now we got a system which uses a microscope to pull nails. The
microscope does not care, but the finishing is all ruined.

I have to admit, it seems rather daft to attempt to reimplement in
perl something which the OS does perfectly well; except on those
OSen which don't. I have serious doubts that ithread creation will
ever become less expensive than process creation on a decent unix,
simply because they are essentially the same operation.

Are you joking? Did you read the code for fork() emulation? Did you
run any benchmarks? The code is 2 orders of magnitude slower than
process creation.

Try to create 20 ithreads/sec; it may be possible on very recent
hardware, but I think it is close to the current maximum...

Hope this helps,
Ilya

 

[Ben Morrow]

[A complimentary Cc of this posting was sent to
Ben Morrow

I have to admit, it seems rather daft to attempt to reimplement in
perl something which the OS does perfectly well; except on those
OSen which don't. I have serious doubts that ithread creation will
ever become less expensive than process creation on a decent unix,
simply because they are essentially the same operation.

Are you joking? Did you read the code for fork() emulation? Did you
run any benchmarks? The code is 2 orders of magnitude slower than
process creation.

I think you must have misread me. I said that not only is ithread
creation *currently* slower than process creation, but also there are
good reasons for thinking that on a unix system it will *always* be
slower, no matter how much p5p optimize it; namely, that it is
emulating in userland something already implemented perfectly well in
the kernel.

Ben

 

[Ilya Zakharevich]

[A complimentary Cc of this posting was sent to
Ben Morrow

I think you must have misread me. I said that not only is ithread
creation *currently* slower than process creation, but also there are
good reasons for thinking that on a unix system it will *always* be
slower, no matter how much p5p optimize it; namely, that it is
emulating in userland something already implemented perfectly well in
the kernel.

Still: I do not consider your disclaimer sufficient. ;-) The overhead of
creation of a *thread* should/could be orders of magnitude smaller
than the overhead of creation of a new process.

However, the overhead of creation of a *new interpreter* is very large.
And the overhead of *cloning* an interpreter (as opposed to creating a
new one) is jet orders of magnitude larger.

Hope this helps,
Ilya