Simple distributed example for learning purposes?

[Tim Golden]

I'm trying to work up a programming course using Python,
aimed at secondary school students [*] here in London. One
of my aims is to have a series of compact but functional
examples, each demonstrating a particular field in which
Python (and programming) can be useful.

I'm trying to come up with something which will illustrate
the usefulness of a distributed processing model. Since I
may not be using the term "distributed" exactly, my
criteria are:

* It should be clear that the application produces results
sooner when run via multiple cooperating computers
than when run on one.

* The problem being solved should be broadly comprehensible by
the students. This rules out some abstruse mathematical
calculation which would benefit from multiple processors but
which will fail to engage their interest.

* I don't mind using / installing some library as a black box
so long as the other criteria are met. This is at least partly
because I want the code to be compact and I'm quite happy to
point to a library call and say "this does the hard work".


I'm not asking anyone to write code: what I'm after is some sort
of problem space which will meet my criteria. Part of the issue
is that computers are just so fast these days that almost anything
I can come up with can be managed so quickly on one laptop that
the overhead of distribution (or of the IO) will dwarf the benefits
of distributing.

The kind of things I've considered briefly include: speech recognition;
image analysis; large scale indexing (effectively: building a search
engine). At present I'm looking at that last one, not least because I
have little knowledge of the other domains so there's a overhead for
me in setting the example up.

Any suggestions, either from your own experience of something similar,
or from sheer inspiration, will be gratefully received.

Thanks

TJG

[*] roughly, ages 14-18

 

[Nobody]

I'm trying to come up with something which will illustrate
the usefulness of a distributed processing model. Since I
may not be using the term "distributed" exactly, my
criteria are:

* It should be clear that the application produces results
sooner when run via multiple cooperating computers
than when run on one.

* The problem being solved should be broadly comprehensible by
the students. This rules out some abstruse mathematical
calculation which would benefit from multiple processors but
which will fail to engage their interest.

Rendering fractals? Easy to parallelise, easy to code, easy to make slow
enough that the speed-up is noticeable.

 

[Tim Golden]

Rendering fractals? Easy to parallelise, easy to code, easy to make slow
enough that the speed-up is noticeable.

Thanks. I'll add that to the list of possibilities.

TJG

 

[Aahz]

I'm trying to come up with something which will illustrate the
usefulness of a distributed processing model. Since I may not be using
the term "distributed" exactly, my criteria are:

Distributed spider with firewall that limits network I/O per computer
(the firewall is useful but not essential).

 

[Tim Golden]

Distributed spider with firewall that limits network I/O per computer
(the firewall is useful but not essential).

Thanks; I'll add that to the list.

TJG

 

[Mike Driscoll]

I'm trying to work up a programming course using Python,
aimed at secondary school students [*] here in London. One
of my aims is to have a series of compact but functional
examples, each demonstrating a particular field in which
Python (and programming) can be useful.

I'm trying to come up with something which will illustrate
the usefulness of a distributed processing model. Since I
may not be using the term "distributed" exactly, my
criteria are:

* It should be clear that the application produces results
sooner when run via multiple cooperating computers
than when run on one.

* The problem being solved should be broadly comprehensible by
the students. This rules out some abstruse mathematical
calculation which would benefit from multiple processors but
which will fail to engage their interest.

* I don't mind using / installing some library as a black box
so long as the other criteria are met. This is at least partly
because I want the code to be compact and I'm quite happy to
point to a library call and say "this does the hard work".

I'm not asking anyone to write code: what I'm after is some sort
of problem space which will meet my criteria. Part of the issue
is that computers are just so fast these days that almost anything
I can come up with can be managed so quickly on one laptop that
the overhead of distribution (or of the IO) will dwarf the benefits
of distributing.

The kind of things I've considered briefly include: speech recognition;
image analysis; large scale indexing (effectively: building a search
engine). At present I'm looking at that last one, not least because I
have little knowledge of the other domains so there's a overhead for
me in setting the example up.

Any suggestions, either from your own experience of something similar,
or from sheer inspiration, will be gratefully received.

Thanks

TJG

[*] roughly, ages 14-18

I think distributed transcoding of hi-def videos would be cool, but I
haven't found much with Google. Still, you might find this useful for
your project:

http://pypi.python.org/pypi/AsynCluster/0.3

For reasons I don't understand, the home page listed in that link is
blocked here.

-------------------
Mike Driscoll

Blog: http://blog.pythonlibrary.org

PyCon 2010 Atlanta Feb 19-21 http://us.pycon.org/

 

[Tim Golden]

I think distributed transcoding of hi-def videos would be cool, but I
haven't found much with Google. Still, you might find this useful for
your project:

http://pypi.python.org/pypi/AsynCluster/0.3

Thanks. I suspect that that would be overkill for what I need
but I'm always interested in whatever's out there so thanks
for pointing me that way.

For reasons I don't understand, the home page listed in that link is
blocked here.

Interestingly, it just fails to resolve over here. I'll try Googling
for it later. Fortunately, altho' we do have filtering here at work
via transparent proxy, our tech support guys are quite sane: if you
let them know a page is blocked and they can see it's perfectly ok
they'll just unlock it without hassle.

TJG

 

[r0g]

Thanks. I'll add that to the list of possibilities.

TJG

Distributed password cracking? It's really simple yet potentially
engaging and could be a good segway into teaching them about
computability and security. All you need is a bunch of password hashes
and a wordlist.

Roger.

 

[Tim Golden]

Distributed password cracking? It's really simple yet potentially
engaging and could be a good segway into teaching them about
computability and security. All you need is a bunch of password hashes
and a wordlist.

At the moment, that's pretty much what my example is. I'm hanging
on to other people's examples in the event that I get the chance
to make use of them. And, yes, I intend to use it as an opportunity
to talk about security issues.

Thanks for confirming that the idea is at least credible.

TJG

(BTW, is segway a conventional way of spelling that where you come
from? I've not seen it written that way before.)

 

[r0g]

At the moment, that's pretty much what my example is. I'm hanging
on to other people's examples in the event that I get the chance
to make use of them. And, yes, I intend to use it as an opportunity
to talk about security issues.

Thanks for confirming that the idea is at least credible.

TJG

(BTW, is segway a conventional way of spelling that where you come
from? I've not seen it written that way before.)

No, it turns out not. I've never encountered the written form of it
before (until I googled it just now to check!) so I had assumed it was
spelled as per the freaky gyroscopically stabilized scooter things.

Good luck with the course.

Cheers,

Roger.