Advice needed on parallel processing in python

Discussion in 'Python' started by Glazner, Mar 24, 2010.

  1. Glazner

    Glazner Guest

    Hi!

    I need to replace an app that does number crunching over a local
    network.
    it have about 50 computers as slaves
    each computer needs to run COM that will do the "job"
    right now the system uses MFC threads and DCOM to distribute the load.

    as i said, Now i'm trying to replace this system with python.
    I already use win32all and read about Pareller Python and Pyro.

    a requirement is that messages can be sent to the remote worker.

    If anyone can share experience I'll love to hear


    Many Thanks,
    Yoav Glazner
     
    Glazner, Mar 24, 2010
    #1
    1. Advertising

  2. Glazner

    Jon Clements Guest

    On 24 Mar, 15:27, Glazner <> wrote:
    > Hi!
    >
    > I need to replace an app that does number crunching over a local
    > network.
    > it have about 50 computers as slaves
    > each computer needs to run COM that will do the "job"
    > right now the system uses MFC threads and DCOM to distribute the load.
    >
    > as i said, Now i'm trying to replace this system with python.
    > I already use win32all and read about Pareller Python and Pyro.
    >
    > a requirement is that messages can be sent to the remote worker.
    >
    > If anyone can share experience I'll love to hear
    >
    > Many Thanks,
    > Yoav Glazner


    Would Celery suit? http://celeryproject.org/

    hth

    Jon.
     
    Jon Clements, Mar 24, 2010
    #2
    1. Advertising

  3. On Mar 24, 1:13 pm, Jon Clements <> wrote:
    > On 24 Mar, 15:27, Glazner <> wrote:
    >
    >
    >
    > > Hi!

    >
    > > I need to replace an app that does number crunching over a local
    > > network.
    > > it have about 50 computers as slaves
    > > each computer needs to run COM that will do the "job"
    > > right now the system uses MFC threads and DCOM to distribute the load.

    >
    > > as i said, Now i'm trying to replace this system with python.
    > > I already use win32all and read about Pareller Python and Pyro.

    >
    > > a requirement is that messages can be sent to the remote worker.

    >
    > > If anyone can share experience I'll love to hear

    >
    > > Many Thanks,
    > > Yoav Glazner

    >
    > Would Celery suit?http://celeryproject.org/
    >
    > hth
    >
    > Jon.



    If you can add just one Linux server onto that network you could use
    SIMPL-Python to do what you want.

    http://www.icanprogram.com/06py/lesson1/lesson1.html

    ie. SIMPL messages going seamlessly from Python (Windows) to Python
    (Windows) with the SIMPL sandbox being hosted on the Linux node.

    Of course if you wanted to add Linux nodes as processing nodes.
    Python (Windows) modules could seamlessly exchange messages with those
    as well. Same would be true if you wanted to distribute part of your
    app into the cloud. All your Python code would look virtually
    identical in each of these cases.

    As an example of an inexpensive Linux node the SIMPL toolkit has been
    ported to the ~$100 Sheeva Plug computer (http://www.icanprogram.com/
    simpl/plugsimplbin.self.html) (http://www.plugcomputer.org).

    bob
     
    bobicanprogram, Mar 25, 2010
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. mike
    Replies:
    10
    Views:
    8,688
    Cristian
    Feb 6, 2009
  2. Soren
    Replies:
    4
    Views:
    1,341
    c d saunter
    Feb 14, 2008
  3. Dennis Lee Bieber

    Re: [PYTHON] threading: Parallel processing

    Dennis Lee Bieber, Jan 18, 2011, in forum: Python
    Replies:
    0
    Views:
    366
    Dennis Lee Bieber
    Jan 18, 2011
  4. Vivek Menon
    Replies:
    5
    Views:
    3,491
    Paul Uiterlinden
    Jun 8, 2011
  5. Vivek Menon
    Replies:
    0
    Views:
    1,801
    Vivek Menon
    Jun 10, 2011
Loading...

Share This Page