Command line and GUI tools : need a single threading solution

Discussion in 'Python' started by Adrian Casey, Jan 10, 2005.

  1. Adrian Casey

    Adrian Casey Guest

    I have a collection of multi-threaded command line tools which I want wrap a
    PyQt gui around. I'm using queues to route messages from the command line
    tools to the PyQt gui. The command line tools use python threads to do
    their work. The gui uses a QThread object to read incoming messages.

    This does not work consistently - and I've since read that mixing python
    threads and QThreads is a bad idea. The command line tools work well using
    python threads.

    I don't want to maintain two copies of the tools - one for command line and
    another for the PyQt version.

    I'm thinking it may be possible to modify the command line tools to use qt
    threads instead of native python threads. Is this the way to go? Are
    there other options?

    Adrian.
     
    Adrian Casey, Jan 10, 2005
    #1
    1. Advertising

  2. > I have a collection of multi-threaded command line tools which I want wrap
    > a
    > PyQt gui around. I'm using queues to route messages from the command line
    > tools to the PyQt gui. The command line tools use python threads to do
    > their work. The gui uses a QThread object to read incoming messages.
    >
    > This does not work consistently - and I've since read that mixing python
    > threads and QThreads is a bad idea. The command line tools work well
    > using
    > python threads.


    How well mixing the two threading APIs works depends on version numbers. A
    PyQt generated with SIP v4.x (rather than SIP v3.x) with Python v2.4
    should be Ok.

    > I don't want to maintain two copies of the tools - one for command line
    > and
    > another for the PyQt version.
    >
    > I'm thinking it may be possible to modify the command line tools to use qt
    > threads instead of native python threads. Is this the way to go? Are
    > there other options?


    Using QThreads only should work - just tell the QApplication ctor that you
    have a console application.

    Phil
     
    Phil Thompson, Jan 10, 2005
    #2
    1. Advertising

  3. > I'm thinking it may be possible to modify the command line tools to use qt
    > threads instead of native python threads. Is this the way to go? Are
    > there other options?


    Why don't you use python threads in qt - I do so and so far it didn't make
    any troubles for me. And I would strongly advise against using qthreads
    with your commandline-tools, as these then would only run on machines where
    pyqt is installed - which opens a small part of "dependency hell" for your
    users.

    --
    Regards,

    Diez B. Roggisch
     
    Diez B. Roggisch, Jan 10, 2005
    #3
  4. Adrian Casey

    Adrian Casey Guest

    Diez B. Roggisch wrote:

    >> I'm thinking it may be possible to modify the command line tools to use
    >> qt
    >> threads instead of native python threads. Is this the way to go? Are
    >> there other options?

    >
    > Why don't you use python threads in qt - I do so and so far it didn't make
    > any troubles for me. And I would strongly advise against using qthreads
    > with your commandline-tools, as these then would only run on machines
    > where pyqt is installed - which opens a small part of "dependency hell"
    > for your users.
    >

    I have a QThread which polls a queue object via queue.get(). The command
    line tools spawn a number of threads each of which writes its output to
    this queue using queue.put(). As soon as the gui gets something off the
    queue, it creates a QCustomEvent and sets the data property with the data
    read from the queue. My application has a customEvent() method which reads
    the data item from the customEvent and processes it accordingly.

    The odd thing is, I have a non-threaded version of the command line tools
    which work 100% with the gui. The multi-threaded version of the command
    line tools all work OK at the console - just not with the gui.

    I will try your suggestion and replace my QCustomEvent mechanism with a
    plain python queue.
     
    Adrian Casey, Jan 11, 2005
    #4
  5. Adrian Casey

    Adrian Casey Guest

    Phil Thompson wrote:

    >> I have a collection of multi-threaded command line tools which I want
    >> wrap a
    >> PyQt gui around. I'm using queues to route messages from the command
    >> line tools to the PyQt gui. The command line tools use python threads to
    >> do
    >> their work. The gui uses a QThread object to read incoming messages.
    >>
    >> This does not work consistently - and I've since read that mixing python
    >> threads and QThreads is a bad idea. The command line tools work well
    >> using
    >> python threads.

    >
    > How well mixing the two threading APIs works depends on version numbers. A
    > PyQt generated with SIP v4.x (rather than SIP v3.x) with Python v2.4
    > should be Ok.
    >
    >> I don't want to maintain two copies of the tools - one for command line
    >> and
    >> another for the PyQt version.
    >>
    >> I'm thinking it may be possible to modify the command line tools to use
    >> qt
    >> threads instead of native python threads. Is this the way to go? Are
    >> there other options?

    >
    > Using QThreads only should work - just tell the QApplication ctor that you
    > have a console application.
    >
    > Phil

    Phil,

    I'm running python 2.4, sip-4.0 and PyQt-x11-gpl-3.11 (or .13 - not sure as
    I'm not at work).

    A little more detail.

    My script, if invoked from the command line, creates a queue object (let's
    call it logQueue) which is used to take the output from each individual
    thread. If invoked via the gui, the script is passed a logQueue created by
    the gui. Each thread dumps its output using logQueue.put(). There are
    other queues which are used to service threaded access to a database etc.

    The scripts use the pexpect module to login to remote systems and do
    sysadmin tasks (e.g. reset passwords). This works like a charm in console
    mode.

    The gui uses a QThread to poll logQueue (logQueue.get()). When it gets
    something, it creates a QCustomEvent and sets the data portion using the
    data read from the logQueue. I have a customEvent method in my application
    which grabs the data from the customEvent and processes it accrodingly
    (writing output to a QTable).

    The gui locks up after an arbitrary number of rows have been inserted in the
    QTable. It is not consistent. Sometimes it does not lock at all.

    I have a non-threaded set of the command line tools which run perfectly with
    the gui.

    Adrian.
     
    Adrian Casey, Jan 11, 2005
    #5
  6. Adrian Casey

    Adrian Casey Guest

    Adrian Casey wrote:

    > Diez B. Roggisch wrote:
    >
    >>> I'm thinking it may be possible to modify the command line tools to use
    >>> qt
    >>> threads instead of native python threads. Is this the way to go? Are
    >>> there other options?

    >>
    >> Why don't you use python threads in qt - I do so and so far it didn't
    >> make any troubles for me. And I would strongly advise against using
    >> qthreads with your commandline-tools, as these then would only run on
    >> machines where pyqt is installed - which opens a small part of
    >> "dependency hell" for your users.
    >>

    > I have a QThread which polls a queue object via queue.get(). The command
    > line tools spawn a number of threads each of which writes its output to
    > this queue using queue.put(). As soon as the gui gets something off the
    > queue, it creates a QCustomEvent and sets the data property with the data
    > read from the queue. My application has a customEvent() method which
    > reads the data item from the customEvent and processes it accordingly.
    >
    > The odd thing is, I have a non-threaded version of the command line tools
    > which work 100% with the gui. The multi-threaded version of the command
    > line tools all work OK at the console - just not with the gui.
    >
    > I will try your suggestion and replace my QCustomEvent mechanism with a
    > plain python queue.

    I tried replacing the QThread part with native python threads and, although
    it worked for a few minutes, I started to see XWindow errors and my
    application would then crash.

    On Phil Thompson's advice, I updated PyQt and sip. The problem appears to
    be fixed.

    Adrian.
     
    Adrian Casey, Jan 12, 2005
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Errol Smith
    Replies:
    12
    Views:
    1,373
    Errol Smith
    Sep 24, 2004
  2. Bernd Oninger
    Replies:
    1
    Views:
    819
    Martin Honnen
    Jul 7, 2004
  3. Replies:
    0
    Views:
    396
  4. Daniel Pitts
    Replies:
    7
    Views:
    438
    Erik Wikström
    Apr 1, 2008
  5. ela
    Replies:
    12
    Views:
    363
    Uri Guttman
    Apr 6, 2009
Loading...

Share This Page