Command line and GUI tools : need a single threading solution

[Adrian Casey]

I have a collection of multi-threaded command line tools which I want wrap a
PyQt gui around. I'm using queues to route messages from the command line
tools to the PyQt gui. The command line tools use python threads to do
their work. The gui uses a QThread object to read incoming messages.

This does not work consistently - and I've since read that mixing python
threads and QThreads is a bad idea. The command line tools work well using
python threads.

I don't want to maintain two copies of the tools - one for command line and
another for the PyQt version.

I'm thinking it may be possible to modify the command line tools to use qt
threads instead of native python threads. Is this the way to go? Are
there other options?

Adrian.

 

[Phil Thompson]

I have a collection of multi-threaded command line tools which I want wrap

a
PyQt gui around. I'm using queues to route messages from the command line
tools to the PyQt gui. The command line tools use python threads to do
their work. The gui uses a QThread object to read incoming messages.

This does not work consistently - and I've since read that mixing python
threads and QThreads is a bad idea. The command line tools work well
using
python threads.

How well mixing the two threading APIs works depends on version numbers. A
PyQt generated with SIP v4.x (rather than SIP v3.x) with Python v2.4
should be Ok.

I don't want to maintain two copies of the tools - one for command line
and
another for the PyQt version.

I'm thinking it may be possible to modify the command line tools to use qt
threads instead of native python threads. Is this the way to go? Are
there other options?

Using QThreads only should work - just tell the QApplication ctor that you
have a console application.

Phil

 

[Diez B. Roggisch]

I'm thinking it may be possible to modify the command line tools to use qt

threads instead of native python threads. Is this the way to go? Are
there other options?

Why don't you use python threads in qt - I do so and so far it didn't make
any troubles for me. And I would strongly advise against using qthreads
with your commandline-tools, as these then would only run on machines where
pyqt is installed - which opens a small part of "dependency hell" for your
users.

 

[Adrian Casey]

Why don't you use python threads in qt - I do so and so far it didn't make
any troubles for me. And I would strongly advise against using qthreads
with your commandline-tools, as these then would only run on machines
where pyqt is installed - which opens a small part of "dependency hell"
for your users.

I have a QThread which polls a queue object via queue.get(). The command
line tools spawn a number of threads each of which writes its output to
this queue using queue.put(). As soon as the gui gets something off the
queue, it creates a QCustomEvent and sets the data property with the data
read from the queue. My application has a customEvent() method which reads
the data item from the customEvent and processes it accordingly.

The odd thing is, I have a non-threaded version of the command line tools
which work 100% with the gui. The multi-threaded version of the command
line tools all work OK at the console - just not with the gui.

I will try your suggestion and replace my QCustomEvent mechanism with a
plain python queue.

 

[Adrian Casey]

How well mixing the two threading APIs works depends on version numbers. A
PyQt generated with SIP v4.x (rather than SIP v3.x) with Python v2.4
should be Ok.


Using QThreads only should work - just tell the QApplication ctor that you
have a console application.

Phil

Phil,

I'm running python 2.4, sip-4.0 and PyQt-x11-gpl-3.11 (or .13 - not sure as
I'm not at work).

A little more detail.

My script, if invoked from the command line, creates a queue object (let's
call it logQueue) which is used to take the output from each individual
thread. If invoked via the gui, the script is passed a logQueue created by
the gui. Each thread dumps its output using logQueue.put(). There are
other queues which are used to service threaded access to a database etc.

The scripts use the pexpect module to login to remote systems and do
sysadmin tasks (e.g. reset passwords). This works like a charm in console
mode.

The gui uses a QThread to poll logQueue (logQueue.get()). When it gets
something, it creates a QCustomEvent and sets the data portion using the
data read from the logQueue. I have a customEvent method in my application
which grabs the data from the customEvent and processes it accrodingly
(writing output to a QTable).

The gui locks up after an arbitrary number of rows have been inserted in the
QTable. It is not consistent. Sometimes it does not lock at all.

I have a non-threaded set of the command line tools which run perfectly with
the gui.

Adrian.

 

[Adrian Casey]

I have a QThread which polls a queue object via queue.get(). The command
line tools spawn a number of threads each of which writes its output to
this queue using queue.put(). As soon as the gui gets something off the
queue, it creates a QCustomEvent and sets the data property with the data
read from the queue. My application has a customEvent() method which
reads the data item from the customEvent and processes it accordingly.

The odd thing is, I have a non-threaded version of the command line tools
which work 100% with the gui. The multi-threaded version of the command
line tools all work OK at the console - just not with the gui.

I will try your suggestion and replace my QCustomEvent mechanism with a
plain python queue.

I tried replacing the QThread part with native python threads and, although
it worked for a few minutes, I started to see XWindow errors and my
application would then crash.

On Phil Thompson's advice, I updated PyQt and sip. The problem appears to
be fixed.

Adrian.