Async serial communication/threads sharing data

[Nick Timkovich]

I've been working on a program that will talk to an embedded device
over the serial port, using some basic binary communications with
messages 4-10 bytes long or so. Most of the nuts and bolts problems
I've been able to solve, and have learned a little about the threading
library to avoid blocking all action while waiting for responses
(which can take 50 ms to 10 s). Ultimately, this program will test
the device on the COM port by sending it messages and collecting
responses for 10k-100k cycles; a cycle being:
1. tell it to switch a relay,
2. get it's response from the event,
3. ask it for some measurements,
4. get measurements,
5. repeat.
Later I would like to develop a GUI as well, but not a big issue now
(another reason to use threads? not sure).

The overall structure of what I should do is not very apparent to me
on how to efficiently deal with the communications. Have the main
loop handle the overall timing of how often to run the test cycle,
then have a thread deal with all the communications, and within that
another thread that just receives data? My main issue is with how to
exchange data between different threads; can I just do something like
have a global list of messages, appending, modifying, and removing as
needed? Does the threading.Lock object just prevent every other
thread from running, or is it bound somehow to a specific object (like
my list)?

 

[Paul Rubin]

My main issue is with how to
exchange data between different threads; can I just do something like
have a global list of messages, appending, modifying, and removing as
needed? Does the threading.Lock object just prevent every other
thread from running, or is it bound somehow to a specific object (like
my list)?

The favored approach to this is to use the Queue module, which gives a
thread-safe double ended queue. This is cleaner than messing with
lower level synchronization primitives like Lock. You use Queues as
communications channels between threads.

 

[Hendrik van Rooyen]

I've been working on a program that will talk to an embedded device
over the serial port, using some basic binary communications with
messages 4-10 bytes long or so. Most of the nuts and bolts problems
I've been able to solve, and have learned a little about the threading
library to avoid blocking all action while waiting for responses
(which can take 50 ms to 10 s). Ultimately, this program will test
the device on the COM port by sending it messages and collecting
responses for 10k-100k cycles; a cycle being:
1. tell it to switch a relay,
2. get it's response from the event,
3. ask it for some measurements,
4. get measurements,
5. repeat.
Later I would like to develop a GUI as well, but not a big issue now
(another reason to use threads? not sure).

The overall structure of what I should do is not very apparent to me
on how to efficiently deal with the communications. Have the main
loop handle the overall timing of how often to run the test cycle,
then have a thread deal with all the communications, and within that
another thread that just receives data? My main issue is with how to
exchange data between different threads; can I just do something like
have a global list of messages, appending, modifying, and removing as
needed? Does the threading.Lock object just prevent every other
thread from running, or is it bound somehow to a specific object (like
my list)?

What I have been doing for similar stuff is to modularize using processes
and pipes instead of queues.
This allows you to have the communications in one process, in which
you hide all the protocol issues, and allows you to deal with the resulting
data at a slightly higher level of abstraction.
Such a structure also makes it easier to have different user interfaces -
command line or GUI, it does not matter as the commands and responses
to the comms module can be standardised via an input and output
pipe to and from the comms module.

HTH
- Hendrik