Python in Process Control?

W

Wolfgang Keller

Hello,

is it that my know-how to use Google is insufficient or...

....does really noone use Python for industrial control applications?

At least I didn't manage to find any publicly available modules for such
things as OPC/fieldbus communication etc...

TIA,

best regards,

Wolfgang Keller
 
G

Grant Edwards

is it that my know-how to use Google is insufficient or...

...does really noone use Python for industrial control applications?

I do -- I use Python to do DeviceNet stuff.
At least I didn't manage to find any publicly available
modules for such things as OPC/fieldbus communication etc...

I couldn't find any publically available DeviceNet modules for
_any_ language. That doesn't mean nobody is writing programs
that talk DeviceNet. ;)
 
D

David Fraser

Wolfgang said:
Hello,

is it that my know-how to use Google is insufficient or...

...does really noone use Python for industrial control applications?

At least I didn't manage to find any publicly available modules for such
things as OPC/fieldbus communication etc...

I had a go at OPC once. I had a working simple OPC client I think
(couldn't get the server working).
Basically OPC is all COM so it should be too hard with the win32com methods

David
 
V

Ville Vainio

Wolfgang> ...does really noone use Python for industrial control
Wolfgang> applications?

People are known to do this.

Wolfgang> At least I didn't manage to find any publicly available
Wolfgang> modules for such things as OPC/fieldbus communication
Wolfgang> etc...

Such things don't necessarily float into open source. My adventures w/
OPC involved both Python and C++ code - with my current knowledge of
Python there would probably have been much less C++ code :). OPC is
just DCOM, which should work directly w/ Python+pywin32 extensions
(used to be called win32all, which might help your googlings). Get
Mark Hammond's book Python Programming on Win32:

http://www.oreilly.com/catalog/pythonwin32/

Alternatively, you could buy/find a C library that makes OPC simple
(i.e. no need to deal w/ COM), and wrap it up in Python.

And BTW, OPC sucks. Those industrial decision makers must have smoked
lots of crack when they came up w/ the idea of using DCOM as the
integrator-facing interface that is supposed to be easy and
straightforward.
 
C

Cameron Laird

Wolfgang> ...does really noone use Python for industrial control
Wolfgang> applications?

People are known to do this.

Wolfgang> At least I didn't manage to find any publicly available
Wolfgang> modules for such things as OPC/fieldbus communication
Wolfgang> etc...

Such things don't necessarily float into open source. My adventures w/
OPC involved both Python and C++ code - with my current knowledge of
Python there would probably have been much less C++ code :). OPC is
just DCOM, which should work directly w/ Python+pywin32 extensions
(used to be called win32all, which might help your googlings). Get
Mark Hammond's book Python Programming on Win32:

http://www.oreilly.com/catalog/pythonwin32/

Alternatively, you could buy/find a C library that makes OPC simple
(i.e. no need to deal w/ COM), and wrap it up in Python.

And BTW, OPC sucks. Those industrial decision makers must have smoked
lots of crack when they came up w/ the idea of using DCOM as the
integrator-facing interface that is supposed to be easy and
straightforward.
.
.
.
Ville, I can point at far greater irrationalities in process control.

I like process control. I've threatened on occasion to launch at least
a new mailing list, and maybe an online magazine, for "highly-productive
process control", or something similar that captures the difference be-
tween, say, Python and C++. In any case, I certainly can echo the points
everyone in this thread has made: that the programming is not really so
hard, just mind-bogglingly mis-documented; that there's a lot more going
on than is googl-able; and that Python, in particular, makes a fine
vehicle.

The vendors are thick with defensive instincts about their intellectual
property. Not only are their business models difficult ones, but *they*
understand them poorly.
 
P

Peter L Hansen

Wolfgang said:
is it that my know-how to use Google is insufficient or...

...does really noone use Python for industrial control applications?

At least I didn't manage to find any publicly available modules for such
things as OPC/fieldbus communication etc...

The others have said mostly everything there is to say. I just
wanted to point out a few more items to show that Python really
is used for industrial control, and is very suitable, even if
it isn't widely used, *yet*. (And I'm sure my thoughts on this
subject are in line with Cameron's.)

While at Kaval Wireless my group used Python *extensively* for
industrial control. I can't give much in the way of detail,
though some past postings probably contain more tidbits than
they should ;-), but we were doing lots of CAN work, talking
to equipment via GPIB, and so on.

Now that I'm back to consulting as Engenuity Corporation, I'm
working with Turnkey Automation (sorry for all the name-
dropping, but that was kinda the point of the post) on a variety
of robotic and industrial control projects *all* focusing on
the use of Python to provide vastly increased productivity,
reliability, maintainability, and flexibility. (I believe
some wag in this newsgroup once opined that Python was good
at providing all the "ilities"...)

As one tiny example, I had to whip up a provisional driver for
a Tecan (Cavro) robot for a customer who needed to talk to it
with LabVIEW. I spent the majority of two weeks learning
ActiveX issues from scratch, and wasted a fair bit of time trying
to do a COM server using the win32com stuff before switching
to and successfully using ctypes. I also used the excellent
PySerial (thanks Chris!) module.

In the end, the code that represents the robot-specific aspects
of this, the protocol and command set and such, took only two
days to write. And it worked, first time (after my log messages
allowed us to troubleshoot the customer's incorrectly wired
cable) *without ever visiting the site for testing*. This was
for me an unprecedent occurrence in the history of my work in
software. (For those that don't do control: code that talks
to robots *never* works the first time. :) )

Had I done this with any of the past languages I've used for
the task, whether C, C++, Delphi, Java, I'm certain it would
have taken several times longer, probably at least as long
as two weeks. (And I *have* done similar though simpler stuff
in each of those languages which did take much longer,
especially to debug.)

I attribute the productivity to two things, roughly equally:
Python, and test-driven development.

Cameron: let's set up that list! If nothing else it might
encourage cowardly companies to start to share more things
via open source, such as drivers for robots, which while they
took time to develop are *not* part of a given company's core
IP and which, if they contributed them to the community,
would likely benefit from more improvements than the company's
own programmers will ever be able to make...

-Peter
 
W

Wolfgang Keller

Hello,
Such things don't necessarily float into open source.

Well, Python is pretty common in the university world, so I had thought
that quite a few people have already done lots of things with it and maybe
with a little bit of luck published one or the other basic module...

There seem to be some Python bits out there, for example MatPLC and
Lintouch use Python and some commercial products as well (PyDACHS), but
unfortunately there doesn't seem to be a more or less consistent library
for industrial automation or a coordinated effort to build up one.
OPC is just DCOM,

Well, if it is sooo easy, why has noone done a freeware twisted OPC module
yet...? >;->

To start with I "just" need to "emulate" an OPC server to play around a
little bit with a SCADA package, so everything more than importing a module
and instantiating an OPC server class with a few lines is a little bit over
my head...

(Could it be that I'm a little bit pampered by the vast Python module
collection available for other domains than industrial automation...? >:->)
which should work directly w/ Python+pywin32 extensions

I have the PythonWin IDE here at work.

Best regards,

Wolfgang Keller
 
P

Peter L Hansen

Wolfgang said:
There seem to be some Python bits out there, for example MatPLC and
Lintouch use Python and some commercial products as well (PyDACHS), but
unfortunately there doesn't seem to be a more or less consistent library
for industrial automation or a coordinated effort to build up one.

What, in your opinion, would it take to start one? I'm able to
offer the use of a server to host a web site, and would be happy
to register a domain name or something, but those are the easy
bits. Some kind of core group to get it off the group is likely
more important than a name, though sometimes threads in this
group might lead one to suspect otherwise. ;-)

-Peter
 
N

Neil Benn

Hello,

Just out of interest - which tecan robot were you interfacing
to - every Tecan robot that I've ever programmed comes with a high level
API (Toolbox, Gemini, Logic, WinWash, Magellen). I would think however
that controlling a tecan robot in the environment that it is typically
being used in (life sciences R+D) is not industrial control (its my job
BTW!). It's not like making milk....

IMHO, I think that Python is not a suitable piece of software for
industrial control as it has poor support for bytes (a string shouldn't
be used to store bytes!) and also bit twiddling is difficult (people who
ask about his on the Python newsgroup are usually shouted at). The
other thing that is a problem for python in this is that it is very
difficult in python to make your your client is aware of all the errors
that you can throw and as you know, when controlling equipment there is
a lot of things that are out of your control (an exception that is
rarely thrown will often not be caught if the programmer never sees it
running and gets lazy).
This is why checked exceptions (as they said in Fantastic Four -
flame on!) in this area are very handy because you are forcing the
non-expert to actively think about these use cases. One thing that
python is good for in this area is comms and threading, the GIL which is
usually a pain in the arse makes multi-threaded IO heavy apps a little
easier to write - although I still spent about 2 days trying to work out
how to make a variable volatile (culminating in understanding the
threading module!). The serial module is good but I think that a model
of input and output streams with listeners/observers would make far more
sense, currently I have to make a polling loop to receive the data, the
reason for this is that a machine may send serial data with being
prompted and this obviously needs to be lexed seperatly than a <I've
done your command here is the response> communication.

Good to see that you didn't use ActiveX (if one more company
_boasts_ that their equipment comes with 'industry standard' ActiveX I
think I'm going to cry)!! Believe it or not I still have to interact
with some equipment using DDE!!!!

This is probably off topic but it's Friday afternoon!!

Cheers,

Neil

--

Neil Benn
Senior Automation Engineer
Cenix BioScience
BioInnovations Zentrum
Tatzberg 47
D-01307
Dresden
Germany

Tel : +49 (0)351 4173 154
e-mail : (e-mail address removed)
Cenix Website : http://www.cenix-bioscience.com
 
V

Ville Vainio

Wolfgang> Well, if it is sooo easy, why has noone done a freeware
Wolfgang> twisted OPC module yet...? >;->

I didn't say it is easy, it just shouldn't be harder than, say, doing
it in C++ (even if you will find more example programs in C++).

Wolfgang> To start with I "just" need to "emulate" an OPC server
Wolfgang> to play around a little bit with a SCADA package, so
Wolfgang> everything more than importing a module and
Wolfgang> instantiating an OPC server class with a few lines is a
Wolfgang> little bit over my head...

Well, I can say it's definitely not going to be that easy. You, too
will soon learn to apprecate how much OPC sucks ;-). I did implement
an OPC client library in my previous job (after which OPC was actually
quite usable), but it's proprietary.

Wolfgang> (Could it be that I'm a little bit pampered by the vast
Wolfgang> Python module collection available for other domains
Wolfgang> than industrial automation...? >:->)

Probably. Industrial automation is not necessarily "interesting" if
you are not working there. Open industrial automation infrastructure
would be an extremely interesting avenue for academic research and
subsequent lucrative consulting, though...
 
P

Peter L Hansen

Neil said:
Just out of interest - which tecan robot were you interfacing
to - every Tecan robot that I've ever programmed comes with a high level
API (Toolbox, Gemini, Logic, WinWash, Magellen).

I meant to mention, but only barely implied, that the source is
actually available, on Engenuity's project page. It was a
Cavro RSP9000 robot.
I would think however
that controlling a tecan robot in the environment that it is typically
being used in (life sciences R+D) is not industrial control (its my job
BTW!). It's not like making milk....

I use the terms "industrial control" and "automation" rather
interchangeably, along with "laboratory robotics" and other
terms. It's all the same thing, regardless of the end user,
except when regulatory issues get involved (aeronautics, FDA
approvals, etc).
IMHO, I think that Python is not a suitable piece of software for
industrial control as it has poor support for bytes (a string shouldn't
be used to store bytes!) and also bit twiddling is difficult (people who
ask about his on the Python newsgroup are usually shouted at).

Well, let me see how to answer this politely. :) What you *think*
and what is reality are not necessarily the same thing. ;-) No,
the more polite answer is really that this is an area where perhaps
it's more a matter of opinion than anything else, and mine differs
from yours.

Python certainly has *excellent* ("perfect"?) support for manipulating
bytes, and if you consider that a string is not a string, but is
actually a sequence of bytes, then other than being misnamed for
this application area, a string provides great support for bytes...
certainly *in practice* it works wonderfully, and I guess as someone
who has used Python very successfully for industrial control I'm
somewhat qualified to say so.

As for bit-twiddling, it's rarer than you might think to have to
do more than the odd shift, OR, and AND masking, and maybe an
exclusive-OR or two for spice, and Python is well enough suited that
this isn't really an issue. Performance-wise, if you have to do
such things to a very high-bandwidth stream of data, it might run
into troubles, but (a) there are options, and (b) it hasn't bothered
me yet...

I was going to show an example from the Cavro code mentioned above,
but all it took was a few | and & operations and it's too boring
to show.
The
other thing that is a problem for python in this is that it is very
difficult in python to make your your client is aware of all the errors
that you can throw and as you know, when controlling equipment there is
a lot of things that are out of your control (an exception that is
rarely thrown will often not be caught if the programmer never sees it
running and gets lazy).

Absolutely! Uncaught exceptions would be a real pain to a customer.

So would untested code.

Not using good testing procedures would be a real sign of carelessness
or sheer incompetence. I hope no one would subcontract for industrial
control software with anyone who would actually produce software of
the sort you describe above. Of course, I recommend and use test-driven
development for my code, and don't suffer from the problems you
describe. And did I mention that the software worked flawlessly the
first time it ran, and has had no bug reports in the month or two
of use it has seen since then? ;-)

Note also that these problems have nothing to do with Python,
and everything to do with poor programming practices.
apps a little easier to write - although I still spent about 2 days
trying to work out how to make a variable volatile (culminating in
understanding the threading module!).

All variables are "volatile" in Python, at least in most senses of
the word. I don't understand what you were doing...
The serial module is good but I
think that a model of input and output streams with listeners/observers
would make far more sense, currently I have to make a polling loop to
receive the data,

You might want to investigate the use of Twisted's serial module,
which wraps around PySerial and effectively provides this, instead
of doing the polling loop approach (which, by the way, I also used
in the Cavro code but might change to Twisted in a later version).
Good to see that you didn't use ActiveX (if one more company _boasts_
that their equipment comes with 'industry standard' ActiveX I think I'm
going to cry)!! Believe it or not I still have to interact with some
equipment using DDE!!!!

Actually, we *did* use ActiveX, unfortunately, because that's what
the customer wanted to use to interface LabVIEW to the driver. It
does work, and works reliably, but in the end it's the use of
LabVIEW for this stuff to which I object more strongly than I
object to the use of ActiveX. (Without LabVIEW in the picture,
however, ActiveX would also have been unnecessary, as we would
have gone pure-Python.)

-Peter
 
D

David Fraser

Peter said:
What, in your opinion, would it take to start one? I'm able to
offer the use of a server to host a web site, and would be happy
to register a domain name or something, but those are the easy
bits. Some kind of core group to get it off the group is likely
more important than a name, though sometimes threads in this
group might lead one to suspect otherwise. ;-)

No need to do servers/websites/domains, just use sourceforge...

David
 
G

Grant Edwards

IMHO, I think that Python is not a suitable piece of software
for industrial control as it has poor support for bytes (a
string shouldn't be used to store bytes!)

It's incovenient, but it works. I often end up converting the
string representation to/from a list of integers.
and also bit twiddling is difficult (people who ask about his
on the Python newsgroup are usually shouted at).

Bit twiddling didn't used to be difficult, but it's getting
more so. :( Programs that used to be fine are now throwing
warnings about shift operations, hex constants, and things like
that.

A while back, somebody wrote a pure-python fixed-length integer
class that looked like it would be handy. A C implimentation of
byte arrays would be very handy, since then you wouldn't have
to convert back and forth between strings and lists of
integers.

The combination of a C-language byte-array and C-language
fixed-length integer operations would kill.
 
P

Peter L Hansen

David said:
No need to do servers/websites/domains, just use sourceforge...

SourceForge is, from my perspective, an incredibly unreliable
and awkward beast, and I'd be loathe to host anything on it
at this point. I _might_ host an actual project on it, but
I certainly wouldn't consider it for what I'm actually picturing
in this discussion, which is some sort of "Python in Automation"
"portal" or whatever they're called these days... something
along the lines of a combination of a wiki, maybe an RSS feed
(if someone will explain what the benefit is), a mailing
list, and maybe some sort of Vaults-style way for people to
provide links to their own projects (some of which, doubtless
would be SourceForge-hosted).

The simplest thing that could possibly work (tm) of course,
is a wiki and a mailing list, and I can have that up in
a few minutes if desired...

-Peter
 
C

Cameron Laird

.
[constructive
name-dropping]
.
.
As one tiny example, I had to whip up a provisional driver for
a Tecan (Cavro) robot for a customer who needed to talk to it
with LabVIEW. I spent the majority of two weeks learning
ActiveX issues from scratch, and wasted a fair bit of time trying
to do a COM server using the win32com stuff before switching
to and successfully using ctypes. I also used the excellent
PySerial (thanks Chris!) module.

In the end, the code that represents the robot-specific aspects
of this, the protocol and command set and such, took only two
days to write. And it worked, first time (after my log messages
allowed us to troubleshoot the customer's incorrectly wired
cable) *without ever visiting the site for testing*. This was
for me an unprecedent occurrence in the history of my work in
software. (For those that don't do control: code that talks
to robots *never* works the first time. :) )

Had I done this with any of the past languages I've used for
the task, whether C, C++, Delphi, Java, I'm certain it would
have taken several times longer, probably at least as long
as two weeks. (And I *have* done similar though simpler stuff
in each of those languages which did take much longer,
especially to debug.)

I attribute the productivity to two things, roughly equally:
Python, and test-driven development.

Cameron: let's set up that list! If nothing else it might
encourage cowardly companies to start to share more things
via open source, such as drivers for robots, which while they
took time to develop are *not* part of a given company's core
IP and which, if they contributed them to the community,
would likely benefit from more improvements than the company's
own programmers will ever be able to make...

-Peter

It's possible to eliminate one of us; Peter and I just write
the same things.

Despite my usual abhorrence of redundancy, though, I'll relate
my part. My industrial automation experience is quite shallow,
really, noteworthy only for my enthusiasm--and for the
consistency of my observations. EVERY time I do process control,
there are days (sometimes--hold your hats; this is no
exaggeration--months (!)) of pleading, "No, I really want the
documentation where you say what it's supposed to do, and what it
really does", then the the getting-the-right-cables interval of
frustration, then we get a signal through, then various miracles
occur, and the conclusion is, "You mean it only takes thirteen
lines of code to do that? Our software folks said it's not
possible."

The point is not any particular virtue in me; it's that the
advantages of high-level, test-driven development simply are not
known NEARLY as well as they might be. It consistently blows
process control vets away to see:
1. efficient delivery and diagnosis through
test-driven development; and
2. compensation for undocumented product
defects on-site, in real-time, often
while I or others are hundreds of miles
away.

My conclusions for today: Peter's right. Moreover, I *know* I
can document huge swathes of the industrial automation world far
better than they're done now. A lot of this stuff is easy, if
just explained sensibly. There are huge efficiencies still to be
gained.

Peter et al.: I'm crazy-busy the next week. Stay in touch,
though; there's got to be a better way to do this stuff, and I'm
thinking about it seriously.
 
C

Cameron Laird

.
[specific points
that deserve
reply--later]
.
.
Good to see that you didn't use ActiveX (if one more company
_boasts_ that their equipment comes with 'industry standard' ActiveX I
think I'm going to cry)!! Believe it or not I still have to interact
with some equipment using DDE!!!!
.
.
.
Believe it? Why, I'd guess if you're working in the real
world at all, there are times when you wish, "If only this
were advanced enough to support DDE."

I know about those tears that vendors' advertisements can
inspire.
 
R

Richard Brodie

IMHO, I think that Python is not a suitable piece of software for
industrial control as it has poor support for bytes (a string shouldn't
be used to store bytes!)

That's largely a matter of attitude I think; many would argue that byte
strings shouldn't be used to store text.
 
C

Cameron Laird

.
.
.
Good to see that you didn't use ActiveX (if one more company
_boasts_ that their equipment comes with 'industry standard' ActiveX I
think I'm going to cry)!! Believe it or not I still have to interact
.
.
.
The best thing about software in process control: I've
yet to hear any vendor talk about how .Net is an "industry
standard".
 
D

Dennis Lee Bieber

consistency of my observations. EVERY time I do process control,
there are days (sometimes--hold your hats; this is no
exaggeration--months (!)) of pleading, "No, I really want the
documentation where you say what it's supposed to do, and what it
really does", then the the getting-the-right-cables interval of

Not process control, but my current assignment fits this
roadblock quite well.

I'm stuck with writing a requirements document, software design
document, and then the software, to read a binary stream from device-X.
The only documentation I have for device-X is that I have a 2400baud
RS-232 connection. There is no documentation as to parity, word-length,
stop bits, or handshaking. I'm guessing 8N1 as it is binary data, but
will have to determine handshaking by trial&error.

Furthermore, I don't know how many bytes are in the data stream
per data item. The people who would know if it is fixed or variable
length, or if it even has a delimiter between data items, stonewall with
a "you have no need to know that information".

Scarier thought, device-X is loaded from device-Y which appears
to be using 5-level (+parity?) paper tape! (I'm sure it isn't a 9-pin
tape -- looked like 4pin, sprocket, 2pin). Maybe I'm expected to receive
5-bit data and convert to 8-bit to write as hex...

Just typical government imposed SNAFU... (and people wonder why
so many projects go over budget?)

--
 
E

erik.wilsher

I work with process automation myself, and I belive that writing an OPC
server is *not* a trivial task, even in python. There are some good
books available on this subject, the OPC-foundations home page contains
some good references.

If you want to get started with client code quickly, you could try the
QuickOPC ($) library from OPC-labs (www.opclabs.com). Setting up a
client in python with that library is three lines of code (really!):
------
import win32com.client
import time
s = win32com.client.Dispatch('OPCLabs.EasyDA')

def test(times=10000):
v = s.ReadItem('','Matrikon.OPC.Simulation', 'Saw-toothed
Waves.Real8','',0)
t0 = time.clock()
for i in range(times):
v = s.ReadItem('','Matrikon.OPC.Simulation', 'Saw-toothed
Waves.Real8','',0)
print time.clock()-t0

test(1000)
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top