concise code (beginner)

  • Thread starter Bruno Desthuilliers
  • Start date
B

Bruno Desthuilliers

bambam a écrit :
I have about 30 pages (10 * 3 pages each) of code like this
(following). Can anyone suggest a more compact way to
code the exception handling? If there is an exception, I need
to continue the loop, and continue the list.

Steve.

-----------------------------------
for dev in devs
try:
dev.read1()
except
print exception
remove dev from devs


for method_name in ['read1', 'read2', 'read3']:
for dev in devs:
try:
meth = getattr(dev, method_name)
except AttributeError, e:
# should not happen, but we want to handle it anyway
your_code_here()
else:
try:
meth()
except (SomePossibleException, SomeOtherPossibleException), e:
print e
# do what's needed to remove dev from devs
# paying attention not to screw the looping machinery...

(snip)
 
B

bambam

I have about 30 pages (10 * 3 pages each) of code like this
(following). Can anyone suggest a more compact way to
code the exception handling? If there is an exception, I need
to continue the loop, and continue the list.

Steve.

-----------------------------------
for dev in devs
try:
dev.read1()
except
print exception
remove dev from devs

for dev in devs
try:
dev.read2()
except
print exception
remove dev from devs

for dev in devs
try:
dev.read3()
except
print exception
remove dev from devs

etc.
 
D

Diez B. Roggisch

bambam said:
I have about 30 pages (10 * 3 pages each) of code like this
(following). Can anyone suggest a more compact way to
code the exception handling? If there is an exception, I need
to continue the loop, and continue the list.

Steve.

-----------------------------------
for dev in devs
try:
dev.read1()
except
print exception
remove dev from devs

for dev in devs
try:
dev.read2()
except
print exception
remove dev from devs

for dev in devs
try:
dev.read3()
except
print exception
remove dev from devs

for method in (DevClass.read1, DevClass.read2, ...):
for dev in devs:
try:
method(dev)
except:
print execption
remove dev from devs

Diez
 
W

Wesley Brooks

Try adding all the functions into a list such as;

funcList = [dev.read1, dev.read2, dev.read3]

for func in funcList:
for dev in devs:
try:
func()
except:
print exception
remove dev from devs

Wes.
 
W

Wesley Brooks

Sorry, just seen a mistake in my code, however Diez beat me to what I
was actually thinking!

Wes

Try adding all the functions into a list such as;

funcList = [dev.read1, dev.read2, dev.read3]

for func in funcList:
for dev in devs:
try:
func()
except:
print exception
remove dev from devs

Wes.
 
J

James Stroud

bambam said:
I have about 30 pages (10 * 3 pages each) of code like this
(following). Can anyone suggest a more compact way to
code the exception handling? If there is an exception, I need
to continue the loop, and continue the list.

Steve.

-----------------------------------
for dev in devs
try:
dev.read1()
except
print exception
remove dev from devs

for dev in devs
try:
dev.read2()
[etc.]

My keen sense of pattern recognition tells me that all of your read's
follow the same naming pattern--or was that just an accidental naming
coincidence on your part for the sake of example?


for i in xrange(number_of_reads):
for dev in devs:
try:
_reader = getattr(dev, 'read%d' % i)
_reader()
except Exception, e:
print e
devs.remove(dev)

James

--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/
 
K

Karthik Gurusamy

bambam said:
I have about 30 pages (10 * 3 pages each) of code like this
(following). Can anyone suggest a more compact way to
code the exception handling? If there is an exception, I need
to continue the loop, and continue the list.

-----------------------------------
for dev in devs
try:
dev.read1()
except
print exception
remove dev from devs
for dev in devs
try:
dev.read2()

[etc.]

My keen sense of pattern recognition tells me that all of your read's
follow the same naming pattern--or was that just an accidental naming
coincidence on your part for the sake of example?

for i in xrange(number_of_reads):
for dev in devs:
try:
_reader = getattr(dev, 'read%d' % i)
_reader()
except Exception, e:
print e
devs.remove(dev)

I see in many of the solutions suggested above, the devs sequence/
iterator is being modified while iterating. I know it is not defined
for interation over dictionary keys. Are they defined for other
collections like lists?

Karthik
 
J

James Stroud

Karthik said:
I see in many of the solutions suggested above, the devs sequence/
iterator is being modified while iterating. I know it is not defined
for interation over dictionary keys. Are they defined for other
collections like lists?

Good eye! My code is broke as you have noticed:

py> r = range(5)
py> for i in r:
.... print i
.... if i % 2:
.... r.remove(i)
....
0
1
3


For longer sequences, the algorithm I've used in these cases in the past
goes something like this:

py> r = range(10, 17)
py> print r
[10, 11, 12, 13, 14, 15, 16]
py>
py> i = 0
py> while i < len(r):
.... j = r
.... print j
.... if j % 2:
.... r.remove(j)
.... else:
.... i += 1
....
10
11
12
13
14
15
16
py> print r
[10, 12, 14, 16]


Which would change my problematic code above to:


for i in xrange(number_of_reads):
j = 0
while j < len(devs):
try:
_reader = getattr(devs[j], 'read%d' % i)
_reader()
j += 1
except Exception, e:
print e
devs.remove(dev)

Another way is to make a copy of devs, if devs is short, which makes my
problematic code into a matter of a "typo"--maybe I can make this claim
to save some face?

for i in xrange(number_of_reads):
for dev in devs[:]:
try:
_reader = getattr(dev, 'read%d' % i)
_reader()
except Exception, e:
print e
devs.remove(dev)

James

--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/
 
B

bambam

First, thank you.

All of the suggestions match what we want to do much better
than what we are doing. We have a script, written in python,
which is doing testing. But the python script doesn't look anything
like the test script, because the python script is written in python,
and the test script is a series of instrument command macros.

By putting the script sequence into a collection that is separate
from the python code, we will get script list that general engineering
will find much easier to understand:


def script(self)
def a0010(): global self; self.power_on([self.dev]);
def a0020(): global self; self.dev.addLog([self.name, ' started']);
def a0030(): global self; self.resetMinuteReg([self.dev]);
def a0040(): global self; self.disablePLmessages([self.dev]);
def a0050(): global self; self.dev.testH.writePLram((PL.BCAL12<<8));

Most of these won't generate exceptions: exceptions are expected
only on the calculations following the reads, but of course the
problem is that the exceptions are unexpected... The semi-colons
were already there, I've just stripped out the loops and exception
handlers. The original code is a mixture of functions with internal
and external [dev] loops.

Because of the sequence names, I have a choice of using generated
call names (James), or a static list of some sort.

Other possibilities might be
1) Using dir(script) to get a list of line functions
2) Using frame.f_lineno instead of line functions
3) Use an expression list instead of line functions
4) Multiple script objects with yield on each line.

The devices are in a list, and are removed by using pop(i). This
messes up the loop iteration, so it is actually done by setting a
flag on each device in the exception handler, with ANOTHER
loop after each write/read/calculate sequence. I left that out also
because I wanted to show the intended logic.

I'm not wedded to the idea of using sequence numbers for
line functions. Comments are welcome. Sorry I didn't include
this level of detail in the original, but I am interested in the
question at all levels.

Note that this achieves synchronous parallel processing --
another area I know nothing about -- but I'm just starting
with the code as I got it, and coding so far was focused
on hardware integration.

Steve.
 
M

Marc 'BlackJack' Rintsch

def script(self)
def a0010(): global self; self.power_on([self.dev]);
def a0020(): global self; self.dev.addLog([self.name, ' started']);
def a0030(): global self; self.resetMinuteReg([self.dev]);
def a0040(): global self; self.disablePLmessages([self.dev]);
def a0050(): global self; self.dev.testH.writePLram((PL.BCAL12<<8));

What is this ``global self`` meant to do? And the first line is missing a
colon at the end.

Ciao,
Marc 'BlackJack' Rintsch
 
S

Steven D'Aprano

First, thank you.

All of the suggestions match what we want to do much better than what we
are doing. We have a script, written in python, which is doing testing.
But the python script doesn't look anything like the test script,
because the python script is written in python, and the test script is a
series of instrument command macros.

By putting the script sequence into a collection that is separate from
the python code, we will get script list that general engineering will
find much easier to understand:


def script(self)
def a0010(): global self; self.power_on([self.dev]);
def a0020(): global self; self.dev.addLog([self.name, ' started']);
def a0030(): global self; self.resetMinuteReg([self.dev]);
def a0040(): global self; self.disablePLmessages([self.dev]);
def a0050(): global self; self.dev.testH.writePLram((PL.BCAL12<<8));

That's rather hideous code, and I'm not sure I understand what it's
supposed to do. As far as I can tell, you have a function called "script"
which takes a single argument. It then defines five functions, throws
them away, and returns None.

"self" is not a reserved word in Python (although perhaps it should
be...) but it has a VERY strong convention for when to use it: when
defining instance methods, it is used for the automatically-supplied
instance argument. The above functions look like they were written by
somebody who has just copied some methods from another piece of code
without understanding what they were seeing. (Sorry.)

Most of these won't generate exceptions: exceptions are expected only on
the calculations following the reads, but of course the problem is that
the exceptions are unexpected...

I don't think so. I think they are expected. You say so yourself.

The semi-colons were already there,

Fine. Time to remove them.

I've just stripped out the loops and exception handlers. The original
code is a mixture of functions with internal and external [dev] loops.

Because of the sequence names, I have a choice of using generated call
names (James), or a static list of some sort.

Other possibilities might be
1) Using dir(script) to get a list of line functions 2) Using
frame.f_lineno instead of line functions 3) Use an expression list
instead of line functions 4) Multiple script objects with yield on each
line.

My brain hurts.

The devices are in a list, and are removed by using pop(i). This messes
up the loop iteration, so it is actually done by setting a flag on each
device in the exception handler, with ANOTHER loop after each
write/read/calculate sequence. I left that out also because I wanted to
show the intended logic.

Try something like this: define a module holding the device functions.
Rather than have the functions access a global variable, which is almost
always a bad thing to do, have them take a single argument.

# module device

__all__ = [a0010, a002, a0030, a0040, a0050]

def a0010(obj):
obj.power_on([obj.dev])
def a0020(obj):
obj.dev.addLog([obj.name, ' started'])
def a0030(obj):
obj.resetMinuteReg([obj.dev])
def a0040(obj):
obj.disablePLmessages([obj.dev])
def a0050(obj):
obj.dev.testH.writePLram((PL.BCAL12<<8))


Now define a second module to call those functions.

# module tester

import device
passed = device.__all__[:] # a copy of the list
some_object = Something() # I have no idea what this should be...

for function in device.__all__:
try:
function(some_object)
except Exception, e:
print e
passed.remove(function)

print "The following functions passed:"
for function in passed:
print function


And then I suggest you spend some time reading up about doc tests and
unit tests.

Hope this helps,
 
K

Karthik Gurusamy

I see in many of the solutions suggested above, the devs sequence/
iterator is being modified while iterating. I know it is not defined
for interation over dictionary keys. Are they defined for other
collections like lists?

Good eye! My code is broke as you have noticed:

py> r = range(5)
py> for i in r:
... print i
... if i % 2:
... r.remove(i)
...
0
1
3

For longer sequences, the algorithm I've used in these cases in the past
goes something like this:

py> r = range(10, 17)
py> print r
[10, 11, 12, 13, 14, 15, 16]
py>
py> i = 0
py> while i < len(r):
... j = r
... print j
... if j % 2:
... r.remove(j)
... else:
... i += 1
...
10
11
12
13
14
15
16
py> print r
[10, 12, 14, 16]

Which would change my problematic code above to:

for i in xrange(number_of_reads):
j = 0
while j < len(devs):
try:
_reader = getattr(devs[j], 'read%d' % i)
_reader()
j += 1
except Exception, e:
print e
devs.remove(dev)

Another way is to make a copy of devs, if devs is short, which makes my
problematic code into a matter of a "typo"--maybe I can make this claim
to save some face?

for i in xrange(number_of_reads):
for dev in devs[:]:
try:
_reader = getattr(dev, 'read%d' % i)
_reader()
except Exception, e:
print e
devs.remove(dev)


Thanks, personally I like this duplicate copy solution. It's cleaner
and easier on the eye/brain. (Moreover the C like first solution
assumes the collection is a sequence, which is not true for dictionary/
set like collections).

That said, it may be a good future language enhancement to define a
reasonable consistent behavior for an iterator over a changing
collection. This occurs quite common when we walk a collection and
usually delete the current item.

For a sequence, what the expected behavior is quite obvious (just
remove this element and go over to the next). For other collections
like dictionary/set, again if the operation is delete, the expected
behavior is obvious. If we are doing insertion, for sequence a well-
defined behavior can be formulated (based on insert before or after
current position -- if after we will see it in the walk, if before we
won't see it) . For dict/set I see this isn't simple (as based on hash
key we may insert ahead or later of the current 'cursor'/position.

Karthik
 
B

bambam

Hi Steven.

Looking at your code, why are you naming the value
__all__? It looks like a built-in variable?

Unless there is an automatic way to correctly get the
function list, I will probably be better off giving the lines
sequence numbers, and generating the function list from
that.

Steve.

Steven D'Aprano said:
(e-mail address removed)...
Try something like this: define a module holding the device functions.

# module script

__all__ = [a0010, a002, a0030, a0040, a0050]
....
# module test1
import script
class Test1(pl_test.Pl_test)
"""ADC calibration and Battery Voltage calibration"""
def run(self,devlist):
for line in script.__all__:
for self.dev in devlist
if self.dev.active and not self.dev.failed
try
line(self)
except Exception,e:
print e
self.dev.active=False

print "The following devices passed:"
for dev in devlist
if dev.active and not dev.failed
print dev

print "The following devices need to be re-tested"
for dev in devlist
if not dev.active
print dev

print "The remaining devices failed"
for dev in delist
if dev.active and dev.failed
print dev
 
S

Steven D'Aprano

Hi Steven.

Looking at your code, why are you naming the value __all__? It looks
like a built-in variable?

When you say:

from module import *

Python looks in the module for a list of names called "__all__", and
imports only the names in that list. It is recommended that your modules
take advantage of that feature. I'm just using the same name.

Unless there is an automatic way to correctly get the function list, I
will probably be better off giving the lines sequence numbers, and
generating the function list from that.

I don't understand what you mean by "giving the lines sequence numbers".
Where do they come from? How do you go from sequence numbers to functions?

As far as I am concerned, the right behavior is for the module containing
the functions to define which functions need to be tested. Since modules
aren't actually intelligent, that means some person needs to list the
functions. Why list the function NAME when you can list the function
itself?
 
L

Lawrence D'Oliveiro

The devices are in a list, and are removed by using pop(i). This
messes up the loop iteration, so it is actually done by setting a
flag on each device in the exception handler, with ANOTHER
loop after each write/read/calculate sequence.

Why not just build a new list? E.g.

newdevs = []
for dev in devs :
...
if not removing_dev :
newdevs.append(dev)
#end if
#end for
devs = newdevs
 
M

Michele Simionato

First, thank you.

All of the suggestions match what we want to do much better
than what we are doing. We have a script, written in python,
which is doing testing. But the python script doesn't look anything
like the test script, because the python script is written in python,
and the test script is a series of instrument command macros.

By putting the script sequence into a collection that is separate
from the python code, we will get script list that general engineering
will find much easier to understand:

def script(self)
def a0010(): global self; self.power_on([self.dev]);
def a0020(): global self; self.dev.addLog([self.name, ' started']);
def a0030(): global self; self.resetMinuteReg([self.dev]);
def a0040(): global self; self.disablePLmessages([self.dev]);
def a0050(): global self; self.dev.testH.writePLram((PL.BCAL12<<8));

Look at generative tests in py.test or nose: they are a much
better solution for what you are doing.

Michele Simionato
 
R

Rhamphoryncus

That said, it may be a good future language enhancement to define a
reasonable consistent behavior for an iterator over a changing
collection. This occurs quite common when we walk a collection and
usually delete the current item.

For a sequence, what the expected behavior is quite obvious (just
remove this element and go over to the next). For other collections
like dictionary/set, again if the operation is delete, the expected
behavior is obvious. If we are doing insertion, for sequence a well-
defined behavior can be formulated (based on insert before or after
current position -- if after we will see it in the walk, if before we
won't see it) . For dict/set I see this isn't simple (as based on hash
key we may insert ahead or later of the current 'cursor'/position.

Removing from a list while you iterate will had quadratic performance
though. O(n) to find the element you wish to remove and move over
everything after it, multiplied by your original O(n) of iterating,
gives O(n**2). That, combined with the fact that adding enough
accounting to invalidate or update your iterator would be a cost on
all the correct users too, is why it's not done.

The best approach in almost all cases in python is to create a new
container as you iterate over the old one. After you finish, you
replace the old one with the new one. This lets you keep an overall
O(n) performance, as well as avoiding the tricky semantics.
 
B

bambam

Removing from a list while you iterate will had quadratic performance

Anecdote:
I was doing a route-finding program for a railway
ticketing system. My replacement explained to my boss
that it couldn't be done: the problem was one of that
class of problems that has no good optimum solution.
My replacement told me this while we were doing the
hand-over. I told him that the route-finding optimiser
was already complete.

It did a search of possible routes over the 3 loops, 15
branches and 75 stations in less time than it took to
redraw the screen, and that was without even bothering
to recode the tail recursion.

In my present case, the lists I am working with have ~10
elements. Defined behaviour, even if it was inapropriate
for my needs, would be welcome. Language enhancement
that make the code clearer and easier would be welcome.
Optimisers for large sets I could continue to do by other means.

Raw cPython is probably not a good choice for real-time
signal processing, but that's not what I am doing.
O(n) to find the element you wish to remove and move over
everything after it,

Is that how lists are stored in cPython? It seems unlikely?

Steve.
 
B

bambam

I'm testing a series of scripts.
The scripts are testing a series of hardware devices.
The scripts are a sequence of device commands.
The scripts have sequence numbers.

I am adding exception handling to the to the 'inner
platform' that executes sequences.

I am doing this because testing of error cases has
demonstrated that the 'inner platform' is fragile.

http://worsethanfailure.com/Articles/The_Inner-Platform_Effect.aspx

on the other hand:
http://www.amazon.com/gp/cdp/member...isplay=public&sort_by=MostRecentReview&page=5
and
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/207645

In any case, the hardware test instruments are not programmable
in python, and most of them do not have loop constructs in the
platform language.

I'm not sure how you are reading these messages. Can you see the
thread history?

Steve.
 
B

bambam

I can try that, but I'm not sure that it will work. The problem
is that devList is just a pointer to a list owned by someone else.
Making devList point to a new list won't work: I need to make
the parent list different. I could do this by adding an extra
level of indirection, but I think at the risk making the call
environment more complex.

Still, the main thing is that I hadn't even thought of doing it
that way.

Thank you,

Steve.


Lawrence D'Oliveiro said:
The devices are in a list, and are removed by using pop(i). This
messes up the loop iteration, so it is actually done by setting a
flag on each device in the exception handler, with ANOTHER
loop after each write/read/calculate sequence.

Why not just build a new list? E.g.

newdevs = []
for dev in devs :
...
if not removing_dev :
newdevs.append(dev)
#end if
#end for
devs = newdevs
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,767
Messages
2,569,572
Members
45,045
Latest member
DRCM

Latest Threads

Top