Newbie - How to iterate list or scalar ?

A

Andy Dingley

I seem to be writing the following fragment an awful lot, and I'm sure
it's not the best and Pythonic way to do things. Any advice on better
coding style?

pluginVersionNeeded is a parameter passed into a method and it can
either be a simple scalar variable, or it can be a list of the same
variables. My problem is how to iterate it, whether it's a a list or
not.

I can't simply throw it into the for...in loop -- if the scalar
variable were to be a string (which is considered to be atomic in my
application) then Python sees this string as iterable and iterates over
the characters in it!


versionsNeeded = pluginVersionNeeded
if isinstance( versionsNeeded, list):
versionsToTest = versionsNeeded
else:
versionsToTest = [ versionsNeeded ]
for versionNeeded in versionsToTest:
pass


Thanks for any advice
 
R

Richard Brodie

pluginVersionNeeded is a parameter passed into a method and it can
either be a simple scalar variable, or it can be a list of the same
variables.

The obvious question would be, "is there a good reason why you don't
change the API to always require a list?" Then you can just write:
myFunction( [scalarParameter] ) when you have only one variable.
 
D

Diez B. Roggisch

Andy said:
I seem to be writing the following fragment an awful lot, and I'm sure
it's not the best and Pythonic way to do things. Any advice on better
coding style?

pluginVersionNeeded is a parameter passed into a method and it can
either be a simple scalar variable, or it can be a list of the same
variables. My problem is how to iterate it, whether it's a a list or
not.

I can't simply throw it into the for...in loop -- if the scalar
variable were to be a string (which is considered to be atomic in my
application) then Python sees this string as iterable and iterates over
the characters in it!


versionsNeeded = pluginVersionNeeded
if isinstance( versionsNeeded, list):
versionsToTest = versionsNeeded
else:
versionsToTest = [ versionsNeeded ]
for versionNeeded in versionsToTest:
pass

Use a function.

def listify(v):
if not isinstance(v, list):
return [v]
return v

versionsToTest = listify(versionsNeeded)

Diez
 
B

Bruno Desthuilliers

Andy said:
I seem to be writing the following fragment an awful lot, and I'm sure
it's not the best and Pythonic way to do things. Any advice on better
coding style?

pluginVersionNeeded is a parameter passed into a method and it can
either be a simple scalar variable, or it can be a list of the same
variables. My problem is how to iterate it, whether it's a a list or
not.

I can't simply throw it into the for...in loop -- if the scalar
variable were to be a string (which is considered to be atomic in my
application) then Python sees this string as iterable and iterates over
the characters in it!

Yes, this is a common gotcha. Problem is that, as someone here noticed
sometimes ago, what should be a scalar and what should be an iterable is
often application or context dependent... And according to the context,
string being iterable can be either a royal PITA or the RightThing(tm).

versionsNeeded = pluginVersionNeeded
if isinstance( versionsNeeded, list):
versionsToTest = versionsNeeded
else:
versionsToTest = [ versionsNeeded ]
for versionNeeded in versionsToTest:
pass

<OT>
Too much names essentially refering to the same thing IMHO... this could
be simplified to:

if not isinstance(pluginVersionNeeded, list):
pluginVersionNeeded = [pluginVersionNeeded]
for versionNeeded in pluginVersionNeeded:
pass
</OT>

The second problem here is that, given the above (I mean in your
original snippet) use of versionsToTest, there's really no reason to
assume it should be a list - any iterable could - and IMHO should - be
accepted... expect of course for strings (royal PITA case, duh).

AS far as I'm concerned, I'd rather either:

1/ test versionsToTest against basestring (which is the parent class for
both strings and unicode) and turn it into a list if needed

if isinstance( pluginVersionsNeeded, basestring):
pluginVersionsNeeded = [pluginVersionsNeeded]
for versionNeeded in pluginVersionsNeeded:
pass

The problem is that it limits 'scalars' to strings (and unicodes), and
will fail with say ints or floats etc...

This could be handled with a tuple of 'to-be-considered-as-scalar' types:
scalars = (basestring, int, float)
if isinstance( pluginVersionsNeeded, scalars):
pluginVersionsNeeded = [pluginVersionsNeeded]
for versionNeeded in pluginVersionsNeeded:
pass

2/ test for pluginVersionsNeeded.__iter__ (an attribute of most
iterables except strings...):

if not hasattr(pluginVersionsNeeded, '__iter__'):
pluginVersionsNeeded = [pluginVersionsNeeded]
for versionNeeded in pluginVersionsNeeded:
pass

It's probably the most general scheme, but won't work if you intend to
consider tuples as scalars since tuples have the __iter__ attribute.


Also, if this is a common pattern in your module, you perhaps want to
abstract this out (example for both of the above solutions):

1/
def enumerateVersion(versions, scalars=(basestring, int, float)):
if isinstance(versions, scalars):
yield versions
raise StopIteration # not required but clearer IMHO
else:
for version in versions:
yield version

NB : the default arg 'scalars' let you customize what is to be
considered as a scalar...

2/
def enumerateVersion(versions):
if hasattr(versions, '__iter__'):
for version in versions:
yield version
raise StopIteration # not required but clearer IMHO
else:
yield versions


and in the calling code:
...
for versionNeeded in enumerateVersions(pluginVersionNeeded):
do_whatever_with(versionNeeeded)
...



HTH
 
A

Andy Dingley

Bruno said:
there's really no reason to
assume it should be a list - any iterable could - and IMHO should - be
accepted... expect of course for strings (royal PITA case, duh).

2/ test for pluginVersionsNeeded.__iter__ (an attribute of most
iterables except strings...):

strings don't have __iter__ ?!?!

I'm never going to get my head round this language 8-(

I can understand strings and tuples being iterable, if you take a
sufficiently first-class view of things, but why shouldn't everything
"that is iterable" support the function that makes iteration work?
 
B

Bruno Desthuilliers

Andy said:
strings don't have __iter__ ?!?!

No. Should they ?-)
I'm never going to get my head round this language 8-(

Hmmm.... While simple to get started with, Python is *not* a 'simple'
language. There's a price to pay for it's flexibility, and this price is
named "complexity". While one doesn't necessary needs to deal with it,
this complexity shows as soon as you start to dig deeper into "advanced"
features.


FWIW, if I judge on source code I've seen so far, the canonical way to
distinguish a string from another sequence type or iterable is still to
use isinstance(obj, basestring), and I don't know if I should have
mentionned the hasattr(obj, '__iter__') hack at all.

I can understand strings and tuples being iterable, if you take a
sufficiently first-class view of things, but why shouldn't everything
"that is iterable" support the function that makes iteration work?

Actually, __iter__ is not needed to allow for loops on an object if the
object defines __getitem__ so that it supports numeric indexing:

class MySequence(object):
def __init__(self):
self._data = range(3)
def __getitem__(self, key):
return self._data[key]
....
0
1
2

FWIW, the iterator protocol appeared with Python 2.2. Before this
version, the above solution was the only one that allowed iteration over
a container type.

Now if you wonder why string, unicode and buffer don't have __iter__
while other sequence types have it, ask your favourite Python guru (and
please keep me informed !-). Fact is that Python is not exactly a
newborn language (first release was around 1990 IIRC), and it has
greatly evolved over the year. The BDFL being very conservative, great
cares have been taken to ensure compatibility with older versions - with
the side effect that there are lots of stuff that now looks like warts
or inconsistencies. The 3K release is supposed to be the big cleanup,
but until then, we'll have to live with all the compat stuff.

HTH
 
T

Terry Reedy

Bruno Desthuilliers said:
FWIW, the iterator protocol appeared with Python 2.2. Before this
version, the above solution was the only one that allowed iteration over
a container type.

Now if you wonder why string, unicode and buffer don't have __iter__
while other sequence types have it, ask your favourite Python guru (and
please keep me informed !-).

Giving that Python still has to support the old __getitem__ protocol in for
statements, there was no need to rewrite stuff that worked and no one did
so for those types. The emergent problem of identifying an 'iterable' had
been noted and Guido has, I believe, approved the addition of '__iter__' to
those types for 2.6. (But someone still has to write the patches ;-)
Fact is that Python is not exactly a
newborn language (first release was around 1990 IIRC), and it has
greatly evolved over the year. The BDFL being very conservative, great
cares have been taken to ensure compatibility with older versions - with
the side effect that there are lots of stuff that now looks like warts
or inconsistencies. The 3K release is supposed to be the big cleanup,
but until then, we'll have to live with all the compat stuff.

I am sure the support for iteration via __getitem__ will disappear in 3.0.

Terry Jan Reedy
 
B

Bruno Desthuilliers

Terry said:
Giving that Python still has to support the old __getitem__ protocol in for
statements, there was no need to rewrite stuff that worked and no one did
so for those types.

What I wonder here is why __iter__ has been added to lists and tuples
but not to strings (not that I'm complaining, it's just curiousity...)
The emergent problem of identifying an 'iterable' had
been noted and Guido has, I believe, approved the addition of '__iter__' to
those types for 2.6. (But someone still has to write the patches ;-)

Which is a GoodThing(tm) wrt/ consistency - and makes clear I should not
have talked about the hasattr(obj, '__iter__') hack.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,756
Messages
2,569,533
Members
45,007
Latest member
OrderFitnessKetoCapsules

Latest Threads

Top