string issue

[Jeremy Bowers]

In this case then, shouldn't my 'except Exception' raise an error or
warning like:

"Hey, stupid, you can't iterate and object and change it at the same time!"

But you can do that, so why should it?

Certain operations are dangerous, but, well, it's impossible to enumerate
all the dangerous things and it's better to not give people false
assurances when they find one of them that slipped by.

One idiom that does what you want, though it doesn't always work for all
situations, is to iterate over the list backwards. If you're only removing
a few items from a short list, that can be faster than building a new one.
However, removing items gets to be n^2 pretty fast, so it can be better to
build a new list, which the list comprehensions make easy:

newList = [x for x in oldList if myCondition(x)]

 

[rbt]

Either I'm crazy and I'm missing the obvious here or there is something
wrong with this code. Element 5 of this list says it doesn't contain the
string 255, when that's *ALL* it contains... why would it think that???

import time

ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98',
'127.0.0.1', '255.0.0.0', '255', '128.173.255.34']

for ip in ips:
if '255' in ip:
try:
print "Removing", ip
ips.remove(ip)
except Exception, e:
print e

print ips
time.sleep(5)

Someone tell me I'm going crazy

 

[Steve Holden]

Either I'm crazy and I'm missing the obvious here or there is something
wrong with this code. Element 5 of this list says it doesn't contain the
string 255, when that's *ALL* it contains... why would it think that???

import time

ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98',
'127.0.0.1', '255.0.0.0', '255', '128.173.255.34']

for ip in ips:
if '255' in ip:
try:
print "Removing", ip
ips.remove(ip)
except Exception, e:
print e

print ips
time.sleep(5)

Someone tell me I'm going crazy

You are modifying the list as you iterate over it. Instead, iterate over
a copy by using:

for ip in ips[:]:
...

regards
Steve

 

[Alan McIntyre]

I think it's because you're modifying the list as you're iterating over
it. Try this:

import time

ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98',
'127.0.0.1', '255.0.0.0', '255', '128.173.255.34']
ips_new = []
for ip in ips:
if '255' not in ip:
ips_new.append(ip)

print ips_new



Or this:

ips_new = [ip for ip in ips if '255' not in ip]
print ips_new


Hope this helps,
Alan McIntyre
http://www.esrgtech.com

 

[Steven Bethard]

Either I'm crazy and I'm missing the obvious here or there is something
wrong with this code. Element 5 of this list says it doesn't contain the
string 255, when that's *ALL* it contains... why would it think that???

import time

ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98',
'127.0.0.1', '255.0.0.0', '255', '128.173.255.34']

for ip in ips:
if '255' in ip:
try:
print "Removing", ip
ips.remove(ip)
except Exception, e:
print e

print ips
time.sleep(5)

Someone tell me I'm going crazy

You're going crazy.

py> ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98',
'127.0.0.1', '255.0.0.0', '255', '128.173.255.34']
py> for ip in ips:
.... # debugging statement:
.... print "Looking at", ip
.... if '255' in ip:
.... try:
.... print "Removing", ip
.... ips.remove(ip)
.... except Exception, e:
.... print e
....
Looking at 255.255.255.255
Removing 255.255.255.255
Looking at 198.82.247.98
Looking at 127.0.0.1
Looking at 255.0.0.0
Removing 255.0.0.0
Looking at 128.173.255.34
Removing 128.173.255.34

Notice how elements of your list are being skipped. The problem is that
you're modifying a list while you iterate over it.

Why don't you try a list comprehension:

py> ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98',
'127.0.0.1', '255.0.0.0', '255', '128.173.255.34']
py> [ip for ip in ips if '255' not in ip]
['128.173.120.79', '198.82.247.98', '127.0.0.1']

Steve

 

[Bill Mill]

Either I'm crazy and I'm missing the obvious here or there is something
wrong with this code. Element 5 of this list says it doesn't contain the
string 255, when that's *ALL* it contains... why would it think that???

import time

ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98',
'127.0.0.1', '255.0.0.0', '255', '128.173.255.34']

for ip in ips:
if '255' in ip:
try:
print "Removing", ip
ips.remove(ip)
except Exception, e:
print e

print ips
time.sleep(5)

You're gong crazy:

ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98', .... '127.0.0.1', '255.0.0.0', '255', '128.173.255.34']
for ip in ips:

.... if '255' in ip: print ip
....
255.255.255.255
255.0.0.0
255
128.173.255.34

The problem is that you're operating in-place on an array while it's
being iterated over. Since the iterator is only created once, you're
can't change the array while you're iterating over it. Instead, try a
list comprehension:

ips = [ip for ip in ips if '255' not in ip]
ips

['128.173.120.79', '198.82.247.98', '127.0.0.1']

Peace
Bill Mill
bill.mill at gmail.com

 

[rbt]

Thanks guys... list comprehension it is!

Either I'm crazy and I'm missing the obvious here or there is something
wrong with this code. Element 5 of this list says it doesn't contain the
string 255, when that's *ALL* it contains... why would it think that???

import time

ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98',
'127.0.0.1', '255.0.0.0', '255', '128.173.255.34']

for ip in ips:
if '255' in ip:
try:
print "Removing", ip
ips.remove(ip)
except Exception, e:
print e

print ips
time.sleep(5)

You're gong crazy:

ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98',

... '127.0.0.1', '255.0.0.0', '255', '128.173.255.34']

... if '255' in ip: print ip
...
255.255.255.255
255.0.0.0
255
128.173.255.34

The problem is that you're operating in-place on an array while it's
being iterated over. Since the iterator is only created once, you're
can't change the array while you're iterating over it. Instead, try a
list comprehension:

ips = [ip for ip in ips if '255' not in ip]
ips

['128.173.120.79', '198.82.247.98', '127.0.0.1']

Peace
Bill Mill
bill.mill at gmail.com

 

[Alan McIntyre]

Wow, that's cool; I'd never seen that before. Thanks, Steve..

 

[rbt]

Either I'm crazy and I'm missing the obvious here or there is
something wrong with this code. Element 5 of this list says it doesn't
contain the string 255, when that's *ALL* it contains... why would it
think that???

import time

ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98',
'127.0.0.1', '255.0.0.0', '255', '128.173.255.34']

for ip in ips:
if '255' in ip:
try:
print "Removing", ip
ips.remove(ip)
except Exception, e:
print e

print ips
time.sleep(5)

Someone tell me I'm going crazy

You are modifying the list as you iterate over it. Instead, iterate over
a copy by using:

for ip in ips[:]:
...

regards
Steve

Very neat. That's a trick that everyone should know about. I vote it
goes in Dr. Dobbs newsletter.

 

[Steven Bethard]

You are modifying the list as you iterate over it. Instead, iterate over
a copy by using:

for ip in ips[:]:
...

Also worth noting, you can write this like:

for ip in list(ips):
...

if you're afraid of smiley-faces [:] in your code.

Steve

 

[rbt]

I think it's because you're modifying the list as you're iterating over

In this case then, shouldn't my 'except Exception' raise an error or
warning like:

"Hey, stupid, you can't iterate and object and change it at the same time!"

Or, perhaps something similar?

 

[Fredrik Lundh]

You are modifying the list as you iterate over it. Instead, iterate over a copy by using:

for ip in ips[:]:
...

regards
Steve

Very neat. That's a trick that everyone should know about.

I suppose that's why it's included in the Python tutorial:

http://docs.python.org/tut/node6.html#SECTION006200000000000000000

It is not safe to modify the sequence being iterated over in the loop
(this can only happen for mutable sequence types, such as lists). If
you need to modify the list you are iterating over (for example, to
duplicate selected items) you must iterate over a copy. The slice
notation makes this particularly convenient:

>>> for x in a[:]: # make a slice copy of the entire list

... if len(x) > 6: a.insert(0, x)

(slice notation is explained in an earlier chapter)

</F>

 

[rbt]

I think it's because you're modifying the list as you're iterating over
it.

One last clarification on this. It's OK to modify the elements of a
list, but not the list itself while iterating over it... is that the
correct way to think about this?

 

[Bill Mill]

Either I'm crazy and I'm missing the obvious here or there is
something wrong with this code. Element 5 of this list says it doesn't
contain the string 255, when that's *ALL* it contains... why would it
think that???

import time

ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98',
'127.0.0.1', '255.0.0.0', '255', '128.173.255.34']

for ip in ips:
if '255' in ip:
try:
print "Removing", ip
ips.remove(ip)
except Exception, e:
print e

print ips
time.sleep(5)

Someone tell me I'm going crazy

You are modifying the list as you iterate over it. Instead, iterate over
a copy by using:

for ip in ips[:]:
...

regards
Steve

Very neat. That's a trick that everyone should know about. I vote it
goes in Dr. Dobbs newsletter.

Once you know it, it's neat, and I use it sometimes. However, it's a
little too "magical" for my tastes; I'd rather be more explicit about
what's going on.

Peace
Bill Mill
bill.mill at gmail.com

 

[John J. Lee]

You are modifying the list as you iterate over it. Instead, iterate
over a copy by using:

for ip in ips[:]:
...

Just to help popularise the alternative idiom, which IMO is
significantly less cryptic (sane constructors of mutable objects
almost always make a copy, and list is no exception: it's guaranteed
to do so):

for ip in list(ips):
...


Works back to at least Python 1.5.2.


John

 

[rbt]

You are modifying the list as you iterate over it. Instead, iterate
over a copy by using:

for ip in ips[:]:
...

Just to help popularise the alternative idiom, which IMO is
significantly less cryptic (sane constructors of mutable objects
almost always make a copy, and list is no exception: it's guaranteed
to do so):

for ip in list(ips):
...


Works back to at least Python 1.5.2.


John

I don't know that that approach is less cryptic. ips is already a
list... it looks cryptic to make it a list again, doesn't it? IMO, the
two are equally cryptic. The epitome of clarity would be copy(ips)...
now *that* makes sense, of course, ips[:] or list(ips) work equally well
to the programmer who has learned them.

 

[Bill Mill]

You are modifying the list as you iterate over it. Instead, iterate
over a copy by using:

for ip in ips[:]:
...

Just to help popularise the alternative idiom, which IMO is
significantly less cryptic (sane constructors of mutable objects
almost always make a copy, and list is no exception: it's guaranteed
to do so):

for ip in list(ips):
...


Works back to at least Python 1.5.2.


John

I don't know that that approach is less cryptic. ips is already a
list... it looks cryptic to make it a list again, doesn't it? IMO, the
two are equally cryptic. The epitome of clarity would be copy(ips)...
now *that* makes sense, of course, ips[:] or list(ips) work equally well
to the programmer who has learned them.
Howsabout:

from copy import copy
ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98', .... '127.0.0.1', '255.0.0.0', '255', '128.173.255.34']
for ip in copy(ips):

.... if '255' in ip:
.... ips.remove(ip)
....['128.173.120.79', '198.82.247.98', '127.0.0.1']

But I still think that the list comprehension is the best.

Peace
Bill Mill
bill.mill at gmail.com

 

[rbt]

[...]


You are modifying the list as you iterate over it. Instead, iterate
over a copy by using:

for ip in ips[:]:
...


Just to help popularise the alternative idiom, which IMO is
significantly less cryptic (sane constructors of mutable objects
almost always make a copy, and list is no exception: it's guaranteed
to do so):

for ip in list(ips):
...


Works back to at least Python 1.5.2.


John

I don't know that that approach is less cryptic. ips is already a
list... it looks cryptic to make it a list again, doesn't it? IMO, the
two are equally cryptic. The epitome of clarity would be copy(ips)...
now *that* makes sense, of course, ips[:] or list(ips) work equally well
to the programmer who has learned them.

Howsabout:

from copy import copy
ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98',

... '127.0.0.1', '255.0.0.0', '255', '128.173.255.34']

... if '255' in ip:
... ips.remove(ip)
...

['128.173.120.79', '198.82.247.98', '127.0.0.1']

But I still think that the list comprehension is the best.

Peace
Bill Mill
bill.mill at gmail.com

Wow, I did not know that a copy module existed. I made all that up about
copy being the perfect example here. Great minds think alike I fell
Guidoish.

 

[Terry Reedy]

You are modifying the list as you iterate over it. Instead, iterate over

a copy by using:

for ip in ips[:]:

Or if you are only deleting items, you can iterate backwards.

You can also build a new list with only the items you want. Do this either
with an explicit loop or even better, use filter or a list comp.

ips = ['255.255.255.255', '128.173.120.79', '198.82.247.98',
'127.0.0.1', '255.0.0.0', '255', '128.173.255.34']

# 2.2 requires single char for 'not in string' test

ips2 = []
for ip in ips:
if ip.find('255') == -1: ips2.append(ip)

print ips2
print filter(lambda ip: ip.find('255') == -1, ips)
print [ip for ip in ips if ip.find('255') == -1]

# thrice prints
['128.173.120.79', '198.82.247.98', '127.0.0.1']

This seems much more sensible to me than building a new list with
everything (a copy), including things you don't want, and then deleting the
things you don't want (an O(n**2) operation).

Terry J. Reedy

 

[Steve Holden]

[ ... ]

This seems much more sensible to me than building a new list with
everything (a copy), including things you don't want, and then deleting the
things you don't want (an O(n**2) operation).

Which is presumably why the OP decided to go with the list
comprehension. My main intention was to point out how the code in error
could be fixed while retaining the same structure. You are of course
correct in saying this is far from optimal in performance terms.

regards
Steve