Efficient way to remove objects from a list

[Ò»Ê×Ê«]

Hi all,

Today I wrote some code like this:

for m in self.messages:
if not m.finished:
continue

#process the message

fini = [m for m in self.messages if m.finished]
for m in fini:
self.messages.remove(m)

As you can, I want to find these finished messages in
"self.messages",
process them, and then remove them from the list.

Because a list can not be modified while iterating it, I have to use
a list "fini" to accomplish the target.

I found a smell of bad performance here.
Is there any faster ways?

 

[M.-A. Lemburg]

Hi all,

Today I wrote some code like this:

for m in self.messages:
if not m.finished:
continue

#process the message

fini = [m for m in self.messages if m.finished]
for m in fini:
self.messages.remove(m)

As you can, I want to find these finished messages in
"self.messages",
process them, and then remove them from the list.

Because a list can not be modified while iterating it, I have to use
a list "fini" to accomplish the target.

I found a smell of bad performance here.
Is there any faster ways?

The typical way to do this is to iterate over the list in reverse
order and then using the item index as basis for removing the
item:

for i, item in enumerate(reversed(mylist)):
# process item
del mylist

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, Nov 03 2008)________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611

 

[M.-A. Lemburg]

Hi all,

Today I wrote some code like this:

for m in self.messages:
if not m.finished:
continue

#process the message

fini = [m for m in self.messages if m.finished]
for m in fini:
self.messages.remove(m)

As you can, I want to find these finished messages in
"self.messages",
process them, and then remove them from the list.

Because a list can not be modified while iterating it, I have to use
a list "fini" to accomplish the target.

I found a smell of bad performance here.
Is there any faster ways?

The typical way to do this is to iterate over the list in reverse
order and then using the item index as basis for removing the
item:

for i, item in enumerate(reversed(mylist)):
# process item
del mylist

Sorry, the above should read:

for i, item in reversed(enumerate(mylist)):
# process item
del mylist

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, Nov 03 2008)________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611

 

[Peter Otten]

The typical way to do this is to iterate over the list in reverse
order and then using the item index as basis for removing the
item:

for i, item in enumerate(reversed(mylist)):
# process item
del mylist

Sorry, the above should read:

for i, item in reversed(enumerate(mylist)):
# process item
del mylist

It's harder to get right than it might first appear:

reversed(enumerate([]))

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: argument to reversed() must be a sequence

I think simplest approach is

for index in reversed(range(len(mylist))):
item = mylist[index]
...

But I usually build a new list.

Peter

 

[Peter Otten]

Hi all,

Today I wrote some code like this:

Build a new list as you go, then overwrite the old list with it.

unfinished = []

for m in self.messages:
if not m.finished: unfinished.append(m)
continue

#process the message

Remove the following code

fini = [m for m in self.messages if m.finished]
for m in fini:
self.messages.remove(m)

self.messages[:] = unfinished

Just

self.messages = unfinished

if also OK unless you have multiple references to the self.messages list.

 

[Ò»Ê×Ê«]

Thanks! That's a more clear way!

Build a new list as you go, then overwrite the old list with it.

unfinished = []

for m in self.messages:
if not m.finished: unfinished.append(m)
continue
#process the message

Remove the following code

fini = [m for m in self.messages if m.finished]
for m in fini:
self.messages.remove(m)

self.messages[:] = unfinished

Just

self.messages = unfinished

if also OK unless you have multiple references to the self.messages list.

This way you aren't calling .remove() multiple times and only iterate
through the list once.