Advantage of the array module over lists?

[Tobiah]

I checked out the array module today. It claims that
arrays are 'efficient'. I figured that this must mean
that they are faster than lists, but this doesn't seem
to be the case:

################ one.py ##############
import array

a = array.array('i')

for x in xrange(10000000):
a.append(x)

for x in a:
a[x] += 1

################ two.py ##############
a = []

for x in xrange(10000000):
a.append(x)

for x in a:
a[x] += 1

######################################


ktops:tobyytest> time python one.py; time python two.py

real 0m28.116s
user 0m17.504s
sys 0m10.435s

real 0m23.026s
user 0m13.027s
sys 0m9.777s


Perhaps the only advantage is that they take less memory
to store a large number of items? It would seem then, that
'economical' might have been a better choice of word than
'efficient'.

Thanks,

Toby

 

[Arnaud Delobelle]

I checked out the array module today. It claims that
arrays are 'efficient'. I figured that this must mean
that they are faster than lists, but this doesn't seem
to be the case:

################ one.py ##############
import array

a = array.array('i')

for x in xrange(10000000):
a.append(x)

for x in a:
a[x] += 1

################ two.py ##############
a = []

for x in xrange(10000000):
a.append(x)

for x in a:
a[x] += 1

######################################


ktops:tobyytest> time python one.py; time python two.py

real 0m28.116s
user 0m17.504s
sys 0m10.435s

real 0m23.026s
user 0m13.027s
sys 0m9.777s


Perhaps the only advantage is that they take less memory
to store a large number of items? It would seem then, that
'economical' might have been a better choice of word than
'efficient'.

I get an even bigger difference with this test (same as yours, but
using timeit and using an allegedly more efficient way of initialising
the array)
.... a = arr(xrange(n))
.... for x in a:
.... a[x] += 1
....

 

[bearophileHUGS]

Their efficiency is mostly regarding the space. I think they aren't
much speed-efficient because they require many conversions from-to
Python types.
You can gain speed efficiency too (sometimes a LOT), in some
situations, using array with Psyco.
Another advantage of arrays (better called "vector"s, probably, so the
name "array" can replace the "list" name used by the built in) is that
they offer you a fixed size representation, so you know what you are
working with.
You can also take a look at the C version of the BList from
cheeseshop, the autor has made them rather efficient for some kinds of
operations.

Bye,
bearophile

 

[sturlamolden]

I checked out the array module today. It claims that
arrays are 'efficient'. I figured that this must mean
that they are faster than lists, but this doesn't seem
to be the case:

################ one.py ##############
import array

a = array.array('i')

for x in xrange(10000000):
a.append(x)

Lists are better optimized for appending to the end. Python lists are
implemented as arrays of pointers, with a few empty slots at the
end.

Arrays are contigous memory buffers. They provide faster read-write
access, particularly for chunks of data, but are slower at resizing.

I never use the array module, as NumPy is superior.

 

[Diez B. Roggisch]

Lists are better optimized for appending to the end. Python lists are
implemented as arrays of pointers, with a few empty slots at the
end.

Arrays are contigous memory buffers. They provide faster read-write
access, particularly for chunks of data, but are slower at resizing.

I doubt that. AFAIK both arrays and lists are continuous memory-areas,
that double (at least to a certain threshold or so) when reaching the
capacity limit.

lists are of type PyObject* of course, whereas arrays are not, instead
they are of their respective primitive type, making them more memory
efficient.

Diez

 

[Piet Delport]

I doubt that. AFAIK both arrays and lists are continuous memory-areas,
that double (at least to a certain threshold or so) when reaching the
capacity limit.

For what it's worth, lists over-allocate by ~1/8, and arrays by ~1/16.
(Details in listobject.c:list_resize and arraymodule.c:array_resize.)