Dynamic or not?

[rishiyoor]

I'm trying to write a program that will find the distance between two
groups of points in space, which have cartesian co-ordinates X,Y and
Z.

I need to find the distances between each point in one group and every
point in the other group. So if group 1 has 6 points and group 2 had 8
points, I will calculate 6 x 8 = 48 distances. But I do not know the
number of points the user will want to use. It is typically between 50
and 500 in both groups combined, but may be more. Since the memory
required for the distances will be much larger than the points
themselves, I am wondering if I will have to allocate memory
dynamically or if there are easier ways of solving such problems in
Python.

Suggestions for amateur programmer will be appreciated.

 

[Steven D'Aprano]

I'm trying to write a program that will find the distance between two
groups of points in space, which have cartesian co-ordinates X,Y and Z.

I need to find the distances between each point in one group and every
point in the other group. So if group 1 has 6 points and group 2 had 8
points, I will calculate 6 x 8 = 48 distances. But I do not know the
number of points the user will want to use. It is typically between 50
and 500 in both groups combined, but may be more. Since the memory
required for the distances will be much larger than the points
themselves, I am wondering if I will have to allocate memory dynamically
or if there are easier ways of solving such problems in Python.

Writing in Python, you never need to manually allocate memory. Sometimes,
for especially complicated data structures, you need to take care about
_de_allocating memory, but that's rare.

In this case, you may not have enough memory to calculate all the
combinations at once, so you should consider an iterator-based solution.
Read up on generators and iterators.

points1 = [ (1.2, 3.4), (5.7, 9.2) ]
points2 = [ (-6.3, 0.0), (14.1, -7.8), (2.6, 12.8) ]

import math
def distance(p1, p2):
return math.hypot(p1[0]-p2[0], p1[1]-p2[1])

def distances(list1, list2):
"""Yield the distances from every pair of points."""
for pt1 in list1:
for pt2 in list2:
yield distance(pt1, pt2)


Now, if you want the distances one at a time you do this:
.... print d
....
8.23468275042
17.0836178838
9.50368349641
15.1208465371
18.9620673978
4.75078940809


and if you want them all at once, you can do this:
[8.2346827504160718, 17.083617883809037, 9.5036834964133785,
15.120846537148639, 18.962067397834023, 4.7507894080878819]

 

[Dennis Lee Bieber]

I need to find the distances between each point in one group and every
point in the other group. So if group 1 has 6 points and group 2 had 8
points, I will calculate 6 x 8 = 48 distances. But I do not know the
number of points the user will want to use. It is typically between 50
and 500 in both groups combined, but may be more. Since the memory
required for the distances will be much larger than the points
themselves, I am wondering if I will have to allocate memory
dynamically or if there are easier ways of solving such problems in
Python.

I didn't know Python had any static memory allocation <G>

Let's see... Say a worst case of 500x500 points. I'm not going to
dig up the specs on storage, but let's say a float take 16bytes (8byte
double precision value, 4byte reference count, 4byte other over head) so
(500+500) * 3 * 16, plus some overhead for making tuples of each
x/y/z... (500+500) * 8... Input data is a mere 56Kbytes. Output is
(500*500)*16 (and do you need to keep the output in memory, or can you
write them to a file as generated?) is a mere 4MB... My cellphone has
more memory than that!
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Tim Roberts]

I'm trying to write a program that will find the distance between two
groups of points in space, which have cartesian co-ordinates X,Y and
Z.

I need to find the distances between each point in one group and every
point in the other group. So if group 1 has 6 points and group 2 had 8
points, I will calculate 6 x 8 = 48 distances. But I do not know the
number of points the user will want to use. It is typically between 50
and 500 in both groups combined, but may be more. Since the memory
required for the distances will be much larger than the points
themselves, I am wondering if I will have to allocate memory
dynamically or if there are easier ways of solving such problems in
Python.

Python already allocates memory dynamically. With 500 in each group,
you'll have 250,000 distances. If you use floating point, that's only 2
megabytes. Piece of cake. With 1,000 in each group, it's 8 megabytes.
Still no sweat.

What are you going to do with these distances? Why do you need them all in
memory?

 

[John Machin]

I'm trying to write a program that will find the distance between two
groups of points in space, which have cartesian co-ordinates X,Y and Z.

points1 = [ (1.2, 3.4), (5.7, 9.2) ]
points2 = [ (-6.3, 0.0), (14.1, -7.8), (2.6, 12.8) ]

import math
def distance(p1, p2):
return math.hypot(p1[0]-p2[0], p1[1]-p2[1])

X, Y, .... umm, aren't we short a dimension?

 

[Steven D'Aprano]

X, Y, .... umm, aren't we short a dimension?

I'm not going to do *everything* for the OP. He can extend it to three
dimensions.

 

[rishiyoor]

I'm trying to write a program that will find the distance between two
groups of points in space, which have cartesian co-ordinates X,Y and Z.

I need to find the distances between each point in one group and every
point in the other group. So if group 1 has 6 points and group 2 had 8
points, I will calculate 6 x 8 = 48 distances. But I do not know the
number of points the user will want to use. It is typically between 50
and 500 in both groups combined, but may be more. Since the memory
required for the distances will be much larger than the points
themselves, I am wondering if I will have to allocate memory dynamically
or if there are easier ways of solving such problems in Python.

Writing in Python, you never need to manually allocate memory. Sometimes,
for especially complicated data structures, you need to take care about
_de_allocating memory, but that's rare.

In this case, you may not have enough memory to calculate all the
combinations at once, so you should consider an iterator-based solution.
Read up on generators and iterators.

points1 = [ (1.2, 3.4), (5.7, 9.2) ]
points2 = [ (-6.3, 0.0), (14.1, -7.8), (2.6, 12.8) ]

import math
def distance(p1, p2):
return math.hypot(p1[0]-p2[0], p1[1]-p2[1])

def distances(list1, list2):
"""Yield the distances from every pair of points."""
for pt1 in list1:
for pt2 in list2:
yield distance(pt1, pt2)

Now, if you want the distances one at a time you do this:

... print d
...
8.23468275042
17.0836178838
9.50368349641
15.1208465371
18.9620673978
4.75078940809

and if you want them all at once, you can do this:

[8.2346827504160718, 17.083617883809037, 9.5036834964133785,
15.120846537148639, 18.962067397834023, 4.7507894080878819]

Thanks, that helps.

When you say python automatically allocates memory, what would you do
if you don't know the size of the list of, say for example, the
nearest pairs between the two groups. I would probably iterate over
all the pairs and create a new list. I did a similar operation in
another program, but I had to initialize the list (statically) with x
= [0]*50 before I could use it in the for loop.

 

[Marc 'BlackJack' Rintsch]

When you say python automatically allocates memory, what would you do
if you don't know the size of the list of, say for example, the
nearest pairs between the two groups. I would probably iterate over
all the pairs and create a new list. I did a similar operation in
another program, but I had to initialize the list (statically) with x
= [0]*50 before I could use it in the for loop.

Only if you used an index instead of starting with an empty list and
appending values to it. Or in simple cases a list comprehension might
replace a ``for`` loop.

"Bad" and "unpythonic" example:

new = [0] * len(old)
for i in xrange(len(old)):
new = do_something(old)

Better written as:

new = list()
for item in old:
new.append(do_something(item))

Or as list comprehension:

new = [do_something(item) for item in old]

Or:

new = map(do_something, old)

Ciao,
Marc 'BlackJack' Rintsch

 

[Bruno Desthuilliers]

(e-mail address removed) a écrit :

On Dec 12, 11:33 pm, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.au> wrote: (snip)

When you say python automatically allocates memory, what would you do
if you don't know the size of the list

thelist = []
thelist.append('Ever')
thelist.append('bothered')
thelist.append('reading')
thelist.append('the')
thelist.append('fine')
thelist.append('manual')
thelist.append('?')

print thelist

of, say for example, the
nearest pairs between the two groups. I would probably iterate over
all the pairs and create a new list. I did a similar operation in
another program, but I had to initialize the list (statically) with x
= [0]*50 before I could use it in the for loop.

You'd be better learning how to properly use lists and iterations in
Python... While you're at it, add list comprehensions,
iterators/generators and itertools to the list (pun intented).

HTH

 

[rishiyoor]

When you say python automatically allocates memory, what would you do
if you don't know the size of the list of, say for example, the
nearest pairs between the two groups. I would probably iterate over
all the pairs and create a new list. I did a similar operation in
another program, but I had to initialize the list (statically) with x
= [0]*50 before I could use it in the for loop.

Only if you used an index instead of starting with an empty list and
appending values to it. Or in simple cases a list comprehension might
replace a ``for`` loop.

"Bad" and "unpythonic" example:

new = [0] * len(old)
for i in xrange(len(old)):
new = do_something(old)

Better written as:

new = list()
for item in old:
new.append(do_something(item))

Or as list comprehension:

new = [do_something(item) for item in old]

Or:

new = map(do_something, old)

Ciao,
Marc 'BlackJack' Rintsch

Thanks Marc.

 

[John Machin]

On Dec 14, 2:29 am, Bruno Desthuilliers While you're at it, add list
comprehensions,

iterators/generators and itertools to the list (pun intented).

'Intented' is a novel word; is it the opposite of 'decamped'?