Pointers/References in Python?

Discussion in 'Python' started by boblatest@googlemail.com, Jul 30, 2008.

  1. Guest

    Hello,

    I have a long list of memory-heavy objects that I would like to access
    in differently sorted order. Let's say I'd like to have lists called
    by_date or by_size that I can use to access the objects in the
    specified order.

    Of course I can just build those lists naively by creating copies of
    the original list and then sorting them according to my wishes. But
    that would create huge memory overhead. Of course I could use lists of
    indices into the "master" list, just as in C I'd create lists or
    arrays of pointers into the original data.

    Is there a clever Python way to do this, or should I just use lists of
    indices?

    I know there is a thing called "shallow copy" that has something to do
    with not duplicating memory content but I don't understand the
    concept. Maybe that's what would help here, too.

    Thanks,
    robert
     
    , Jul 30, 2008
    #1
    1. Advertising

  2. Guest

    boblatest:
    > I have a long list of memory-heavy objects that I would like to access
    > in differently sorted order. Let's say I'd like to have lists called
    > by_date or by_size that I can use to access the objects in the
    > specified order.


    Just create a new list with a different sorting order, for example
    using:

    from operator import attrgetter
    by_date = sorted(input_data, key=attrgetter("date"))
    by_size = sorted(input_data, key=attrgetter("size"))

    Python doesn't copy by value by default, so you end having just a
    light list of references.

    To understand the situation better, there's a FAQ about how Python
    associates names to things.

    Bye,
    bearophile
     
    , Jul 30, 2008
    #2
    1. Advertising

  3. Le Wednesday 30 July 2008 16:46:37 , vous avez écrit :
    > Hello,
    >
    > I have a long list of memory-heavy objects that I would like to access
    > in differently sorted order. Let's say I'd like to have lists called
    > by_date or by_size that I can use to access the objects in the
    > specified order.
    >

    ...
    > I know there is a thing called "shallow copy" that has something to do
    > with not duplicating memory content but I don't understand the
    > concept. Maybe that's what would help here, too.
    >


    In fact, all containers in python contains references of objects.

    >>>[54]: a = [1]


    >>>[55]: b = [a]


    >>>[56]: c = list(sorted(b))


    >>>[57]: b, c

    ...[57]: ([[1]], [[1]])

    >>>[58]: b.append(3)


    >>>[59]: b, c

    ...[59]: ([[1], 3], [[1]])

    >>>[60]: a.append(0)


    >>>[61]: b, c

    ...[61]: ([[1, 0], 3], [[1, 0]])

    This is if you want to make a true copy (called deep copy) that you'll have to
    do extra steps (using copy module for example).

    --
    _____________

    Maric Michaud
     
    Maric Michaud, Jul 30, 2008
    #3
  4. Gary Herron Guest

    wrote:
    > Hello,
    >
    > I have a long list of memory-heavy objects that I would like to access
    > in differently sorted order. Let's say I'd like to have lists called
    > by_date or by_size that I can use to access the objects in the
    > specified order.
    >
    > Of course I can just build those lists naively by creating copies of
    > the original list and then sorting them according to my wishes. But
    > that would create huge memory overhead. Of course I could use lists of
    > indices into the "master" list, just as in C I'd create lists or
    > arrays of pointers into the original data.
    >
    > Is there a clever Python way to do this, or should I just use lists of
    > indices?
    >


    No need. A Python list contains *references* to objects, not copies of
    objects. (The same is true of variables, dictionaries, sets, and so
    on...).

    For example, in the following code, only one copy of HeavyObject
    exists, however, it is referred to many times.

    a = HeavyObject()
    b = a
    A = [a,b]
    B = [b,a]
    C = set([a,b])
    D = {1:a, 2:b}
    .... and do on


    Implementation wise, a long list consumes about 4 bytes per list element
    (that's one address per), plus a tine amount of overhead.


    Gary Herron


    > I know there is a thing called "shallow copy" that has something to do
    > with not duplicating memory content but I don't understand the
    > concept. Maybe that's what would help here, too.
    >
    > Thanks,
    > robert
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >
     
    Gary Herron, Jul 30, 2008
    #4
  5. Terry Reedy Guest

    wrote:
    > Hello,
    >
    > I have a long list of memory-heavy objects that I would like to access
    > in differently sorted order. Let's say I'd like to have lists called
    > by_date or by_size that I can use to access the objects in the
    > specified order.
    >
    > Of course I can just build those lists naively by creating copies of
    > the original list and then sorting them according to my wishes. But
    > that would create huge memory overhead. Of course I could use lists of
    > indices into the "master" list, just as in C I'd create lists or
    > arrays of pointers into the original data.
    >
    > Is there a clever Python way to do this, or should I just use lists of
    > indices?
    >
    > I know there is a thing called "shallow copy" that has something to do
    > with not duplicating memory content but I don't understand the
    > concept. Maybe that's what would help here, too.


    A containers/collections contain/collect references to objects, not the
    objects themselves.

    lcopy = somelist[:] # or list(somelist)

    makes a shallow copy, which only copies the references.

    There are applications for lists of indexes, but they are fairly
    specialized.

    tjr
     
    Terry Reedy, Jul 30, 2008
    #5
  6. Gary Herron wrote:
    > No need. A Python list contains *references* to objects, not copies of
    > objects. (The same is true of variables, dictionaries, sets, and so
    > on...).


    Good to know. I just wanted to make sure before writing more code which in
    the end might not scale well.

    Thanks to all for the help!

    robert
     
    Robert Latest, Aug 1, 2008
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Roger Leigh
    Replies:
    8
    Views:
    473
    Karl Heinz Buchegger
    Nov 17, 2003
  2. Replies:
    3
    Views:
    484
    Victor Bazarov
    Nov 10, 2004
  3. DanielEKFA
    Replies:
    8
    Views:
    641
    DanielEKFA
    May 16, 2005
  4. Replies:
    8
    Views:
    760
    Bruno Desthuilliers
    Dec 12, 2006
  5. cerr

    pointers, pointers, pointers...

    cerr, Apr 7, 2011, in forum: C Programming
    Replies:
    12
    Views:
    731
Loading...

Share This Page