Re: perl hash utilities

Discussion in 'Perl Misc' started by Rainer Weikusat, Oct 4, 2013.

  1. Ben Morrow <> writes:
    > Quoth Cal Dershowitz <>:
    >>
    >> and as long as the $ref itself or a copy of $ref exists, the data is
    >> available. If the references go undefined then all bets are off. I'm
    >> not sure if I understood that section of the alpaca book.

    >
    > It's very simple (or at least, it's very simple as long as you don't
    > involve circular or weak refs). A data structure continues to exist for
    > as long as someone can get to it. As soon as it becomes completely
    > invisible, it self-destructs.


    This is based on the same wrong-headed "we don't want to do this now,
    some student can do this later" 'everything is an IBM 704 running batch
    jobs' assumption as a so-called tracing garbage collector (this is not
    quite true, one can argue that the truth is clearly visible in this
    statement to everyone who already knows it).

    perl provides automatic resource management based on reference
    counting. This means every Perl object has a counter associated with it
    and this counter is incremented whenever a new reference to the object
    is created, eg,

    [rw@sable]~#perl -MDevel::peek -e '$x = []; Dump($x); $y = $x; Dump($x); $x = undef; Dump($y);'
    SV = RV(0x8195ca4) at 0x8195c98
    REFCNT = 1
    FLAGS = (ROK)
    RV = 0x817b818
    SV = PVAV(0x817c880) at 0x817b818
    REFCNT = 1
    FLAGS = ()
    ARRAY = 0x0
    FILL = -1
    MAX = -1
    ARYLEN = 0x0
    FLAGS = (REAL)
    SV = RV(0x8195ca4) at 0x8195c98
    REFCNT = 1
    FLAGS = (ROK)
    RV = 0x817b818
    SV = PVAV(0x817c880) at 0x817b818
    REFCNT = 2
    FLAGS = ()
    ARRAY = 0x0
    FILL = -1
    MAX = -1
    ARYLEN = 0x0
    FLAGS = (REAL)
    SV = RV(0x8195d24) at 0x8195d18
    REFCNT = 1
    FLAGS = (ROK)
    RV = 0x817b818
    SV = PVAV(0x817c880) at 0x817b818
    REFCNT = 1
    FLAGS = ()
    ARRAY = 0x0
    FILL = -1
    MAX = -1
    ARYLEN = 0x0
    FLAGS = (REAL)

    In this example, the anonymous array created with [] is the
    object. That's the SV = PVAV (array value) thing in the output.
    REFCNT is the reference count of the AV.

    As can be seen above, the counter is decremented when a reference to the
    object is destroyed, as in the $x = undef in the executed code. Once
    the counter value drops to zero, the object is destroyed. The obvious
    drawback of this scheme is that it can't handle so-called 'circular
    references' on its own. Example:

    ---------------
    use Devel::peek;

    {
    my $x;
    $x->[0] = \$x;
    Dump($x);
    }
    ---------------

    The output of that is

    SV = RV(0x8195cac) at 0x8195ca0
    REFCNT = 2
    FLAGS = (PADMY,ROK)
    RV = 0x817b968
    SV = PVAV(0x817c86c) at 0x817b968
    REFCNT = 1
    FLAGS = ()
    ARRAY = 0x81906b0
    FILL = 0
    MAX = 3
    ARYLEN = 0x0
    FLAGS = (REAL)
    Elt No. 0
    SV = RV(0x817b704) at 0x817b6f8
    REFCNT = 1
    FLAGS = (ROK)
    RV = 0x8195ca0
    SV = RV(0x8195cac) at 0x8195ca0
    REFCNT = 2
    FLAGS = (PADMY,ROK)
    RV = 0x817b968
    SV = PVAV(0x817c86c) at 0x817b968
    REFCNT = 1
    FLAGS = ()
    ARRAY = 0x81906b0
    FILL = 0
    MAX = 3
    ARYLEN = 0x0
    FLAGS = (REAL)

    The outermost object is an RV (reference value), that's what's directly
    referred to be $x. As can be seen, the reference count of this RV is
    2. As in the previous example, the RV refers to an AV with a reference
    count of one. Inside the AV, there's a reference to the same RV object
    which points to the AV. That's why the reference count of the RV is 2.
    When $x goes out of scope after the block, the reference count of the RV
    is decremented. Because of the additional reference to that in the
    array, it will be 1 and not 0 afterwards, hence, the object won't be
    freed. Because of this, the reference count of the array is never
    decremented and it continues to exist. But the only way to access this
    array was by going through $x which doesn't exist anymore, hence, the AV
    containing the RV will sit inaccessible in the memory of the perl
    interpreter until that exits, causeing a so-called 'memory leak'.

    In order to deal with that, Perl also supports so-called 'weak
    references'. These can be created with the Scalar::Util::weaken
    routine. Example:

    -------------
    use Devel::peek;
    use Scalar::Util qw(weaken);

    {
    my ($x, $y);

    $x->[0] = \$y;
    $y = $x;
    weaken($y);

    Dump($x);

    print STDERR ("\n========\n");

    $x = undef;

    Dump($y);
    }
    --------------

    I'm not going to post the output of that because it is too lengthy. For
    a weak reference the owner/ object relationship is inverted: While an
    ordinary reference owns the object it refers to (and hence, causes its
    reference count to be incremented), weak references are owned by the
    object they refer to whose reference count is not incremented (or, more
    correctly, is decremented as side-effect of weaken) because of them. If
    the reference count of an object owning weak references goes to zero,
    the weak reference are cleared. That's why $y has no value after $x was
    set to undef.

    It is important to understand that 'weakness' is an attribute of the RV
    and that it is not transitive. The following code sequence

    --------------
    use Scalar::Util qw(weaken)

    my ($x, $y, $z);

    $x = [];
    $y = $x;
    weaken($y);
    $z = $y
    -------------

    does not cause the $z RV to be a weak reference as well.

    NB: I could continue this with elaborating about the relative merits and
    historical origins of both schemes for automatic resource management but
    this text is too long already.
     
    Rainer Weikusat, Oct 4, 2013
    #1
    1. Advertising

  2. Ben Morrow <> writes:
    > Quoth Rainer Weikusat <>:
    >> Ben Morrow <> writes:
    >> > Quoth Cal Dershowitz <>:
    >> >>
    >> >> and as long as the $ref itself or a copy of $ref exists, the data is
    >> >> available. If the references go undefined then all bets are off. I'm
    >> >> not sure if I understood that section of the alpaca book.
    >> >
    >> > It's very simple (or at least, it's very simple as long as you don't
    >> > involve circular or weak refs). A data structure continues to exist for
    >> > as long as someone can get to it. As soon as it becomes completely
    >> > invisible, it self-destructs.

    >>
    >> This is based on the same wrong-headed "we don't want to do this now,
    >> some student can do this later" 'everything is an IBM 704 running batch
    >> jobs' assumption as a so-called tracing garbage collector (this is not
    >> quite true, one can argue that the truth is clearly visible in this
    >> statement to everyone who already knows it).

    >
    > What on Earth?
    >
    > Do you have a 'gratuitous-obscure-insult-a-day' calendar, or
    > something?


    Trying to sum this up briefly: As can be seen in this text,

    http://www-formal.stanford.edu/jmc/history/lisp/node3.html

    the inventors of LISP originally considered using reference counting for
    automatic memory management but they chose against it for two reasons:

    1) Because of constraints imposed by both the IBM 704 hardware and the
    code which had been written so far, there was no convenient way to store
    the counters.

    2) By adopting a "well, we didn't run out of memory yet so let's wait
    until we do" approach, they could postpone dealing with the problem
    until 'some later time' ("[...] only toy examples were being done").

    The idea that, once all memory has been allocated, the set of all
    allocated ojects can be partitioned into a 'live set' and a 'dead set'
    by determining whether an object is reachable according to the
    information available in the memory of the machine, with the 'dead set'
    becoming the free store list afterwards, relies on the fact that all
    memory can actually be examined while its contents don't change, which
    means the system must be a single-tasking system, so that stopping the
    'current tasks' means 'all activity stops', the single task must
    actually be able to examine all storage locations, ie there isn't such a
    thing as a higher-privileged kernel whose memory is protected from
    user-mode access, and there must not be any realtime requirements, not
    even a single user using a single computer interactively, so that
    'stopping all activity' will delay the final result but won't change it.

    All of this ceased to be true about 40 years ago (simplification). But
    why abandon an ill-conceived makeshift approach partially chosen because
    of researcher laziness just because it is also technically completely
    outdated?

    > [snip excessive technical detail I already know]
    >
    > How exactly is that meant to help someone who's having trouble
    > understanding the lifetime of Perl data structures?


    Ideally, by avoiding diffuse and subtly wrong analogies like "someone
    can get to it" or "it is visible" in favour of explaining how the thing
    works, including its obvious limitation and the workarounds for that.
     
    Rainer Weikusat, Oct 4, 2013
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. rp
    Replies:
    1
    Views:
    537
    red floyd
    Nov 10, 2011
  2. JWhite

    Perl CGI utilities?

    JWhite, Jul 1, 2008, in forum: Perl Misc
    Replies:
    5
    Views:
    294
    Todd Wade
    Jul 5, 2008
  3. Ben Bacarisse

    Re: perl hash utilities

    Ben Bacarisse, Sep 26, 2013, in forum: Perl Misc
    Replies:
    1
    Views:
    170
    Ben Bacarisse
    Sep 26, 2013
  4. Uri Guttman

    Re: perl hash utilities

    Uri Guttman, Sep 26, 2013, in forum: Perl Misc
    Replies:
    2
    Views:
    178
    Charles DeRykus
    Sep 27, 2013
  5. George Mpouras

    Re: perl hash utilities

    George Mpouras, Sep 26, 2013, in forum: Perl Misc
    Replies:
    9
    Views:
    218
    John W. Krahn
    Oct 6, 2013
Loading...

Share This Page