Please explain the meaning of 'stealing' a ref

Discussion in 'Python' started by Christos TZOTZIOY Georgiou, Nov 4, 2003.

  1. I invested some time today because I got this crazy idea that if I
    implemented a simple attribute-getting caching scheme implemented in
    PyObject_GetAttr / PyObject_SetAttr that would turn out to be good.

    So I did implement one touching Objects/object.c (and pythonrun.c for
    init/final purposes), and I got a 5% increase in pystone... of course,
    when I exited the interpreter, I got a nice segfault :) Also,
    test.testall segfaulted at test_anydbm. I didn't expect anything
    better, to be honest; it's obviously got something to do with inc/dec
    refs.

    Now I'm googling for the various debug building options (I'm not sure
    that -DDEBUG is enough etc), however I realised it's time to ask what I
    never understood so far: what exactly is the meaning of a function
    "stealing a reference"? Is it that the function does an incref or a
    decref? Or is it something more obscure?

    TIA for any answers.

    PS Do we have still time to help Guido be the one that throws the pie?-)
    --
    TZOTZIOY, I speak England very best,
    Ils sont fous ces Redmontains! --Harddix
     
    Christos TZOTZIOY Georgiou, Nov 4, 2003
    #1
    1. Advertising

  2. Christos TZOTZIOY Georgiou

    KefX Guest

    > what exactly is the meaning of a function
    >"stealing a reference"?


    It decrements the reference count. If you still need the reference, Py_INCREF()
    it beforehand.

    - Kef
     
    KefX, Nov 4, 2003
    #2
    1. Advertising

  3. Edward C. Jones, Nov 4, 2003
    #3
  4. Christos "TZOTZIOY" Georgiou <> writes:

    > I invested some time today because I got this crazy idea that if I
    > implemented a simple attribute-getting caching scheme implemented in
    > PyObject_GetAttr / PyObject_SetAttr that would turn out to be good.


    Are you aware of the cache-attr-branch (think it's called that) in
    CVS?

    > So I did implement one touching Objects/object.c (and pythonrun.c for
    > init/final purposes), and I got a 5% increase in pystone... of course,
    > when I exited the interpreter, I got a nice segfault :) Also,
    > test.testall segfaulted at test_anydbm. I didn't expect anything
    > better, to be honest; it's obviously got something to do with inc/dec
    > refs.
    >
    > Now I'm googling for the various debug building options (I'm not sure
    > that -DDEBUG is enough etc), however I realised it's time to ask what I
    > never understood so far: what exactly is the meaning of a function
    > "stealing a reference"? Is it that the function does an incref or a
    > decref? Or is it something more obscure?


    This *really* should be explain in the API reference or the extended
    and embedding manual somewhere... have you looked there?

    Cheers,
    mwh

    --
    There are two ways of constructing a software design: one way is to
    make it so simple that there are obviously no deficiencies and the
    other way is to make it so complicated that there are no obvious
    deficiencies. -- C. A. R. Hoare
     
    Michael Hudson, Nov 4, 2003
    #4
  5. On Tue, 4 Nov 2003 12:32:22 GMT, rumours say that Michael Hudson
    <> might have written:

    >Christos "TZOTZIOY" Georgiou <> writes:
    >

    [my attempting to implement attribute caching]
    >
    >Are you aware of the cache-attr-branch (think it's called that) in
    >CVS?


    <sigh> Obviously, not :(

    [my question about what "stealing means"]
    >
    >This *really* should be explain in the API reference or the extended
    >and embedding manual somewhere... have you looked there?


    Oh, definitely, ext/refcountsInPython.html is the page I believe. I can
    understand the second paragraph, but I wanted some reasoning. For
    example, you call a function that its API specifies that it "steals" a
    reference to its argument; therefore, you got to incref in advance and
    decref afterwards yourself. What's the reason? Efficiency for
    simplicity of the function? If yes, why not enclose the function call
    in a incref / decref cycle and then export the enclosing function in the
    API?

    Such stuff I wanted to know.

    Also, "borrowing" and "stealing" are the same thing? I just think that
    "beautifying" terminology at the C level is more confusing than helpful.

    PS Thanks KefX and Edward for your replies. I'll check the branch.
    --
    TZOTZIOY, I speak England very best,
    Ils sont fous ces Redmontains! --Harddix
     
    Christos TZOTZIOY Georgiou, Nov 4, 2003
    #5
  6. Christos TZOTZIOY Georgiou wrote:
    ...
    > Oh, definitely, ext/refcountsInPython.html is the page I believe. I can
    > understand the second paragraph, but I wanted some reasoning. For
    > example, you call a function that its API specifies that it "steals" a
    > reference to its argument; therefore, you got to incref in advance and


    Yes, you have to incref IF you want to keep owning that reference.

    > decref afterwards yourself. What's the reason? Efficiency for


    You'll decref when you don't want to own that reference any more,
    and you incref'd it, but if you do that soon, why bother incref'ing?-)

    The typical use case to explain the stealing is in the "Reference
    Count Details" section of the API manual:

    PyObject *t;

    t = PyTuple_New(3);
    PyTuple_SetItem(t, 0, PyInt_FromLong(1L));
    PyTuple_SetItem(t, 1, PyInt_FromLong(2L));
    PyTuple_SetItem(t, 2, PyString_FromString("three"));


    PyTuple_SetItem is very often called with a completely new reference
    as its 3d argument, so it steals it to enable this easy idiom... w/o
    the stealing you'd have to code, e.g.:

    PyObject *t;
    PyObject *temp;

    t = PyTuple_New(3);

    temp = PyInt_FromLong(1L);
    PyTuple_SetItem(t, 0, temp);
    Py_DECREF(temp);

    etc -- much less handy.


    > simplicity of the function? If yes, why not enclose the function call
    > in a incref / decref cycle and then export the enclosing function in the
    > API?


    I don't know what you're talking about. Explain please? E.g. with one
    of the few reference-stealing functions, PyTuple_SetItem ?


    > Such stuff I wanted to know.
    >
    > Also, "borrowing" and "stealing" are the same thing? I just think that


    No! You get a borrowed reference when you get a reference but are not
    transferred the ownership of it. That's a pretty widespread use regarding
    references that are returned to objects that surely pre-existed. You
    must then incref in the relatively rare case you want to keep hold of
    that reference for the longer term.

    > "beautifying" terminology at the C level is more confusing than helpful.


    What 'beautifying'...?


    Alex
     
    Alex Martelli, Nov 4, 2003
    #6
  7. KefX wrote:

    >> what exactly is the meaning of a function
    >>"stealing a reference"?

    >
    > It decrements the reference count.


    *NO*! It _takes over ownersip_: it DOESN'T decrement the
    refcount at that point; rather, it takes over the future
    responsibility for decrementing it at some point in time,
    i.e., the abstract "ownership" of the reference. See my
    other post in this thread for the classic example from Python's
    online API docs ... if PyTuple_SetItem _decremented the
    reference_ to the newly created PyObject* it's passed, how
    WOULD it fill the tuple's item?-)

    > If you still need the reference,
    > Py_INCREF() it beforehand.


    If you still need to OWN the reference, yes. If you just
    need to USE it for a while during which you're sure the
    tuple that's now "owning" it isn't going away, no need to
    incref-then-decref.


    Alex
     
    Alex Martelli, Nov 4, 2003
    #7
  8. On Tue, 04 Nov 2003 14:20:18 GMT, rumours say that Alex Martelli
    <> might have written:

    I want to clarify that all the questions in my first post and in this
    post are intended to help me understand the notions of "stealing a
    reference", "borrowing a reference" because I found
    api/refcountDetails.html a little unsatisfying.

    [snip: my referring to ext/refcountsInPython.html and supposing that an
    incref should be issued before calling a function "stealing" a
    reference]

    >Yes, you have to incref IF you want to keep owning that reference.


    So, "owning a reference" means simply that some code did an incref and
    is responsible to decref, right? This is my first request for
    clarification, although I understand that so far I was confusing "owning
    a reference" and "owning an object".

    >> decref afterwards yourself. What's the reason? Efficiency for

    >
    >You'll decref when you don't want to own that reference any more,
    >and you incref'd it, but if you do that soon, why bother incref'ing?-)


    I'd do that because I did not understand what "stealing a reference"
    means; therefore I translated "stealing" = decref, so I presumed that I
    should issue an incref in advance in order to make sure the object
    passed to the function does not cease to exist before the function
    returns. I assumed based on unclear (to me) terms.

    >The typical use case to explain the stealing is in the "Reference
    >Count Details" section of the API manual:
    >
    >PyObject *t;
    >
    >t = PyTuple_New(3);
    >PyTuple_SetItem(t, 0, PyInt_FromLong(1L));
    >PyTuple_SetItem(t, 1, PyInt_FromLong(2L));
    >PyTuple_SetItem(t, 2, PyString_FromString("three"));
    >
    >PyTuple_SetItem is very often called with a completely new reference
    >as its 3d argument, so it steals it to enable this easy idiom... w/o
    >the stealing you'd have to code, e.g.:
    >
    >PyObject *t;
    >PyObject *temp;
    >
    >t = PyTuple_New(3);
    >
    >temp = PyInt_FromLong(1L);
    >PyTuple_SetItem(t, 0, temp);
    >Py_DECREF(temp);
    >
    >etc -- much less handy.


    I understand that. So, a function "stealing" a reference means the
    function takes away from the caller the responsibility for decref'ing
    the object, and "stealing a reference" does not translate into some C
    code, right? This is my second request for clarification.

    >> simplicity of the function? If yes, why not enclose the function call
    >> in a incref / decref cycle and then export the enclosing function in the
    >> API?

    >
    >I don't know what you're talking about. Explain please? E.g. with one
    >of the few reference-stealing functions, PyTuple_SetItem ?


    You don't know what I'm talking about because neither I was sure what I
    was talking about; I now see I should not make assumptions since I was
    not sure as to what "stealing a ref" means. It was confusing to the
    readers of my post.

    >> Such stuff I wanted to know.
    >>
    >> Also, "borrowing" and "stealing" are the same thing? I just think that

    >
    >No! You get a borrowed reference when you get a reference but are not
    >transferred the ownership of it. That's a pretty widespread use regarding
    >references that are returned to objects that surely pre-existed. You
    >must then incref in the relatively rare case you want to keep hold of
    >that reference for the longer term.


    So a function "borrowing" a reference means that the function does not
    incref or decref an object which was passed to it, right? That is the
    third request.
    I assume that most functions of the python API "borrow" references?

    >> "beautifying" terminology at the C level is more confusing than helpful.

    >
    >What 'beautifying'...?


    "Beautifying" in the sense that I thought that both "stealing" and
    "borrowing" a reference translate into actual C code, a mistaken
    assumption.

    Thanks for your time.
    --
    TZOTZIOY, I speak England very best,
    Ils sont fous ces Redmontains! --Harddix
     
    Christos TZOTZIOY Georgiou, Nov 5, 2003
    #8
  9. Christos TZOTZIOY Georgiou wrote:
    ...
    >>Yes, you have to incref IF you want to keep owning that reference.

    >
    > So, "owning a reference" means simply that some code did an incref and
    > is responsible to decref, right? This is my first request for


    "Owning a reference" means "I'll dispose of that reference when I'm
    done with it". As per:

    http://www.python.org/doc/current/api/refcountDetails.html

    """
    The reference count behavior of functions in the Python/C API is best
    explained in terms of ownership of references. Note that we talk of owning
    references, never of owning objects; objects are always shared! When a
    function owns a reference, it has to dispose of it properly -- either by
    passing ownership on (usually to its caller) or by calling Py_DECREF() or
    Py_XDECREF(). When a function passes ownership of a reference on to its
    caller, the caller is said to receive a new reference. When no ownership is
    transferred, the caller is said to borrow the reference. Nothing needs to
    be done for a borrowed reference.
    """

    I really don't know how we could have expressed this any more clearly;
    if you can indicate what is hard for you to understand here, and how
    we could make it easier, I'd be really greateful!

    > clarification, although I understand that so far I was confusing "owning
    > a reference" and "owning an object".


    Ditto: I don't know how we could try to dispell this common doubt
    any more clearly than by saying "we talk of owning references, never
    of owning objects; objects are always shared!".


    >>> decref afterwards yourself. What's the reason? Efficiency for

    >>
    >>You'll decref when you don't want to own that reference any more,
    >>and you incref'd it, but if you do that soon, why bother incref'ing?-)

    >
    > I'd do that because I did not understand what "stealing a reference"
    > means; therefore I translated "stealing" = decref, so I presumed that I
    > should issue an incref in advance in order to make sure the object
    > passed to the function does not cease to exist before the function
    > returns. I assumed based on unclear (to me) terms.


    Hmmm -- wasn't the example on the same URL I just quoted, right after
    the 2nd paragrap, clear enough in terms of no incref being needed?
    Can you suggest a way in which we could make it clearer to you?


    > I understand that. So, a function "stealing" a reference means the
    > function takes away from the caller the responsibility for decref'ing
    > the object, and "stealing a reference" does not translate into some C
    > code, right? This is my second request for clarification.


    Right. There is no specific C code to which you can point and say
    "see, right here, the reference is being stolen". If you looked at
    the source for e.g. PyTuple_SetItem you might notice it *doesn't*
    incref (nor of course decref) its 3rd argument -- it copies that
    PyObject* somewhere _and that's it_ -- contrasted with the "normal"
    case of e.g. PySequence_SetItem which copies the PyObject* s/where
    AND incref's it. So the "stealing" is the copying-somewhere plus
    the LACK of the incref that would normally go with copying, in terms
    of C source.


    >>> Also, "borrowing" and "stealing" are the same thing? I just think that

    >>
    >>No! You get a borrowed reference when you get a reference but are not
    >>transferred the ownership of it. That's a pretty widespread use regarding
    >>references that are returned to objects that surely pre-existed. You
    >>must then incref in the relatively rare case you want to keep hold of
    >>that reference for the longer term.

    >
    > So a function "borrowing" a reference means that the function does not
    > incref or decref an object which was passed to it, right? That is the
    > third request.


    The function will incref the reference passed to if IF it's copying
    it somewhere (and will then arrange for that copy to be decref'd later
    when that reference is not needed any more). If the function is
    just using the reference and is done with it when it returns, then
    the function doesn't bother incref'ing it. The caller keeps ownership
    if it originally had it (i.e. unless it was borrowing from somewhere
    else, then ownership remains with whoever ultimately had it), there
    is no ownership transfer for that reference.

    > I assume that most functions of the python API "borrow" references?


    Yes, and so do all functions you'll be writing to be callable from
    Python -- it's the normal case.

    > Thanks for your time.


    You're welcome! Now I'd like, if possible, to enhance the docs so
    that other people's future doubts might be similarly helped -- that's
    the reason I'm asking you for advice on this!

    If you could spend some time to visit safari.oreilly.com -- subscribe
    and be sure to cancel before 2 weeks so you don't have to pay! -- and
    check the penultimate chapter of Python in a Nutshell, where I've
    tried an alternate "concise mixed reference/tutorial" tack to the
    whole issue of extending Python, I'd be particularly grateful of any
    feedback about that, too... thanks!


    Alex
     
    Alex Martelli, Nov 5, 2003
    #9
  10. On Wed, 05 Nov 2003 15:07:08 GMT, rumours say that Alex Martelli
    <> might have written:

    >Christos TZOTZIOY Georgiou wrote:
    > ...


    [snip]

    >"Owning a reference" means "I'll dispose of that reference when I'm
    >done with it". As per:
    >
    >http://www.python.org/doc/current/api/refcountDetails.html
    >
    >"""
    >The reference count behavior of functions in the Python/C API is best
    >explained in terms of ownership of references. Note that we talk of owning
    >references, never of owning objects; objects are always shared! When a
    >function owns a reference, it has to dispose of it properly -- either by
    >passing ownership on (usually to its caller) or by calling Py_DECREF() or
    >Py_XDECREF(). When a function passes ownership of a reference on to its
    >caller, the caller is said to receive a new reference. When no ownership is
    >transferred, the caller is said to borrow the reference. Nothing needs to
    >be done for a borrowed reference.
    >"""
    >
    >I really don't know how we could have expressed this any more clearly;
    >if you can indicate what is hard for you to understand here, and how
    >we could make it easier, I'd be really greateful!


    The missing part IMO is what "owning a reference" means. I understand
    that my brain moved in mysterious ways on the false assumption that
    "steal"/"borrow" translates to C code, yet I believe a definition of the
    term "own" should be inserted somewhere in a way like:

    "... objects are always shared! Every call to Py_INCREF creates a
    'reference' to the object (actually, it's a way to make 'official' to
    the python memory management system that you've got a pointer pointing
    to the object, so the object better stays where it is!) which reference
    is considered as 'owned' by the caller. This reference should be
    disposed of properly ..."

    I beieve that neither my english nor my brain is fit for documentation
    writing, but I also believe you will get the idea. Perhaps the text in
    the parenthesis needs a complete rewriting --or removal.
    The definition could be appended as a footnote, to avoid changing the
    current text flow.

    >> clarification, although I understand that so far I was confusing "owning
    >> a reference" and "owning an object".

    >
    >Ditto: I don't know how we could try to dispell this common doubt
    >any more clearly than by saying "we talk of owning references, never
    >of owning objects; objects are always shared!".


    This text is obviously clear, and the misconception was my fault: since
    references are just pointer variables at the C level, and I never had a
    concept of a function (or a program) 'owning' its variables,
    automatically my brain converted the word 'reference' to 'reference to
    python object' to 'python object'. That's it.

    [snip]

    >You're welcome! Now I'd like, if possible, to enhance the docs so
    >that other people's future doubts might be similarly helped -- that's
    >the reason I'm asking you for advice on this!


    I proposed a definition for "owning a reference".

    A slight clarification in the following text:
    """Few functions steal references; the two notable exceptions are
    PyList_SetItem()and PyTuple_SetItem()"""
    The word "exceptions" implies "exception to the previous statement".
    IIUC "exceptions" should be changed to "examples".

    Also, it would be helpful if (possibly as footnotes too) there was the
    following definition:
    "stealing a reference": a function that steals a reference takes away
    from the caller the responsibility of calling Py_DECREF on the
    referenced object.

    If all of the above seem redundant to you, then perhaps it would make
    life easier for future readers adding a link to ext/ownershipRules.html
    at the end of the first sentence of api/refcountDetails.html . (as in
    HTML lingo 'see also <A href="/ext/ownershipRules.html">Ownership
    Rules</A>')

    Borrowing and reference ownership are explained in 'Ownership Rules',
    which is in the Extending and Embedding tutorial; one tends to think
    that a reference manual should not depend on definitions in a
    tutorial...

    >If you could spend some time to visit safari.oreilly.com -- subscribe
    >and be sure to cancel before 2 weeks so you don't have to pay! -- and
    >check the penultimate chapter of Python in a Nutshell, where I've
    >tried an alternate "concise mixed reference/tutorial" tack to the
    >whole issue of extending Python, I'd be particularly grateful of any
    >feedback about that, too... thanks!


    Thanks for the hint, I'll try that sometime.
    --
    TZOTZIOY, I speak England very best,
    Ils sont fous ces Redmontains! --Harddix
     
    Christos TZOTZIOY Georgiou, Nov 5, 2003
    #10
  11. Christos TZOTZIOY Georgiou wrote:
    ...
    >>The reference count behavior of functions in the Python/C API is best
    >>explained in terms of ownership of references. Note that we talk of owning
    >>references, never of owning objects; objects are always shared! When a
    >>function owns a reference, it has to dispose of it properly -- either by

    ...
    >>I really don't know how we could have expressed this any more clearly;

    ...
    > The missing part IMO is what "owning a reference" means. I understand


    Hmmm... what if the text was transliterated into "Owning a reference
    means you have to dispose of it properly" from the current "When a
    function owns a reference, it has to dispose of it properly"? The current
    expression and my suggested rewording are meant to convey just the
    same meaning, but perhaps the rewording is more immediately obvious
    to some non-native speakers of English (I'm a non-native speaker myself,
    but sometimes I have trouble gauging what problems others may have).


    >>> clarification, although I understand that so far I was confusing "owning
    >>> a reference" and "owning an object".

    >>
    >>Ditto: I don't know how we could try to dispell this common doubt
    >>any more clearly than by saying "we talk of owning references, never
    >>of owning objects; objects are always shared!".

    >
    > This text is obviously clear, and the misconception was my fault: since
    > references are just pointer variables at the C level, and I never had a
    > concept of a function (or a program) 'owning' its variables,
    > automatically my brain converted the word 'reference' to 'reference to
    > python object' to 'python object'. That's it.


    Ah. So there may be no fix for this -- we can't get any more explicit,
    but if your brain converts this to "we talk of owning python objects,
    never of owning objects" there's really nothing we can do:-(.


    > A slight clarification in the following text:
    > """Few functions steal references; the two notable exceptions are
    > PyList_SetItem()and PyTuple_SetItem()"""
    > The word "exceptions" implies "exception to the previous statement".


    Yes, to the statement terminated by the semicolon - the one saying that few
    functions steal references.

    > IIUC "exceptions" should be changed to "examples".


    Yes, this is surely a reasonable change, if it clarifies the text for some
    readers.


    > Also, it would be helpful if (possibly as footnotes too) there was the
    > following definition:
    > "stealing a reference": a function that steals a reference takes away
    > from the caller the responsibility of calling Py_DECREF on the
    > referenced object.
    >
    > If all of the above seem redundant to you, then perhaps it would make


    Yes, totally -- and incomplete, because it doesn't really capture the
    _concept_ of ownership. E.g., if the called didn't own the reference
    but borrowed it, then to pass it to a function that steals it, the caller
    must Py_INCREF it -- now the "takes away the responsibility" can
    be considered a very misleading statement.

    > life easier for future readers adding a link to ext/ownershipRules.html
    > at the end of the first sentence of api/refcountDetails.html . (as in
    > HTML lingo 'see also <A href="/ext/ownershipRules.html">Ownership
    > Rules</A>')


    Yes, this can't hurt. I don't know how to set cross-book references in
    the Latex source for the docs, but I'm sure their official czar, Fred Drake,
    does know, so I'll ask him.


    > Borrowing and reference ownership are explained in 'Ownership Rules',
    > which is in the Extending and Embedding tutorial; one tends to think
    > that a reference manual should not depend on definitions in a
    > tutorial...


    It doesn't depend on the definitions, but it does depend on the knowledge
    imparted by that tutorial having been absorbed already. If "reference
    ownership" is still a totally alien concept when one starts on the
    reference manual, the latter's in trouble. Some pointers back to the
    tutorial presumably can't hurt, though.


    >>If you could spend some time to visit safari.oreilly.com -- subscribe
    >>and be sure to cancel before 2 weeks so you don't have to pay! -- and
    >>check the penultimate chapter of Python in a Nutshell, where I've
    >>tried an alternate "concise mixed reference/tutorial" tack to the
    >>whole issue of extending Python, I'd be particularly grateful of any
    >>feedback about that, too... thanks!

    >
    > Thanks for the hint, I'll try that sometime.


    Thanks -- and remember I'll appreciate the feedback.


    Alex
     
    Alex Martelli, Nov 5, 2003
    #11
  12. On Wed, 05 Nov 2003 20:24:39 GMT, rumours say that Alex Martelli
    <> might have written:

    >> The missing part IMO is what "owning a reference" means. I understand

    >
    >Hmmm... what if the text was transliterated into "Owning a reference
    >means you have to dispose of it properly" from the current "When a
    >function owns a reference, it has to dispose of it properly"? The current
    >expression and my suggested rewording are meant to convey just the
    >same meaning, but perhaps the rewording is more immediately obvious
    >to some non-native speakers of English (I'm a non-native speaker myself,
    >but sometimes I have trouble gauging what problems others may have).


    For other people (if any) that have similar to my questions, all the
    information about 'owning', 'borrowing' and 'stealing' references is in
    the sections 1.2.1-1.2.2 of the "Python/C API reference", but first have
    a good grasp of the tutorial "Extending and Embedding the Python
    interpreter", subsections 1.10.1-1.10.4 .


    >> This text is obviously clear, and the misconception was my fault: since
    >> references are just pointer variables at the C level, and I never had a
    >> concept of a function (or a program) 'owning' its variables,
    >> automatically my brain converted the word 'reference' to 'reference to
    >> python object' to 'python object'. That's it.

    >
    >Ah. So there may be no fix for this -- we can't get any more explicit,
    >but if your brain converts this to "we talk of owning python objects,
    >never of owning objects" there's really nothing we can do:-(.


    You use present tense for 'converts' above, but I assume you meant to
    use past tense, since this is what I did in the previous paragraph.
    Unusual of you to miss details. Like I said, the text is clear; it
    needs no fix in the case of owning references|objects.

    >> The word "exceptions" implies "exception to the previous statement".

    >
    >Yes, to the statement terminated by the semicolon - the one saying that few
    >functions steal references.
    >
    >> IIUC "exceptions" should be changed to "examples".

    >
    >Yes, this is surely a reasonable change, if it clarifies the text for some
    >readers.


    Since you agree (and make more specific) that 'exceptions' means
    'exception to the statement terminated by the semicolon', and since
    neither PyList_SetItem nor PyTuple_SetItem are exceptions to the few
    functions that steal references, doesn't that make the choice of the
    word 'exceptions' unreasonable (as in exactly the opposite of what was
    meant)? It's not a matter of simple clarification.

    Either the word 'exceptions' should be changed, or it should be changed
    to 'notable exceptions to <here specify the domain to which the two
    abovementioned functions are exceptions, and which domain is not "'few
    functions steal references", since the functions do steal references>'

    The previous two paragraphs were written only because your 'if it
    clarifies the text for some readers' seems to ignore the fact you
    yourself stated two paragraphs above in 'Yes, to the statement...'

    >> Also, it would be helpful if (possibly as footnotes too) there was the
    >> following definition:
    >> "stealing a reference": a function that steals a reference takes away
    >> from the caller the responsibility of calling Py_DECREF on the
    >> referenced object.
    >>
    >> If all of the above seem redundant to you, then perhaps it would make

    >
    >Yes, totally -- and incomplete, because it doesn't really capture the
    >_concept_ of ownership. E.g., if the called didn't own the reference
    >but borrowed it, then to pass it to a function that steals it, the caller
    >must Py_INCREF it -- now the "takes away the responsibility" can
    >be considered a very misleading statement.


    Yes, your example is a good one -as usual for a bot :)- to make my
    suggestion lacking.
    If the python documentation had a terminology dictionary, how would
    'stealing', 'borrowing' and 'owning' a reference be defined in a concise
    and absolutely clear way? I seem to not be able to write these
    definitions, although I would like to.

    Anyway, given time and inspiration, sf.net is just a few blocks down the
    street :)
    --
    TZOTZIOY, I speak England very best,
    Ils sont fous ces Redmontains! --Harddix
     
    Christos TZOTZIOY Georgiou, Nov 6, 2003
    #12
  13. Most of this discussion seems to be veering into English (and natural
    language in general) and style issues, farther and farther away from
    Python. Still, in as much as Python docs are written in English, I
    guess it might still be appropriate here.

    Like most authors, I have very strong opinions on style, nuances, fine
    points of connotation as well as denotation in phrasing, and language
    in general. Developed in long praxis as well as reflection about all
    of these issues, and honed in long, bitter fights with editors and
    co-authors, these opinions are unlikely to change very easily...;-)


    Christos TZOTZIOY Georgiou wrote:
    ...
    >>> The missing part IMO is what "owning a reference" means. I understand

    >>
    >>Hmmm... what if the text was transliterated into "Owning a reference
    >>means you have to dispose of it properly" from the current "When a
    >>function owns a reference, it has to dispose of it properly"? The current
    >>expression and my suggested rewording are meant to convey just the
    >>same meaning, but perhaps the rewording is more immediately obvious
    >>to some non-native speakers of English (I'm a non-native speaker myself,
    >>but sometimes I have trouble gauging what problems others may have).

    >
    > For other people (if any) that have similar to my questions, all the
    > information about 'owning', 'borrowing' and 'stealing' references is in
    > the sections 1.2.1-1.2.2 of the "Python/C API reference", but first have
    > a good grasp of the tutorial "Extending and Embedding the Python
    > interpreter", subsections 1.10.1-1.10.4 .


    So what about my proposed transliteration of "when X, Y" into "X
    means Y"? This is similar to (e.g.) changing "when somebody owns
    a property outright, they can sell it" to "owning a property
    outright means you can sell it" -- essentially equivalent in normal
    English usage, but perhaps more specific or clear to non-natives?


    >>> This text is obviously clear, and the misconception was my fault: since
    >>> references are just pointer variables at the C level, and I never had a
    >>> concept of a function (or a program) 'owning' its variables,
    >>> automatically my brain converted the word 'reference' to 'reference to
    >>> python object' to 'python object'. That's it.

    >>
    >>Ah. So there may be no fix for this -- we can't get any more explicit,
    >>but if your brain converts this to "we talk of owning python objects,
    >>never of owning objects" there's really nothing we can do:-(.

    >
    > You use present tense for 'converts' above, but I assume you meant to
    > use past tense, since this is what I did in the previous paragraph.
    > Unusual of you to miss details. Like I said, the text is clear; it
    > needs no fix in the case of owning references|objects.


    I don't think I missed any details: I was talking about making the docs
    clearer to _other_ (future) readers, as opposed to borrowing Guido's
    time machine to retroactively change them so they might be clearer in
    the past, so the "your brain converts" in my sentence is (clearly, in
    context) a typical case of "generic you"; feel free to mentally
    transliterate it to "if some generic and hypothetical reader's brain
    converts". Using the tense "converted" in such a sentence would mean
    one's having vague regrets about the unchangeable (save for the time
    machine) past, not a particularly interesting subject; I'm focused on
    how to make things better in the future.


    >>> The word "exceptions" implies "exception to the previous statement".

    >>
    >>Yes, to the statement terminated by the semicolon - the one saying that
    >>few functions steal references.
    >>
    >>> IIUC "exceptions" should be changed to "examples".

    >>
    >>Yes, this is surely a reasonable change, if it clarifies the text for some
    >>readers.

    >
    > Since you agree (and make more specific) that 'exceptions' means
    > 'exception to the statement terminated by the semicolon', and since
    > neither PyList_SetItem nor PyTuple_SetItem are exceptions to the few
    > functions that steal references, doesn't that make the choice of the
    > word 'exceptions' unreasonable (as in exactly the opposite of what was
    > meant)? It's not a matter of simple clarification.


    No, the use of the word 'exceptions' is anything but unreasonable,
    except in as much as each natural language has its typical foibles which
    might, abstractly, be considered "unreasonable" if judging natural
    languages from completely inappropriate viewpoints (in this case, I'd
    use exactly the same phrasing in Italian or French, so I'm going to
    staunchly defend the perfect reasonableness of normal English usage).

    The statement "Few people do that" is semantically equivalent to "most
    people don't do that", it's just a smoother and more natural way of
    expression; in _either_ case, when one continues by mentioning people
    as "exceptions", it's then just as reasonable as any other idiomatic
    construction -- and perfectly clear in context -- that the "exceptions"
    are the (few) people who _do_ "do that".

    In an alternative phrasing such as "I don't like any ice-cream brand;
    the few exceptions are... [mention of some brands]", the statement
    after the semicolon would _contradict_ the one before the semicolon,
    which (at least in English and Italian) doesn't sound good (although
    in _spoken_ language this construct would astonish nobody, it should
    really be edited when found in _written_ form). When considering how
    to edit this, I would _definitely_ want to keep the word "exceptions"
    (if I knew I could count on good command of the language on readers'
    parts) because its connotations are _just right_: whenever I mention
    an exception, a reader is inherently attraced to think of a general
    rule -- and in this case the reader finds the rule right before that
    semicolon, reinforcing exactly the "rule/exceptions" message I am
    trying to send. So, rather than changing the part after the semicolon,
    I look for a way to weaken the assertion before the semicolon, which
    was here stated as "absolute rule". "I don't like most ice-cream
    brands" and "I like very few ice-cream brands" are semantically
    equivalent, but I prefer the second: all other things being equal,
    an assertion phrased in the positive form is easier for the reader
    than one phrased in the negative; and the second form is smoother.

    The semantics equivalence of the two forms guarantees against any
    problem with what follows after the semicolon: in either case, that
    part is going to be read as "the few exceptions [[to the general
    rule just stated]] are" -- and _the general rule just stated is
    *the same* in either case_, which is the point.

    All that's in discussion, therefore, is whether to make the
    whole expression not quite as strong and obvious to native readers
    (missing out on the helpful connotation "exceptions" --> "general
    rule") for the purpose of potentially clarifying things for those
    readers who, on seeing "exceptions", do NOT immediately "read the
    unwritten words" ``to this general rule''. Unfortunately "examples",
    for exactly the same readers who DO get the right connotations from
    "exceptions", brings the WRONG connotation here: "examples" leads
    to "reading the unwritten words" ``OF this general rule''.

    But the general rule is that normally functions *do NOT* steal
    references, and yet the alleged ''examples'' are _NOT_ "examples
    of this general rule", they are in fact the _exceptions_ to this
    general rule. This means the connotations become, from just
    right, to just wrong _for native readers_, and makes the whole
    suggested change problematic.


    > Either the word 'exceptions' should be changed, or it should be changed
    > to 'notable exceptions to <here specify the domain to which the two
    > abovementioned functions are exceptions, and which domain is not "'few
    > functions steal references", since the functions do steal references>'


    There's no such general concept as "exceptions to a domain", but rather
    "exceptions to a general rule". The _general rule_ *IS* "normally
    functions don't steal references" (expressed in the preferable positive
    form). Redundantly repeating exactly what you're referring to in each
    case is of course a possibility (I'm not very attuned to it since the
    editors, quite rightly, would never tolerate such mostly-wasted
    wordiness in e.g. magazine articles or Nutshell books, but a reference
    manual can be weighed by other criteria). Given that there _is_ no
    ambiguity (for the mostly-intended audience of people with the right
    level of idiomatic command of the language) I doubt I can defend such
    redundancy, though (particularly not being enthusiastic about it:).


    > The previous two paragraphs were written only because your 'if it
    > clarifies the text for some readers' seems to ignore the fact you
    > yourself stated two paragraphs above in 'Yes, to the statement...'


    I hope I clarified this abundantly, and thank you for forcing me to
    think the issues through in such detail as needed to argue against
    your contention -- I "instinctively knew" that what you said was
    entirely wrong, but explaining exactly how and why meant I had to
    analyze and reason in depth about the whole issue. It's funny --
    though I'll never have the easy command of a native speaker, of
    course, yet 40+ years of constant practice of English do seem to
    have given me _some_ "reflex" reactions (in this case, the fact
    that the applicable idioms _are_ just about the same in Italian
    does, of course, also help:).


    > If the python documentation had a terminology dictionary, how would
    > 'stealing', 'borrowing' and 'owning' a reference be defined in a concise
    > and absolutely clear way? I seem to not be able to write these
    > definitions, although I would like to.


    The concept of "owning a reference" is the crucial one. "Stealing"
    is just one way to permanently transfer ownership, "borrowing" means
    no transfer of ownership takes place, and these meanings are so close
    to those in everyday language that the "terminology dictionary" would
    have few problems, I believe.

    So, focusing on ownership, and foregoing the concision that would
    probably be required in a dictionary, we might say:

    """
    Ownership pertains to references, never to objects (objects are not
    owned: they are always shared). "Owning a reference" means being
    responsible for calling Py_DECREF on it when the reference is no
    longer needed. Ownership can also be transferred, meaning that the
    code that receives ownership of the reference then becomes
    responsible for eventually decref'ing it when it's no longer needed.
    """

    plus, perhaps, pointers to the various appropriate sections in
    the "extending and embedding" tutorial and API reference that deal
    with this concept. Really, the true meaning of "reference
    ownership" is _operational_ -- the "responsibility of eventually
    decref'ing (unless the ownership is transferred)" _together_ with
    the rules for how and when ownership (i.e. responsibility to
    eventually decref) is transferred.


    Alex
     
    Alex Martelli, Nov 6, 2003
    #13
  14. My last post in this thread, since it --like Alex noticed too-- tends to
    move into 'off-topic' territory.

    On Thu, 06 Nov 2003 14:30:30 GMT, rumours say that Alex Martelli
    <> might have written:

    [snip: Alex ponders on whether the thread remains on topic (it merely
    does), then states that, as an author himself, has strong opinions on
    style, nuances, fine point of connotation etc]

    Point taken, and be sure that all my respect goes to acknowledged
    authors like yourself; after all, we don't discuss one of your books or
    your style here, but rather the python documentation and how it could be
    improved for newcomers --not that *I* *know* how to improve it. I only
    pointed parts that might be as unclear to others as to me.

    [snip: I suggesting a defining "owning a reference", Alex
    counter-proposing a variation to the current doc sentence about what a
    function that owns a reference has to do, I accidentally ignoring the
    counter proposal, summarising instead where in the docs the answers to
    my questions can be collected from]

    >So what about my proposed transliteration of "when X, Y" into "X
    >means Y"? This is similar to (e.g.) changing "when somebody owns
    >a property outright, they can sell it" to "owning a property
    >outright means you can sell it" -- essentially equivalent in normal
    >English usage, but perhaps more specific or clear to non-natives?


    I really have no preference between the two. I believe they carry the
    same amount of information. Sorry for not commenting upon your
    proposal.

    >>>> This text is obviously clear

    [snip: I acknowledge a fault on my part in keeping in mind the
    difference between owning an object and owning a reference, Alex
    re-states the fact that the documentation is clearest on this, I mistake
    at a point Alex's use of second person in his sentence as referring to
    me, and Alex explains that he used 'you' as a generic person (my
    comment: à la 'on' in French) instead of me specifically.]

    Explanation and not justification: In English I have met more often the
    idiom 'one can only hope' than 'you can only hope' when speaking in
    general, so I perceived you meant me. After your clarification, I
    understand it's a null issue.

    [snip: discussion about the phrase 'Few functions steal references; the
    two notable exceptions are...']

    Sorry for snipping all of your reply, but this should be kept short. Do
    you believe that the following:

    'Few functions steal references; of these exceptions, the two notable
    are...'

    is more to the point of passing the message that the word 'exceptions'
    after the semicolon means a subset of the exceptions referenced before
    the semicolon?

    If you really believe this is a moot point, please, let's drop the
    subject without further discussion on this. I insist on my point for
    clarity, but I am not an expert, cause I am a self-taught English
    speaker (by listening to songs, watching cinema and TV, and practicing
    whenever I could with native speakers (mostly British, apart from two
    trips to the States)).
    I say to drop the subject if you disagree because, pragmatically
    speaking, your opinion counts more than mine, so let's keep our energies
    for more grave matters.

    [snip some more discussion about the 'exceptions' subject]

    >> If the python documentation had a terminology dictionary, how would
    >> 'stealing', 'borrowing' and 'owning' a reference be defined in a concise
    >> and absolutely clear way? I seem to not be able to write these
    >> definitions, although I would like to.

    >
    >The concept of "owning a reference" is the crucial one. "Stealing"
    >is just one way to permanently transfer ownership, "borrowing" means
    >no transfer of ownership takes place, and these meanings are so close
    >to those in everyday language that the "terminology dictionary" would
    >have few problems, I believe.
    >
    >So, focusing on ownership, and foregoing the concision that would
    >probably be required in a dictionary, we might say:
    >
    >"""
    >Ownership pertains to references, never to objects (objects are not
    >owned: they are always shared). "Owning a reference" means being
    >responsible for calling Py_DECREF on it when the reference is no
    >longer needed. Ownership can also be transferred, meaning that the
    >code that receives ownership of the reference then becomes
    >responsible for eventually decref'ing it when it's no longer needed.
    >"""
    >
    >plus, perhaps, pointers to the various appropriate sections in
    >the "extending and embedding" tutorial and API reference that deal
    >with this concept. Really, the true meaning of "reference
    >ownership" is _operational_ -- the "responsibility of eventually
    >decref'ing (unless the ownership is transferred)" _together_ with
    >the rules for how and when ownership (i.e. responsibility to
    >eventually decref) is transferred.


    There you go, back to your excellent ability to reply to answers in a
    way that seems definite. That's the Alex before the tiresome
    python-syntax-in-lisp thread :)

    The above text in triple-quotes, believe me, is what I sought since the
    start. Our discussion really helped me answer my questions before I
    read it clearly in your post; and I firmly believe that your
    triple-quoted text should be added to the documentation.

    I just submitted patch 837322 merging your words above into the
    documentation text --please revise.

    Thanks for your time.
    --
    TZOTZIOY, I speak England very best,
    Ils sont fous ces Redmontains! --Harddix
     
    Christos TZOTZIOY Georgiou, Nov 6, 2003
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    0
    Views:
    341
  2. Replies:
    22
    Views:
    762
    peter koch
    Apr 30, 2008
  3. Replies:
    6
    Views:
    352
    James Kanze
    Apr 29, 2008
  4. Navindra Umanee

    strong ref from weak ref?

    Navindra Umanee, Feb 12, 2005, in forum: Ruby
    Replies:
    2
    Views:
    145
    Navindra Umanee
    Feb 12, 2005
  5. Juha Nieminen
    Replies:
    13
    Views:
    620
    Edek Pienkowski
    Aug 29, 2012
Loading...

Share This Page