RE: anything like C++ references?

Discussion in 'Python' started by Ian Bicking, Jul 12, 2003.

  1. Ian Bicking

    Ian Bicking Guest

    On Sat, 2003-07-12 at 17:45, Brian Quinlan wrote:
    > > void change(int& i)
    > > {
    > > i++;
    > > }

    >
    > The idiomatic way to write this code in python would be:
    >
    > def change(val):
    > return val + 1


    To be more specific, you would achieve the same effect with:

    def change(val):
    return val + 1
    i = change(i)

    As opposed to the C++ where you'd do:

    change(i) // no assignment needed


    There is no direct Python equivalent to the C++ function, for all sorts
    of reasons (most of them very deliberate).

    Is there someplace in particular to reference people who need to learn
    about what Python variables are (that they are bindings, etc)? This
    question comes up in many forms all the time, particularly from people
    with a C background. Or rather, there are many questions all of which
    are answered by that explanation (so we can't expect people to stop
    asking, but maybe we can make the answers easier and clearer). Or maybe
    a general Python-for-C-programmers tutorial...

    Ian
     
    Ian Bicking, Jul 12, 2003
    #1
    1. Advertising

  2. Tom Plunket wrote:

    > IMHO the variable binding discussion doesn't answer the question,
    > it just provokes more questions. :)


    But the answer to those questions is usually something ending with,
    "Look, Python just doesn't handle variables the same way C++ does." :)

    --
    Erik Max Francis && && http://www.alcyone.com/max/
    __ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
    / \ It seems like Karma's making his rounds
    \__/ Kina
     
    Erik Max Francis, Jul 13, 2003
    #2
    1. Advertising

  3. On Sun, 13 Jul 2003 00:55:22 -0700, Erik Max Francis <>
    wrote:

    >Tom Plunket wrote:
    >
    >> IMHO the variable binding discussion doesn't answer the question,
    >> it just provokes more questions. :)

    >
    >But the answer to those questions is usually something ending with,
    >"Look, Python just doesn't handle variables the same way C++ does." :)


    And my traditional answer to that is... "yes, and the way that Python
    handles variables is definitely a wart."

    One of the few things I hate about Python is that mutable objects are
    implicitly shared using a reference system whereas immutable objects
    are not. It is unnecessarily confusing and error prone.

    A system where all objects are copied unless a reference type is
    explicitly applied (and with an assignment operators recognising
    references on the LHS by default and assigns to the object rather than
    the reference, though with an extra one which overwrites the
    reference) seems much more logical to me. It even seems 'Pythonic' to
    me. But it is much closer to C++ references than current Python
    system.
     
    Stephen Horne, Jul 13, 2003
    #3
  4. Ian Bicking

    Roy Smith Guest

    Stephen Horne <> wrote:
    > One of the few things I hate about Python is that mutable objects are
    > implicitly shared using a reference system whereas immutable objects
    > are not. It is unnecessarily confusing and error prone.


    There's a bit of confusion in the above regarding mutability vs. call by
    reference (they're orthogonal ideas), but I think I know what you were
    trying to say.

    In ancient versions of Fortran, even constants (i.e. immutables) were
    pass by reference. You could write something like this (forgive me if
    I've forgotten some of the syntax details):

    subroutine x (i)
    i = 42
    end

    subroutine main
    x (5)
    write (6, 100) 5
    100 format (1i6)
    end

    and it would print 42. There was a single instance of the constant 5
    for main, and it was passed by reference to x(), which in turn changed
    its value to 42. I don't remember if Fortan 66 did this, or if that
    only happened in even older versions. Talk about confusing and error
    prone!

    The Python version of that would be:

    def mutateString (s):
    s = "some other string"

    s ("foo")
    print "foo"

    If strings were pass by reference, the above would print "some other
    string". Unless, of course, you got away from the idea that multiple
    instances of a string constant with the same value are really the same
    object, in which case:

    x = "foo"
    x is "foo"

    would print 0 instead of 1 like it does now.
     
    Roy Smith, Jul 13, 2003
    #4
  5. Ian Bicking

    Aahz Guest

    In article <>,
    Stephen Horne <> wrote:
    >
    >One of the few things I hate about Python is that mutable objects are
    >implicitly shared using a reference system whereas immutable objects
    >are not.


    Well, that's incorrect. *ALL* Python objects are implicitly shared with
    bindings. The difference is whether updating the value referenced by a
    target requires *re*binding the target or simply updating an object.
    --
    Aahz () <*> http://www.pythoncraft.com/

    "Not everything in life has a clue in front of it...." --JMS
     
    Aahz, Jul 13, 2003
    #5
  6. On 13 Jul 2003 10:37:01 -0400, (Aahz) wrote:

    >In article <>,
    >Stephen Horne <> wrote:
    >>
    >>One of the few things I hate about Python is that mutable objects are
    >>implicitly shared using a reference system whereas immutable objects
    >>are not.

    >
    >Well, that's incorrect. *ALL* Python objects are implicitly shared with
    >bindings. The difference is whether updating the value referenced by a
    >target requires *re*binding the target or simply updating an object.


    Fine - nit-pick.

    All you have proven is that it is the distinction between types that
    get re-bound and those that don't (rather than the use of references)
    that is unnecessarily confusing and error prone.

    The wart remains, even if my description was wrong. And even that is a
    dubious claim.

    A Python user is interested in how an object behaves - not how it is
    internally implemented in the interpreter. Immutable objects don't
    behave as references - the internal use of references for immutable
    objects is basically a lazy copying optimisation and, apart from
    performace and a couple of other technicalities (e.g. the 'is'
    operator), has no relevance. Certainly it has no relevance to the
    point I was making.
     
    Stephen Horne, Jul 13, 2003
    #6
  7. On 13 Jul 2003 12:19:08 -0400, (Aahz) wrote:

    >Whether one can mutate a specific object is simply an
    >attribute of that object, rather than requiring a different syntax.
    >Trying to focus on the mutable/immutable distinction is what causes the
    >mental blowup -- keep your eye on the objects and bindings and you're
    >fine.


    That's exactly it - you have to focus on whether an object is mutable
    or immutable for that exact reason.

    While I admit I'm not sure, I believe that in early versions of Python
    immutable values literally were copied. Mutable types were made
    mutable because of efficiency concerns about copying overheads that
    would occur otherwise. If an object is large enough that it is
    worthwhile modifying items within it in-place then it tends to be
    worthwhile trying to avoid copying overheads, even though these two
    aspects of 'mutability' are actually quite distinct in principle.

    Actually, I'm convinced there are some flaws in Guidos own logic
    *unless* this is the case. Take for instance this...

    http://www.python.org/search/hypermail/python-1992/0292.html

    """
    <A> They serve different purposes. Lists can get quite long, they are
    generally built incrementally, and therefore have to be mutable.
    Tuples on the other hand are generally short and created at once.


    <Q> Then why can't tuples be mutable and serve both purposes?


    <A> Imagine a graphics class that stores coordinates of graphical
    objects as tuples. It may also store rectangles as a tuple of points,
    etc. If tuples were mutable, the class would have to store copies of
    all points and rectangles it receives, since otherwise a caller who
    creates a point variable, passes its value to the graphics class, and
    later changes the point for its own use (e.g., to create another
    graphical object with slightly different coordinates) might violate
    the internal consistency of the graphics class. Note that most
    callers woouldn't modify the points, but the graphics class has no way
    to tell, so it has to make the copies anyway. (And now imaging the
    software is actually layered in such a way that coordinates are passed
    down several levels deep...)
    """

    So what?

    As things are, the points could be stored using a list of tuples, with
    each tuple representing a point. When you store a new point in the
    list, you don't immediately make a copy (so if the supplier of the
    point immediately discards it, no copying is needed at all except for
    pointers) so it is efficient - yet you are still guarenteed that
    external processing will not mutate the value you hold in your list.

    Fine.

    But that can be achieved equally well by applying a copy-on-write
    system to *all* types, without a distinction between mutable and
    immutable for this purpose.

    Equally, you could use the same rationale for storing items which
    Python classes as mutable instead of these point tuples. If you have
    an object which stores class instances or dictionaries or lists for
    you, you need to store copies as - just as Guido recognised for the
    tuple case - your object doesn't know for sure what the caller is
    going to do with those objects after passing them in as parameters.

    This makes no sense to me *unless* there were other reasons for the
    mutable/immutable distinction. If lists had copy-on-write semantics,
    they could still be mutable (ie support in-place modification) yet
    there would be no need to make early copies. Tuples would be redundant
    as their only distinctive feature would be their lack of support for
    in-place modifications. It only makes sense if the copy-on-write for
    immutable values hadn't been implemented when the distinction was
    created - if immutables did not use references, but were always copied
    immediately on assignment - so that the referencing mechanism need
    never worry about copy-on-write at all.


    But it is worse than just being an arbitrary and pointless
    distinction...

    Back when PEP238 was being 'debated', a major argument in favor of the
    change was that there is no legitimate case where a function would
    need to either do 'true' division or integer division depending on the
    argument type. The type of division needed depends on the purpose of
    the function, and behaviour shouldn't change just because someone
    passes an integer into a function that expects a real or visa versa.

    Apply the same logic to mutable and immutable types.

    Tell me one case where it is sensible for a function to behave such
    that whether the caller sees a change in a variable it passed as its
    argument should depend on the type.

    Tell me one case where an object storing values should care about
    callers mutating values it holds *only* for certain types.

    I doubt you can.

    This is an arbitrary distinction created for historic reasons that
    have long since vanished (apart, of course, from backward
    compatibility and psychological inertia), given that copy-on-write is
    implemented for immutable objects anyway. And the fact that immutable
    values do an excellent job of implementing the common semantics of
    most imperitive languages (i.e. assignment sets the LHS to something
    that can be treated as a distinct copy) whereas mutable types behave
    in a fundamentally different way is quite simply a wart -
    inconsistent, confusing and error-prone behaviour.
     
    Stephen Horne, Jul 13, 2003
    #7
  8. Ian Bicking

    Donn Cave Guest

    Quoth Stephen Horne <>:
    | On 13 Jul 2003 12:19:08 -0400, (Aahz) wrote:
    |
    |> Whether one can mutate a specific object is simply an
    |> attribute of that object, rather than requiring a different syntax.
    |> Trying to focus on the mutable/immutable distinction is what causes the
    |> mental blowup -- keep your eye on the objects and bindings and you're
    |> fine.
    |
    | That's exactly it - you have to focus on whether an object is mutable
    | or immutable for that exact reason.
    |
    | While I admit I'm not sure, I believe that in early versions of Python
    | immutable values literally were copied. Mutable types were made
    | mutable because of efficiency concerns about copying overheads that
    | would occur otherwise. If an object is large enough that it is
    | worthwhile modifying items within it in-place then it tends to be
    | worthwhile trying to avoid copying overheads, even though these two
    | aspects of 'mutability' are actually quite distinct in principle.

    Sure, mutable values literally are copied today. If you have a tuple
    of time.localtime(x), for example, and you want that value but with
    tm_mday incremented, then you must copy the tuple to do that (except
    for tm_mday, for which you substitute your own value.) It is indeed
    inefficient - OK for this purpose but not for large sequences, and
    there we tend to prefer a mutable list.

    That's the efficiency argument. Assigment never copied values, in
    any version of Python that anyone cared about. We just don't want
    big immutable arrays that have to be copied just to change a single
    value. There isn't a tuple analogue for the dictionary because it
    wouldn't be so generally useful (the recently implemented struct
    tuple would have been nice to have earlier, though.)

    He's quite right, the notion of mutability is a red herring. It
    isn't that no one cares - the propagation of state changes in a
    program is critical to the operation of the program, and to reasoning
    about how the program will operate.

    It's just that when you're trying to come to grips with how assignment
    and argument passing works in Python, you don't want to have people
    start explaining things in these terms (``see, this object is mutable
    and that one isn't''), because you will understandably but incorrectly
    apply that to assignment and argument passing. Mutability in Python
    has no effect on assignment and argument passing, it is just an attribute
    of the passed or assigned object.

    Donn Cave,
     
    Donn Cave, Jul 13, 2003
    #8
  9. Ian Bicking

    Ian Bicking Guest

    On Sun, 2003-07-13 at 12:51, Stephen Horne wrote:
    > On 13 Jul 2003 12:19:08 -0400, (Aahz) wrote:
    >
    > >Whether one can mutate a specific object is simply an
    > >attribute of that object, rather than requiring a different syntax.
    > >Trying to focus on the mutable/immutable distinction is what causes the
    > >mental blowup -- keep your eye on the objects and bindings and you're
    > >fine.

    >
    > That's exactly it - you have to focus on whether an object is mutable
    > or immutable for that exact reason.


    No you don't! Mutable and immutable objects act the same with respect
    to assignment. However, because you cannot change a immutable object in
    place, to get a different value you must rebind. You must *use* mutable
    and immutable objects differently -- but that is not surprising, because
    they are obviously very different objects! You have to use integers and
    lists differently in part because one is mutable and the other isn't --
    but mostly because one is a number and the other is a list. Different
    objects are different!

    Python is not novel in the way it deals with variables. Scheme and
    Smalltalk, for instance, act exactly the same, as do many other
    dynamically typed languages (though there are different opinions on
    whether strings should be mutable -- but it's agreed there has to be
    some immutable string-like type, e.g. symbol). The reason you are
    getting this reaction is that anyone that comes from those backgrounds
    thinks you are crazy, as does anyone who has embraced the Python model.
    This isn't a funny little feature, this is the way all strong,
    dynamically typed languages work.

    > While I admit I'm not sure, I believe that in early versions of Python
    > immutable values literally were copied. Mutable types were made
    > mutable because of efficiency concerns about copying overheads that
    > would occur otherwise. If an object is large enough that it is
    > worthwhile modifying items within it in-place then it tends to be
    > worthwhile trying to avoid copying overheads, even though these two
    > aspects of 'mutability' are actually quite distinct in principle.


    The problem you have is you are still thinking of variables as slots,
    which is not correct. Variables in Python are bindings. Assignment
    never copies anything, or creates anything except for changing the
    variable to point to a different address location. *Every* Python
    assignment (a=b) is like the C assignment (a=&b).

    > Tell me one case where it is sensible for a function to behave such
    > that whether the caller sees a change in a variable it passed as its
    > argument should depend on the type.


    Generally a function takes either immutable values (e.g., ints and
    floats) or mutable values for a certain argument.

    However, there is a class of operations which are generally operate in
    an immutable manner, that is, create copies of the objects instead of
    changing them in place. So even though lists are mutable, list
    concatenation does not mutate, and in general adding two things (with +)
    will not mutate either object. Immutable values, by design, do not have
    the same methods and operations as the mutable counterparts (or at least
    the mutating methods of those mutable objects). *That* would be a
    design bug.

    > Tell me one case where an object storing values should care about
    > callers mutating values it holds *only* for certain types.


    Objects don't *store* values, they *refer* to values. You are still
    thinking like you're in C (or C++). This is why you are having a
    problem.

    Ian
     
    Ian Bicking, Jul 13, 2003
    #9
  10. On 13 Jul 2003 14:48:09 -0500, Ian Bicking <>
    wrote:

    >On Sun, 2003-07-13 at 12:51, Stephen Horne wrote:
    >> On 13 Jul 2003 12:19:08 -0400, (Aahz) wrote:
    >>
    >> >Whether one can mutate a specific object is simply an
    >> >attribute of that object, rather than requiring a different syntax.
    >> >Trying to focus on the mutable/immutable distinction is what causes the
    >> >mental blowup -- keep your eye on the objects and bindings and you're
    >> >fine.

    >>
    >> That's exactly it - you have to focus on whether an object is mutable
    >> or immutable for that exact reason.

    >
    >No you don't! Mutable and immutable objects act the same with respect
    >to assignment. However, because you cannot change a immutable object in
    >place, to get a different value you must rebind. You must *use* mutable
    >and immutable objects differently -- but that is not surprising, because
    >they are obviously very different objects! You have to use integers and
    >lists differently in part because one is mutable and the other isn't --
    >but mostly because one is a number and the other is a list. Different
    >objects are different!
    >
    >Python is not novel in the way it deals with variables. Scheme and
    >Smalltalk, for instance, act exactly the same, as do many other
    >dynamically typed languages (though there are different opinions on
    >whether strings should be mutable -- but it's agreed there has to be
    >some immutable string-like type, e.g. symbol). The reason you are
    >getting this reaction is that anyone that comes from those backgrounds
    >thinks you are crazy, as does anyone who has embraced the Python model.
    >This isn't a funny little feature, this is the way all strong,
    >dynamically typed languages work.


    In computer science, a variable is a named binding to a value.
    Operations on that variable may rebind it to a different value, but
    values don't change behind your back. A pointer is, in effect, a value
    which indirectly refers to another (possibly anonymous) variable. A
    pointer is something you specifically request, and by doing so you
    allow that the 'variable' being indirectly referenced may be modified
    by something else that has no direct access to your pointer value (via
    your named variable or via copys or pointers-to your pointer held
    elsewhere).

    Python should respect that.

    If you claim that it does, then the values to which we currently
    associate the type name 'list', 'dictionary' and 'class instance' are
    badly named. They should be called 'pointer to list', 'pointer to
    dictionary' and 'pointer to class instance'. And if you want to call
    those references and make the dereferencing implicit, fine. Equally,
    if you want to implement variable binding using references and using
    copy-on-write to implement lazy copying (which does not violate the
    standard computer-theory semantics of the binding to values - only the
    binding to the implementation of values ie memory locations) then
    equally fine.

    However, even if the names included the 'pointer to' prefix so that
    they actually described the real behaviours of the values, this still
    raises two important questions...

    1. Why are the values of mutable objects always forced to be accessed
    via a pointer (or reference or whatever)?

    Note that I am referring to a computer science, semantic
    pointer/reference/whatever - not to any pointers or references that
    may be used in the implementation of the binding of variables to
    values. The use, behind the scenes, of lazy copying as an optimisation
    is irrelevant to the computer science principles.

    2. Why is there no way to reference an immutable object via a
    pointer, other than stuffing it into a mutable object designed for
    some purpose other than simple pointer behaviour?

    The truth is that this system is arbitrary, and that therefore this
    excuse is invalid.

    >The problem you have is you are still thinking of variables as slots,
    >which is not correct. Variables in Python are bindings. Assignment
    >never copies anything, or creates anything except for changing the
    >variable to point to a different address location. *Every* Python
    >assignment (a=b) is like the C assignment (a=&b).


    The problem is that you are confusing implementation with semantics.
    The meanings of 'variable', 'value', 'assignment' etc are defined by
    computer science. Whether you arbitrarily change the meaning of
    'assignment' or whether you arbitrarily change the meaning of
    'variable', it amounts to the same thing.

    >
    >> Tell me one case where it is sensible for a function to behave such
    >> that whether the caller sees a change in a variable it passed as its
    >> argument should depend on the type.

    >
    >Generally a function takes either immutable values (e.g., ints and
    >floats) or mutable values for a certain argument.
    >
    >However, there is a class of operations which are generally operate in
    >an immutable manner, that is, create copies of the objects instead of
    >changing them in place. So even though lists are mutable, list
    >concatenation does not mutate, and in general adding two things (with +)
    >will not mutate either object. Immutable values, by design, do not have
    >the same methods and operations as the mutable counterparts (or at least
    >the mutating methods of those mutable objects). *That* would be a
    >design bug.


    You miss my point. Your argument would equally apply to the need for
    an integer-specific division operator, for instance. Its all about how
    one function reacts in response to varying input. If you pass a
    mutable value to a function which expects an immutable one, you get an
    error. If you pass an immutable value to a function which expects a
    mutable one, you get an error. There is no good reason for a function
    to react to the distinction as evidenced by your own observation that
    what actually happens (and what should happen) is that you have two
    distinct variants of the function.

    >> Tell me one case where an object storing values should care about
    >> callers mutating values it holds *only* for certain types.

    >
    >Objects don't *store* values, they *refer* to values. You are still
    >thinking like you're in C (or C++). This is why you are having a
    >problem.


    No it is not. I'm thinking in terms of computer science, in which
    terms like 'variable', 'value' and 'assignment' are abstract concepts
    independent of the way in which they are implemented in a programming
    language.

    One way or the other, Python is currently choosing not to respect the
    computer science definition of those terms. It may have historic and
    backward-compatability reasons, but that does not change the facts.
    This deviation from computer science definitions, whatever the excuse,
    is arbitrary, confusing and error prone. That is my problem.
     
    Stephen Horne, Jul 13, 2003
    #10
  11. On Sun, 13 Jul 2003 16:11:37 +0100, Stephen Horne <> wrote:

    >On 13 Jul 2003 10:37:01 -0400, (Aahz) wrote:
    >
    >>In article <>,
    >>Stephen Horne <> wrote:
    >>>
    >>>One of the few things I hate about Python is that mutable objects are
    >>>implicitly shared using a reference system whereas immutable objects
    >>>are not.

    >>
    >>Well, that's incorrect. *ALL* Python objects are implicitly shared with
    >>bindings. The difference is whether updating the value referenced by a
    >>target requires *re*binding the target or simply updating an object.

    >
    >Fine - nit-pick.

    Well, it *is* a (very) significant nit ;-)

    >
    >All you have proven is that it is the distinction between types that
    >get re-bound and those that don't (rather than the use of references)
    >that is unnecessarily confusing and error prone.

    I suspect you are reading some Python statements as if they were C or C++.

    Python name assignment does not do anything to an object. It just creates
    an alias name for an object. It may reuse an existing name, but that only
    changes the meaning of that name as an alias, it does not affect the object
    that the name was previously an alias for.

    You can of course have other assignment targets than plain names. In fact,
    a "plain name" is just the simplest expression for an assignment target.
    But whatever the target expression evaluates to, what gets put "there" is
    a reference to an object, not an object itself.

    The confusion, ISTM, is in what to think of as "there." "There" is definitely
    not a memory space for a Python object representation or "value".

    The most accurate (UIAM) C concept would be a "there" of type PyObject* -- i.e.,
    a pointer variable that points to the representation of some kind of Python object.

    We don't have to know the implementation details to use the idea that the left hand
    side of a Python assignment is an *expression* (even if a single name) that yields
    a place to put a reference (effectively a pointer) to the object created by evaluating
    the expression on the right hand side.

    A C programmer will tend to think that a symbol on the left is a static expression
    indicating a specific fixed memory space (static address or stack frame offset). But in Python
    it is first of all a dynamic expression (though an unqualified target name can only have its "there"
    be in the local or global namespace, and the choice between those is made statically
    (at least w.r.t. a given level of compiling/exec-ing).

    In C terms, the Python assignment target expression always evaluates to a place to put a pointer,
    never a place to put object representation bits. Of course, "a place to put a pointer"
    may be *inside* an object. And such inside places are identified by target expressions
    such as x[2] or x.a or x[2]().b[3] etc. Even a bare name really does identify a place inside
    an object -- i.e., inside the local or global dict associated with the context.

    The "place" identified by x= after a global x declaration (or just in global scope) will be the
    same place as globals()['x']= unless someone has subverted something. Either way, the binding
    of the name to the object happens by evaluating to a "there" within the global dict object,
    uniquely associated with the name ('x' here). Evaluated on the right hand side, that name will
    produce the reference/pointer again for use in accessing the object or copying to another "there"
    associated with perhaps some other name or a target within a composite object like a list or tuple,
    or other namespace dict.

    >
    >The wart remains, even if my description was wrong. And even that is a
    >dubious claim.
    >

    Every assignment is effectively stores a referential alias for an object,
    whether associated with a name or not. This is by design, not a wart.

    >A Python user is interested in how an object behaves - not how it is
    >internally implemented in the interpreter. Immutable objects don't

    when you say "immutable objects," are you really speaking of names
    bound to immutable objects? I.e., the integer 123 is an immutable object
    represented by the literal 123. You get an immutable object of the same value
    from the expression int('123'). These are the int *objects* -- how does
    "behave as references" apply to these actual "immutable objects"?
    I.e., could you rewrite your sentence (that this is breaking in two)
    to make it perhaps more understandable for me ?;-)

    Names are not objects. Nor are left-hand-side assignment target expressions
    in general, whether bare names or complicated.

    ISTM what you are discussing is not about objects but about the way names work,
    and ISTM you are thinking of names as if they were C++ variable references,
    which they are not, and they couldn't be. For one thing, a C++ reference type
    has to be initialized once to refer to a specific object. It can't be made to
    refer to something else during the life of that scope. Pointers are a better
    model, since you have to distinguish by expression syntax whether you mean
    to assign a new pointer value or modify the thing pointed to. In python you
    can only assign pointers, if you want to think in those terms. When you
    modify a mutable, you are still assigning pointers into some part of the
    mutable object representation. When you assign to a bare name, you are still assigning
    a pointer into a place in some dictionary object (or workalike). If you
    want to modify an object in the usual C sense, you can code it in C and provide a
    python interface to your C-implemented object. When you pass arguments
    to the various methods of your mutable, you will get pointers that you
    can dereference and you can do what you want to your mutable's C data representation.
    But when you pass the result back to the python world, it will have to be
    as a standard object reference/pointer, and if you assign that, you will be
    storing a pointer somewhere.

    >behave as references - the internal use of references for immutable
    >objects is basically a lazy copying optimisation and, apart from
    >performace and a couple of other technicalities (e.g. the 'is'
    >operator), has no relevance. Certainly it has no relevance to the
    >point I was making.
    >

    The way it works is part of the semantics of the language, not just
    an optimization issue.

    Names aren't variables.

    HTH ;-)

    <disclaimer>I sometimes get things wrong. Corrections welcome.</disclaimer>

    Regards,
    Bengt Richter
     
    Bengt Richter, Jul 13, 2003
    #11
  12. Ian Bicking

    Adam Ruth Guest

    Stephen Horne <> wrote in message news:<>...
    > On 13 Jul 2003 14:48:09 -0500, Ian Bicking <>
    > wrote:
    >
    > >On Sun, 2003-07-13 at 12:51, Stephen Horne wrote:


    > 1. Why are the values of mutable objects always forced to be accessed
    > via a pointer (or reference or whatever)?


    Because *all* objects are accessed via pointer, sorta.

    > Note that I am referring to a computer science, semantic
    > pointer/reference/whatever - not to any pointers or references that
    > may be used in the implementation of the binding of variables to
    > values. The use, behind the scenes, of lazy copying as an optimisation
    > is irrelevant to the computer science principles.
    >
    > 2. Why is there no way to reference an immutable object via a
    > pointer, other than stuffing it into a mutable object designed for
    > some purpose other than simple pointer behaviour?


    They *are* referenced by pointer, sorta. It's just that what is being
    pointed to cannot be changed. If you were using C and you had a
    pointer to a variable in a region of memory that you couldn't change,
    then you'd have, semantically, the equivalent.

    In that case, how you do modify the value? You don't, you only modify
    the *pointer*. If you want to change the pointer, you need to pass a
    pointer to a pointer. This is what doesn't exist in Python, and I
    think it's what you're having problems with. The same "problem"
    exists in Java, but for slightly different reasons.

    The Python method is completely consistent and all objects are treated
    the same. There are no *pointers* even though binding sometimes
    behaves like they. Binding doesn't always work like pointers, but it
    always works like binding.

    >
    > The truth is that this system is arbitrary, and that therefore this
    > excuse is invalid.


    The definition of what is and isn't immutable isn't arbitrary in the
    sense of "Determined by chance, whim, or impulse, and not by
    necessity, reason, or principle". I'd say that the choices were made
    on necessity, reason, and principle. Just not the necessity, reason,
    or principle you would have chosen.

    What happens in this case?

    char *myval = "my little pony";
    myval[1] = 'e';

    It would appear that C and C++ also have immutable types. And in this
    case the reason it's immutable bears no relation to the object or
    type, but to it's location in memory, I'd say that that appears more
    arbitrary. "Some strings are immutable and some aren't" is worse than
    "all strings are immutable".

    > >The problem you have is you are still thinking of variables as slots,
    > >which is not correct. Variables in Python are bindings. Assignment
    > >never copies anything, or creates anything except for changing the
    > >variable to point to a different address location. *Every* Python
    > >assignment (a=b) is like the C assignment (a=&b).

    >
    > The problem is that you are confusing implementation with semantics.
    > The meanings of 'variable', 'value', 'assignment' etc are defined by
    > computer science. Whether you arbitrarily change the meaning of
    > 'assignment' or whether you arbitrarily change the meaning of
    > 'variable', it amounts to the same thing.


    I don't see the arbitrary change. Python has one type of variable: A
    pointer that cannot itself be dereferenced. It's much more consistent
    then having 3 types of variables (in C++).

    >
    > >
    > >> Tell me one case where it is sensible for a function to behave such
    > >> that whether the caller sees a change in a variable it passed as its
    > >> argument should depend on the type.


    A function may see a change in a *value* not a variable. That should
    always depend on type and should depend on nothing else, a function
    should always know what types it's working on. In my C example above,
    it would depend on nothing that could be discernable in the function.

    If someone wrote a mutable integer class, then it would be a distinct
    type from an immutable integer, and the function could work with 3
    types: immutable int, mutable int, and int. If the function doesn't
    know what it's working with, then the function writer should probably
    look for another career.

    > You miss my point. Your argument would equally apply to the need for
    > an integer-specific division operator, for instance. Its all about how
    > one function reacts in response to varying input. If you pass a
    > mutable value to a function which expects an immutable one, you get an
    > error. If you pass an immutable value to a function which expects a
    > mutable one, you get an error. There is no good reason for a function
    > to react to the distinction as evidenced by your own observation that
    > what actually happens (and what should happen) is that you have two
    > distinct variants of the function.


    Not true. If you pass a mutable to a function that expects an
    immutable, you'll get no error. There's nothing that can be done to
    an immutable that can't be done to a mutable. The opposite would give
    an error, and should. You wouln't call a function in C++ with an
    object that has no object::doSomething() function if the function
    expected it. Whether an object is immutable or not, is no different
    than knowing what methods it supports. You wouldn't program against
    an object without knowing its methods, likewise you need to know if
    it's immutable.

    This dicussion brings to light the fact that immutable/mutable isn't
    really special in any way. It just means that the object has no
    methods to support mutability. Just like the classic Dog class
    inheriting from Animal. Animal has no doBark() method, but Dog does.
    A function user shouldn't be surprised that a method that expects a
    Dog would choke on an Animal.

    > >> Tell me one case where an object storing values should care about
    > >> callers mutating values it holds *only* for certain types.

    > >
    > >Objects don't *store* values, they *refer* to values. You are still
    > >thinking like you're in C (or C++). This is why you are having a
    > >problem.

    >
    > No it is not. I'm thinking in terms of computer science, in which
    > terms like 'variable', 'value' and 'assignment' are abstract concepts
    > independent of the way in which they are implemented in a programming
    > language.


    The distinction is more to do with Dynamically version Static
    languages. You're quoting the definitions for Static languages like
    they apply universally to all computer science. They don't. In C you
    can have a variable that's an int, float, pointer, struct, etc. In
    Python you only get name (which can be read binding, or "limited"
    pointer). It's when you start forcing the semantics of value types
    onto variables that you get confused. Variable, value, and
    assignement do mean different things in dynamic vs. static languages,
    and that's because they're sematically different. While they both use
    the = operator to perform similar functions, they're different, and if
    you expect them to be identical, you're going to be frustrated.

    > One way or the other, Python is currently choosing not to respect the
    > computer science definition of those terms. It may have historic and
    > backward-compatability reasons, but that does not change the facts.
    > This deviation from computer science definitions, whatever the excuse,
    > is arbitrary, confusing and error prone. That is my problem.


    It's not respecting the static language definitions of those terms,
    but it shouldn't even try. There are different meanings for those
    words in mathematics, but you wouldn't try to express those
    definitions in terms of C, it would be equally frustrating. The
    static definitions are merely abstractions of the mathematic meanings,
    and the dynamic meanings are also abstract, but in a different
    direction.

    Adam Ruth
     
    Adam Ruth, Jul 14, 2003
    #12
  13. On 13 Jul 2003 21:43:41 -0400, (Aahz) wrote:

    >In article <>,
    >Stephen Horne <> wrote:
    >>
    >>In computer science, a variable is a named binding to a value.
    >>Operations on that variable may rebind it to a different value, but
    >>values don't change behind your back. A pointer is, in effect, a value
    >>which indirectly refers to another (possibly anonymous) variable. A
    >>pointer is something you specifically request, and by doing so you
    >>allow that the 'variable' being indirectly referenced may be modified
    >>by something else that has no direct access to your pointer value (via
    >>your named variable or via copys or pointers-to your pointer held
    >>elsewhere).
    >>
    >>Python should respect that.

    >
    >That's why I (and others) prefer to use "name" and "target" instead of
    >"variable". Would that assuage your ire?


    Not really, for reasons defined elsewhere. The problem isn't
    theoretical - it is practical, as shown by the fact that people do
    trip over it time and time again. I'm just saying that the problem
    arises out of peoples intuition of how variables should work - an
    intuition that matches very well with theory.

    If there is a clear concept of the right thing expressed in theory
    which happens to match common intuitions, and if the wrong thing tends
    to lead to subtle errors in functions that act differently depending
    on whether the values passed in happen to be of mutable or immutable
    types, all I can do is refer back to the arguments used against me
    back in PEP238 when the majority *wanted* to make the massively
    incompatible change on the grounds of theory, intuition and
    reliability.
     
    Stephen Horne, Jul 14, 2003
    #13
  14. Stephen Horne <> writes:

    > Funny thing. When I use algebra, the variables I define don't end up
    > referring to different values, functions or whatever unless I
    > explicitly redefine them. When I write a definition on one piece of
    > paper, the things I wrote earlier on another sheet don't change.


    That, typically, is because you don't have any mutating operations in
    your formulae. All functions you use are side-effect-free. If you
    translated the sheets of paper literally into Python, the formulae
    would work the way you expect them to work.

    Regards,
    Martin
     
    Martin v. =?iso-8859-15?q?L=F6wis?=, Jul 14, 2003
    #14
  15. Ian Bicking <> writes:

    > Is there someplace in particular to reference people who need to learn
    > about what Python variables are (that they are bindings, etc)?


    I like

    http://starship.python.net/crew/mwh/hacks/objectthink.html

    which has the advantage of doing things the pictorial (my) way, and
    the verbal (Alex Martelli's) way.

    Cheers,
    M.

    --
    First time I've gotten a programming job that required a drug
    test. I was worried they were going to say "you don't have
    enough LSD in your system to do Unix programming". -- Paul Tomblin
    -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html
     
    Michael Hudson, Jul 14, 2003
    #15
  16. Ian Bicking

    Aahz Guest

    In article <>,
    Stephen Horne <> wrote:
    >
    >C++ has precisely one type of variable. That variable is a placeholder
    >for a value of a datatype which is specified in the declaration. The
    >datatype may be a pointer datatype, but so what? Pointer datatypes are
    >not treated any differently than other datatype except that they, like
    >all datatypes, they have their own set of functionality.


    Really? I would argue that the differing syntaxes is an argument
    against a single type of variable. What happens with this:

    char a;
    a->b;
    --
    Aahz () <*> http://www.pythoncraft.com/

    "Not everything in life has a clue in front of it...." --JMS
     
    Aahz, Jul 14, 2003
    #16
  17. On 13 Jul 2003 22:46:41 GMT, (Bengt Richter) wrote:
    [...]
    Re-reading, I see that I should probably fill in some details (still subject to
    disclaimer ;-)
    >
    >Python name assignment does not do anything to an object. It just creates
    >an alias name for an object. It may reuse an existing name, but that only
    >changes the meaning of that name as an alias, it does not affect the object
    >that the name was previously an alias for.
    >
    >You can of course have other assignment targets than plain names. In fact,
    >a "plain name" is just the simplest expression for an assignment target.
    >But whatever the target expression evaluates to, what gets put "there" is
    >a reference to an object, not an object itself.
    >
    >The confusion, ISTM, is in what to think of as "there." "There" is definitely
    >not a memory space for a Python object representation or "value".
    >
    >The most accurate (UIAM) C concept would be a "there" of type PyObject* -- i.e.,
    >a pointer variable that points to the representation of some kind of Python object.
    >

    I should have mentioned that there are various access mechanisms for storing the "pointer".
    I.e., the left hand side doesn't evaluate to a raw address until you are in the machine language
    of the implementation. When storage is actually happening, of course byte codes are being
    interpreted by the virtual machine of the current Python version. The byte codes will
    differ according to the translation of the target expression, and even for byte codes
    that looks the same, the implementation may involve complex dynamic behavior, e.g., in
    searches through an inheritance graph for an appropriate method that will accept the
    rh object reference and do the ultimate implementation-pointer storage.

    The byte codes with STORE in their names, and some typical source statements that generate them,
    are (2.2.3 windows):

    STORE_SLICE: x[a:b]=y # four byte codes numbered as STORE_SLICE+(1 if a is present)+(2 if b is present)
    STORE_SUBSCR: x[a]=y
    STORE_NAME: x=y #in global scope
    STORE_ATTR: x.a=y
    STORE_GLOBAL: def foo(): global x; x=y
    STORE_FAST: def foo(): x=y
    STORE_DEREF: def foo():
    x=y # the STORE_DEREF
    def bar():return x
    return bar

    Note that, e.g., STORE_SUBSCR looks the same for lists and dictionaries, or even something
    undetermined that will give a run time error because the type of x doesn't support that operation.

    Storage is often mediated by various methods, e.g., __setitem__ but finding the ultimate
    method may be the result of a search through an inheritance graph,
    and/or it could find the setter function of a named property, and that function in turn
    may do much work that is not visible in the assignment statement source, or the immediate
    level of corresponding byte codes.

    >We don't have to know the implementation details to use the idea that the left hand
    >side of a Python assignment is an *expression* (even if a single name) that yields
    >a place to put a reference (effectively a pointer) to the object created by evaluating
    >the expression on the right hand side.

    Again, I should have been clearer. It *ultimately* yields a place, but potentially deep
    in the bytecode and ultimately machine code of some chain of method invocations and implementations.
    >
    >A C programmer will tend to think that a symbol on the left is a static expression
    >indicating a specific fixed memory space (static address or stack frame offset). But in Python
    >it is first of all a dynamic expression (though an unqualified target name can only have its "there"
    >be in the local or global namespace, and the choice between those is made statically

    Hm. There is a third choice. When variables of the local namespace are destined to be captured
    in a closure, the "there" is in the closure, stored by way of STORE_DEREF.
    >(at least w.r.t. a given level of compiling/exec-ing).
    >
    >In C terms, the Python assignment target expression always evaluates to a place to put a pointer,

    I elided a fair abount of implementation detail in saying that, but I think the effective
    semantics are ok.

    >never a place to put object representation bits. Of course, "a place to put a pointer"
    >may be *inside* an object. And such inside places are identified by target expressions
    >such as x[2] or x.a or x[2]().b[3] etc. Even a bare name really does identify a place inside
    >an object -- i.e., inside the local or global dict associated with the context.
    >
    >The "place" identified by x= after a global x declaration (or just in global scope) will be the
    >same place as globals()['x']= unless someone has subverted something. Either way, the binding
    >of the name to the object happens by evaluating to a "there" within the global dict object,
    >uniquely associated with the name ('x' here). Evaluated on the right hand side, that name will
    >produce the reference/pointer again for use in accessing the object or copying to another "there"
    >associated with perhaps some other name or a target within a composite object like a list or tuple,
    >or other namespace dict.
    >
    >>
    >>The wart remains, even if my description was wrong. And even that is a
    >>dubious claim.
    >>

    >Every assignment is effectively stores a referential alias for an object,
    >whether associated with a name or not. This is by design, not a wart.
    >

    The details of *how* an assignment "effectively stores" is a longer story though ;-)

    >>A Python user is interested in how an object behaves - not how it is
    >>internally implemented in the interpreter. Immutable objects don't

    >when you say "immutable objects," are you really speaking of names
    >bound to immutable objects? I.e., the integer 123 is an immutable object
    >represented by the literal 123. You get an immutable object of the same value
    >from the expression int('123'). These are the int *objects* -- how does
    >"behave as references" apply to these actual "immutable objects"?
    >I.e., could you rewrite your sentence (that this is breaking in two)
    >to make it perhaps more understandable for me ?;-)
    >
    >Names are not objects. Nor are left-hand-side assignment target expressions
    >in general, whether bare names or complicated.
    >
    >ISTM what you are discussing is not about objects but about the way names work,
    >and ISTM you are thinking of names as if they were C++ variable references,
    >which they are not, and they couldn't be. For one thing, a C++ reference type
    >has to be initialized once to refer to a specific object. It can't be made to
    >refer to something else during the life of that scope. Pointers are a better
    >model, since you have to distinguish by expression syntax whether you mean
    >to assign a new pointer value or modify the thing pointed to. In python you
    >can only assign pointers, if you want to think in those terms. When you
    >modify a mutable, you are still assigning pointers into some part of the
    >mutable object representation. When you assign to a bare name, you are still assigning
    >a pointer into a place in some dictionary object (or workalike). If you
    >want to modify an object in the usual C sense, you can code it in C and provide a
    >python interface to your C-implemented object. When you pass arguments
    >to the various methods of your mutable, you will get pointers that you
    >can dereference and you can do what you want to your mutable's C data representation.
    >But when you pass the result back to the python world, it will have to be
    >as a standard object reference/pointer, and if you assign that, you will be
    >storing a pointer somewhere.
    >
    >>behave as references - the internal use of references for immutable
    >>objects is basically a lazy copying optimisation and, apart from
    >>performace and a couple of other technicalities (e.g. the 'is'
    >>operator), has no relevance. Certainly it has no relevance to the
    >>point I was making.
    >>

    >The way it works is part of the semantics of the language, not just
    >an optimization issue.
    >
    >Names aren't variables.
    >
    >HTH ;-)
    >
    ><disclaimer>I sometimes get things wrong. Corrections welcome.</disclaimer>
    >


    Regards,
    Bengt Richter
     
    Bengt Richter, Jul 14, 2003
    #17
  18. Ian Bicking

    Andrew Dalke Guest

    Moshe Zadka:
    > In computer science, the theoretical model one first learns is a turing
    > machine. Turing machines have no variables or values, merely tapes.


    Actually, we started off with state machine, then worked our way though
    DFA/NFA, PDA, etc., ending up with a TM. I remember being intrigued
    that the way we learned these was in opposite order to they way they
    were first researched.

    Can anyone suggest why? My thought was that mathematicians
    like generalizing, so the most general models were pondered first.
    Only as people started to experiment with variations, and with actual
    computer hardware to back them up, did people start thinking about
    more limited cases.

    Andrew
     
    Andrew Dalke, Jul 14, 2003
    #18
  19. Ian Bicking

    Adam Ruth Guest

    Stephen Horne <> wrote in message news:<>...
    > On 13 Jul 2003 21:03:59 -0700, (Adam Ruth) wrote:
    >


    > C++ has precisely one type of variable. That variable is a placeholder
    > for a value of a datatype which is specified in the declaration. The
    > datatype may be a pointer datatype, but so what? Pointer datatypes are
    > not treated any differently than other datatype except that they, like
    > all datatypes, they have their own set of functionality.


    Granted. Pointers are no different than other data types, but they
    are typically used for operations that are semantically very different
    than other datatypes are used for. In that sense, they are, at a high
    level, different data types.

    It's like how c programming is taught has having pass by reference and
    pass by value, when there is only pass by value at the implementation
    level. Pass by reference is a concept added on top of the language.

    > C++ references are tellingly also called self-dereferencing pointers.
    > They are not a distinct concept - they are syntactic sugar. I suspect
    > they mainly arise out of the modern desire to disguise pointers and
    > fantasize that they're not there, though they certainly work very well
    > in certain contexts.


    Syntactic sugar or no, they still behave differently than other
    datatypes and are therefore not consistent... IMHO.

    > Funny thing. When I use algebra, the variables I define don't end up
    > referring to different values, functions or whatever unless I
    > explicitly redefine them. When I write a definition on one piece of
    > paper, the things I wrote earlier on another sheet don't change.
    >
    > Seems to me that the math equivalent of assignment (defining named
    > things) works very much like the 'static language definitions' as you
    > put it.


    The devil is in the details. Math assignment is static assignment is
    dynamic assignment. They really are all the same thing at a high
    level, but it's the implementation and the subtleties that make them
    vary.

    Adam Ruth
     
    Adam Ruth, Jul 14, 2003
    #19
  20. Ian Bicking

    Moshe Zadka Guest

    [Moshe Zadka]
    > In computer science, the theoretical model one first learns is a turing
    > machine. Turing machines have no variables or values, merely tapes.


    [Andrew Dalke]
    > Actually, we started off with state machine, then worked our way though
    > DFA/NFA, PDA, etc., ending up with a TM.


    Obviously, I meant "first turing-equivalent". Of course, whether this
    is true also depends: in CS "Computability" this is true, but
    in Math, we covered this in "Logic 2", and there we did, IIRC,
    recursive functions and counting machines first. Counting machines
    are mostly useful as something on the way to prove recursive functions
    are turing complete.

    > I remember being intrigued
    > that the way we learned these was in opposite order to they way they
    > were first researched.
    >
    > Can anyone suggest why? My thought was that mathematicians
    > like generalizing, so the most general models were pondered first.
    > Only as people started to experiment with variations, and with actual
    > computer hardware to back them up, did people start thinking about
    > more limited cases.


    Well, I think that it's natural to study DFAs first, as a turing
    machine is a DFA with a tape, so it's useful to know *why* the
    tape is needed. Also, DFAs are a good explanation of "what are
    computers", as a computer is necessarily finite.
    --
    Moshe Zadka -- http://moshez.org/
    Buffy: I don't like you hanging out with someone that... short.
    Riley: Yeah, a lot of young people nowadays are experimenting with shortness.
    Agile Programming Language -- http://www.python.org/
     
    Moshe Zadka, Jul 14, 2003
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Dave Brueck

    Re: anything like C++ references?

    Dave Brueck, Jul 13, 2003, in forum: Python
    Replies:
    4
    Views:
    370
    =?ISO-8859-1?Q?Hannu_Kankaanp=E4=E4?=
    Jul 14, 2003
  2. David McNab

    Re: anything like C++ references?

    David McNab, Jul 13, 2003, in forum: Python
    Replies:
    19
    Views:
    522
    Christos TZOTZIOY Georgiou
    Jul 18, 2003
  3. Tim Peters

    RE: anything like C++ references?

    Tim Peters, Jul 13, 2003, in forum: Python
    Replies:
    44
    Views:
    1,059
    Tim Roberts
    Jul 20, 2003
  4. Michael Chermside

    RE: anything like C++ references?

    Michael Chermside, Jul 14, 2003, in forum: Python
    Replies:
    2
    Views:
    283
    Michael Hudson
    Jul 15, 2003
  5. Replies:
    5
    Views:
    328
    Stephen Horne
    Jul 17, 2003
Loading...

Share This Page