RE: Pass-by-reference : Could a C#-like approach work in Python?

Discussion in 'Python' started by Michael Chermside, Sep 11, 2003.

  1. First, thanks to Stephen Horne for bringing up this subject. I learn
    more about language design in this newsgroup than anywhere else
    (perhaps people who appreciate good language design gravitate to
    Python?).

    I started by looking for use cases... where would this proposed
    pass-by-reference actually prove useful? Stephen basically found
    one good use case:

    I wish to "transfer ownership" of an object that gets passed
    into a method call, and protect against the caller accidently
    continuing to use the object. [So, we'll use a "ref" parameter and
    set the caller's variable to None.]

    But "pass by reference" isn't the first idea I have on reading the
    use case. I agree with Peter Otten, who writes:

    > if copying is costly and only
    > sometimes necessary, wrap the item into a copy-on-write proxy for every
    > logically distinct but physically identical instance.


    Excellent idea! But, how do I write a copy-on-write proxy in Python?
    Seems like this would make an excellent cookbook recipie, but I couldn't
    figure out how to do it. Proxying is easy, but how does one distinguish
    between mutating and non-mutating methods? Does the proxy need to be
    customized for each class to be wrapped?

    Also I wanted to mention what _I_ saw as the major reason why including
    reference parameters in Python would be a bad idea. Stephen himself
    pointed out that without compelling use cases, YAGNI should rule. But
    the counter argument of "it will allow unexpected changes whenever I
    invoke a method" is completely bogus -- from the beginning Stephen
    pointed out that special syntax should be required BY THE CALLER. And
    the counter argument of "it will slow things down" is dubious at best
    (cPython method calls are already so slow that it's unclear how much
    difference this would make).

    But my counter argument would be that this breaks one of the elegent
    simplicities of Python -- that callables of all kinds are equivalent
    and have a very simple interface. A callable receives (in effect)
    one tuple of values, and one map of values. That's it. That's a very
    simple protocol. Functions and methods are written with a bunch of
    ordered, named parameters, (some of which may have defaults), that
    are filled in from this tuple and map, and perhaps an overflow tuple
    and map for additional inputs. There's no place in this simple
    protocol for type declarations (of course), and there would be no
    place for tracking whether an argument was "ref" or not. To add even
    that one boolean property would significantly complicate callables.

    I think that having a small number of protocols that are very simple
    yet very powerful is a wonderful way to design a language. Python's
    callables are very simple, yet powerful. Iterators are another such
    example. Making classes consist primarily of a namespace (__dict__)
    was another, although that has gotten much more complicated as
    things like descriptors are introduced.

    Anyway, just wanted to share some thoughts you had triggered. Thanks
    for the ideas.

    I'd-cc-Stephen-Horne-directly-but-it'd-be-rejected-as-a-Nigerian-scam lly,

    -- Michael Chermside
     
    Michael Chermside, Sep 11, 2003
    #1
    1. Advertising

  2. On Thu, 11 Sep 2003 10:22:22 -0700, Michael Chermside
    <> wrote:

    >> if copying is costly and only
    >> sometimes necessary, wrap the item into a copy-on-write proxy for every
    >> logically distinct but physically identical instance.

    >
    >Excellent idea! But, how do I write a copy-on-write proxy in Python?


    I suspect this is one of those things which is doable, but you end up
    having to write a specialised proxy for each case (ie each class being
    proxied). Depending on how many methods you need to support, it may
    not be that much of a headache of course.

    There are other ways of doing it that don't need a change to Python,
    though - I guess the particular method will probably depend on
    particular cases.

    >To add even
    >that one boolean property would significantly complicate callables.


    Absolutely true - it should only for affect calls with at least one
    'ref' parameter, but even then, without a compelling use case this
    more than enough reason to reject the idea.

    >I'd-cc-Stephen-Horne-directly-but-it'd-be-rejected-as-a-Nigerian-scam lly,


    You'd have a hard job - just noticed my sig is missing from my posts,
    and my e-mail address deliberately isn't in the header field.

    I'll fix the sig in a minute, when I can face the newsreaders
    settings, but if its needed my email is steve at ninereeds dot fsnet
    dot co dot uk.
     
    Stephen Horne, Sep 11, 2003
    #2
    1. Advertising

  3. Michael Chermside

    Peter Otten Guest

    Michael Chermside wrote:

    >>> if copying is costly and only

    >> sometimes necessary, wrap the item into a copy-on-write proxy for every
    >> logically distinct but physically identical instance.

    >
    > Excellent idea! But, how do I write a copy-on-write proxy in Python?
    > Seems like this would make an excellent cookbook recipie, but I couldn't
    > figure out how to do it. Proxying is easy, but how does one distinguish
    > between mutating and non-mutating methods? Does the proxy need to be
    > customized for each class to be wrapped?
    >


    I have to admit that the generic copy-on-write proxy is still vaporware :-(
    However, I've come up with a way to make a class its own proxy, with
    relatively small overhead and no need for advanced stuff like metaclasses
    (which I have yet to master).

    class COW(object):
    def __init__(self, template=None):
    self._template = template
    def __getattribute__(self, name):
    try:
    return object.__getattribute__(self, name)
    except AttributeError:
    return getattr(self._template, name)
    def __str__(self):
    return "--%s---\n" % self._name + "\n".join(["%s = %r" % (n,
    getattr(self, n)) for n in "a b".split()])
    def proxy(self):
    return COW(self)

    def printBoth():
    print "%s\n\n%s\n---\n" % (o1, o2)

    o1 = COW()
    o1.a = "default A"
    o1.b = "default B"
    o1._name = "Obj1"

    o2 = o1.proxy() # same as o2 = COW(o1)
    o2._name = "Obj2"
    printBoth()

    o1.a = "override" # seen in both o1 and o2
    printBoth()

    o2.a = "shade" # hides o1.a
    printBoth()

    del o2.a # o1.a visible again
    printBoth()

    print o2.nonExistent # misleading error message

    All methods are identical for proxy and original by design. Attributes are
    rather shaded than copied, and thus there is no need to copy the whole
    attribute set if two instances differ only in a single attribute.
    The overhead in the class definition is not that large: one method plus one
    attribute.

    There are some loose ends:
    - Should mutable attributes be recursively proxied?
    - How does the above behave in an inheritance tree?
    - Performance may suffer as the "proxy chain" grows
    - I've got a hunch that one could come up with an even more general/less
    intrusive solution using metaclasses

    Always-speculatingly yours,
    Peter
     
    Peter Otten, Sep 11, 2003
    #3
  4. Michael Chermside

    Duncan Booth Guest

    Michael Chermside <> wrote in
    news::

    > I started by looking for use cases... where would this proposed
    > pass-by-reference actually prove useful? Stephen basically found
    > one good use case:
    >
    > I wish to "transfer ownership" of an object that gets passed
    > into a method call, and protect against the caller accidently
    > continuing to use the object. [So, we'll use a "ref" parameter and
    > set the caller's variable to None.]


    That only works if the caller hasn't assigned that object to any other
    variables. If they have, they can still accidentally continue to use the
    object. Since you have to trust the caller anyway, why not trust them to
    stop using the object?

    --
    Duncan Booth
    int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
    "\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?
     
    Duncan Booth, Sep 12, 2003
    #4
  5. Michael Chermside

    Paul Foley Guest

    On Tue, 23 Sep 2003 16:52:41 +1200, using news cis dfn de wrote:

    > Michael Chermside wrote:
    >> But my counter argument would be that this breaks one of the elegent
    >> simplicities of Python -- that callables of all kinds are equivalent
    >> and have a very simple interface. ... There's no place in this simple
    >> protocol for type declarations (of course), and there would be no
    >> place for tracking whether an argument was "ref" or not.


    > Apologies for reviving this thread, but I'd just like to
    > point out how this proposal ("ref" parameters, for anyone
    > who's forgotten) could be implemented without adding any
    > complication to the argument passing model. In fact, it
    > could be decoupled from argument passing almost completely.


    > Suppose there were an expression


    > ref x


    > which would return what I will call a Namespace Reference
    > Object (NRO) which refers to the binding of the name 'x'
    > in the current scope. It would have some suitable protocol,
    > such as get() and set() methods, for accessing this binding.
    > Note that this would be a general expression, usable anywhere,
    > not just in a function call.


    I think that's just what I said in <>
    [complete with an implementation in CL; can't do it in Python because
    lambda is too broken (and there's no way to add syntactic support)]

    --
    Cogito ergo I'm right and you're wrong. -- Blair Houghton

    (setq reply-to
    (concatenate 'string "Paul Foley " "<mycroft" '(#\@) "actrix.gen.nz>"))
     
    Paul Foley, Sep 23, 2003
    #5
  6. Michael Chermside wrote:
    > But my counter argument would be that this breaks one of the elegent
    > simplicities of Python -- that callables of all kinds are equivalent
    > and have a very simple interface. ... There's no place in this simple
    > protocol for type declarations (of course), and there would be no
    > place for tracking whether an argument was "ref" or not.


    Apologies for reviving this thread, but I'd just like to
    point out how this proposal ("ref" parameters, for anyone
    who's forgotten) could be implemented without adding any
    complication to the argument passing model. In fact, it
    could be decoupled from argument passing almost completely.

    Suppose there were an expression

    ref x

    which would return what I will call a Namespace Reference
    Object (NRO) which refers to the binding of the name 'x'
    in the current scope. It would have some suitable protocol,
    such as get() and set() methods, for accessing this binding.
    Note that this would be a general expression, usable anywhere,
    not just in a function call.

    Thats' really all that is strictly needed, because we can
    now write

    def Inc(xref):
    xref.set(xref.get() + 1)

    and call it with

    Inc(ref a)

    However, as a convenience, we might want to allow the 'ref'
    keyword to be usable on the left hand side of an expression
    as well, in a kind of "unpacking" sense, so that

    ref x = xref
    x = y

    would be equivalent to

    xref.set(y)

    (The semantics of this are perhaps a bit murky, but would
    be something line "assign xref to x, and have the
    compiler note that any other mention of x in this scope
    is to use the NRO protocol implicitly".)

    Now we can write

    def Inc(xref):
    ref x = xref
    x += 1

    Further, analogous to the way tuples can be implicitly
    unpacked in an argument list, we should similarly allow
    arguments to be implicitly "unrefed" in an argument list:

    def Inc(ref x):
    x += 1

    This is identical to the original proposal, but is implemented
    using mechanisms which are completely orthogonal to argument
    passing, and could be used in other contexts if desired.

    Not that I'm necessarily advocating the proposal -- I tend
    to agree that there's no pressing need for passing arguments
    by reference in Python. I'm just setting down these ideas
    for posterity.

    Okay, you can let this thread die properly now. :)

    --
    Greg Ewing, Computer Science Dept,
    University of Canterbury,
    Christchurch, New Zealand
    http://www.cosc.canterbury.ac.nz/~greg
     
    Greg Ewing (using news.cis.dfn.de), Sep 23, 2003
    #6
  7. Greg Ewing (using news.cis.dfn.de) wrote:
    > However, as a convenience, we might want to allow the 'ref'
    > keyword to be usable on the left hand side of an expression
    > as well, in a kind of "unpacking" sense, so that
    >
    > ref x = xref
    > x = y
    >
    > would be equivalent to
    >
    > xref.set(y)


    You're moving too close to overloadable assignment for my taste (as a
    magical __assign__ method wouldn't even require the ref syntax). Don't
    mention this when impressionable kids are around.

    Daniel
     
    Daniel Dittmar, Sep 23, 2003
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jerry
    Replies:
    20
    Views:
    7,950
    Roedy Green
    Sep 9, 2005
  2. Stephen Horne
    Replies:
    37
    Views:
    893
    Stephen Horne
    Sep 13, 2003
  3. blufox
    Replies:
    2
    Views:
    563
  4. Mr A
    Replies:
    111
    Views:
    2,128
  5. Robert
    Replies:
    10
    Views:
    1,370
    E. Robert Tisdale
    Aug 24, 2005
Loading...

Share This Page