Re: Verbose and flexible args and kwargs syntax

Discussion in 'Python' started by Terry Reedy, Dec 12, 2011.

  1. Terry Reedy

    Terry Reedy Guest

    On 12/12/2011 4:12 AM, Eelco Hoogendoorn wrote:
    >> The above examples are seldom needed in Python because we have one
    >> general method to repeatedly split a sequence into head and tail.

    >
    >> it = iter(iterable) # 'it' now represents the sequenced iterable
    >> head = next(it) # 'it' now represents the tail after removing the head

    >
    >> In other words, next(it) encompasses all of your examples and many more.
    >> Because 'it' is mutated to represent the tail, it does not need to be
    >> rebound and therefore is not.

    >
    >
    > The question in language design is never 'could we do these things
    > before'. The answer is obvious: yes our CPUs are turing complete; we can
    > do anything. The question is; how would we like to do them?
    >
    > So do you think the new head/tail unpacking features in python 3 are
    > entirely uncalled for?


    No, *target unpacking (singular) is quite useful in specialized cases.
    But it is not specifically head/tail unpacking.

    >>> a,*b,c = 1,2,3,4,5,6
    >>> a,b,c

    (1, [2, 3, 4, 5], 6)
    >>> *a,b,c = 1,2,3,4,5
    >>> a,b,c

    ([1, 2, 3], 4, 5)

    > I personally quite like them, but I would like them to be more general.


    It already is. The *target can be anywhere in the sequence.

    --
    Terry Jan Reedy
     
    Terry Reedy, Dec 12, 2011
    #1
    1. Advertising

  2. Terry Reedy

    Eelco Guest


    > > I personally quite like them, but I would like them to be more general.

    >
    > It already is. The *target can be anywhere in the sequence.


    Im not sure if this is a genuine understanding, or trollish
    obtuseness.

    Yes, the target can be anywhere in the sequence. And yes, the
    resulting list can contain objects of any type, so its very flexible
    in that regard too.

    But to relate it to the topic of this thread: no, the syntax does not
    allow one to select the type of the resulting sequence. It always
    constructs a list.

    Yes, we can cast the list to be whatever we want on the next line, but
    the question is whether this language design can be improved upon. The
    choice of a list feels arbitrary, adding another line to cast it to
    something else would be even more verbose, and whats more, there would
    be serious performance implications if one should seek to apply this
    pattern to a deque/linkedlist; it would make taking off the head/tail
    of the list from a constant to a linear operation.

    That is:

    >>> head, deque(tail) = somedeque


    Is better in every way I can think of (readability, consistence,
    performance) than:

    >>> head, *tail = somedeque
    >>> tail = deque(tail)
     
    Eelco, Dec 12, 2011
    #2
    1. Advertising

  3. Terry Reedy

    Terry Reedy Guest

    On 12/12/2011 12:12 PM, Eelco wrote:

    > Im not sure if this is a genuine understanding, or trollish
    > obtuseness.


    If you are referring to what I write, it is based on genuine
    understanding of Python.

    > Yes, the target can be anywhere in the sequence. And yes, the
    > resulting list can contain objects of any type, so its very flexible
    > in that regard too.
    >
    > But to relate it to the topic of this thread: no, the syntax does not
    > allow one to select the type of the resulting sequence. It always
    > constructs a list.


    One use case of *target is to ignore the stuff collected in the target
    because one only wants a few end values from the iterable. Another is to
    pull stuff out because one wants to iterate through the rest. For both
    uses, a list is as good as anything.

    > Yes, we can cast the list to be whatever we want on the next line,


    Convert. For the very few cases one wants to do this, it is quite adequate.

    > but the question is whether this language design can be improved upon.


    Not easily.

    > The choice of a list feels arbitrary,


    On the contrary, a list is precisely what is needed to collect an
    indefinite number of leftovers.

    > adding another line to cast it to
    > something else would be even more verbose, and whats more, there would
    > be serious performance implications if one should seek to apply this
    > pattern to a deque/linkedlist; it would make taking off the head/tail
    > of the list from a constant to a linear operation.


    For a linked list, no *target and no copying is needed:

    head, tail = llist

    >>>> head, deque(tail) = somedeque

    >
    > Is better in every way I can think of (readability, consistence,
    > performance) than:


    >>>> head, *tail = somedeque
    >>>> tail = deque(tail)


    But your suggestion is much worse in each way than

    head = somedeque.popleft()

    To repeat, there is no reason to copy even once. If one does not want to
    mutate the deque, then one mutates an iterator view of the deque.

    --
    Terry Jan Reedy
     
    Terry Reedy, Dec 12, 2011
    #3
  4. Terry Reedy

    Eelco Guest

    > > Im not sure if this is a genuine understanding, or trollish
    > > obtuseness.

    >
    > If you are referring to what I write, it is based on genuine
    > understanding of Python.


    This is getting 'interesting'. In a way. I meant to write
    'misunderstanding', as I think the context made quite clear. So again
    this adds another layer of doubt as to whether you are being obtuse
    for its own sake, or if there is yet another layer of misunderstanding
    stacked on top of the others. Either way, lets continue with the
    benefit of the doubt.

    > One use case of *target is to ignore the stuff collected in the target
    > because one only wants a few end values from the iterable. Another is to
    > pull stuff out because one wants to iterate through the rest. For both
    > uses, a list is as good as anything.


    So what is the point of having different sequence types, if a list can
    do anything? I would argue that the different performance
    characteristics and differences in semantics give each sequence type
    ample reason for existence.

    > Convert. For the very few cases one wants to do this, it is quite adequate.


    Or so you opine. As you may recall, the original idea was motivated by
    args/kwargs syntactic clarity and flexibility; I merely shifted focus
    to this use case of collection unpacking since it is somewhat simpler
    and easier to discuss. Keep that in mind, should you seek to build a
    case for that assertion in the future.

    >  > but the question is whether this language design can be improved upon.
    >
    > Not easily.


    Again, so you opine. Care to give a refutation of my proposed double
    colon syntax? Ignoring for a moment whether or not it is useful; does
    it clash with anything? I havnt been able to think of anything in the
    past few hours, but im not taking that to mean much; there are
    probably some subtleties I am overlooking. I think it dos not clash
    with slicing syntax for instance, but im not sure.

    > For a linked list, no *target and no copying is needed:
    >
    > head, tail = llist


    I have no idea what this means.

    > >>>> head, deque(tail) = somedeque

    >
    > > Is better in every way I can think of (readability, consistence,
    > > performance) than:
    > >>>> head, *tail = somedeque
    > >>>> tail = deque(tail)

    >
    > But your suggestion is much worse in each way than
    >
    > head = somedeque.popleft()


    No its not. First of all, its not semantically equivalent; popleft
    mutates a collection type, and collection unpacking does not; it
    creates a set of disjoint views of the collection's data. Whether its
    worse in terms of readability is debatable, but in terms of the other
    two stated metrics, your claim of it being any kind of worse is
    objectively false.

    Furthermore, this brings us back again to the point I raised several
    times before. Yes, obviously you can already DO these things, but NOT
    within the same uniform collection unpacking syntactic construct.
    Again, you have failed to point out what is wrong with supplying a
    type constrain to python and let it do the right thing based on that;
    to reiterate:

    head, tail::deque = deque

    No need to mutate anything; python can create the view of the
    linkedlist internally. A single unambigious syntax, and single
    unambigious semantics, for all sequence types. Whats not to like?

    If you dont like the extra characters you have to type; there is of
    course such a thing as defaults. You can choose:
    head, tail:: = deque; if the type constraint is omitted, we could make
    tail a list by default, or my preferred solution, infer it from the
    right hand side type. In case of the former, all you had to do was
    type :: instead of *, and your python universe would otherwise be
    completely unchanged. If thats 'not easily', I dont know what is.

    > To repeat, there is no reason to copy even once. If one does not want to
    > mutate the deque, then one mutates an iterator view of the deque.


    And arrive at yet another construction to do the same thing.
     
    Eelco, Dec 12, 2011
    #4
  5. Terry Reedy

    alex23 Guest

    On Dec 13, 3:12 am, Eelco <> wrote:
    > But to relate it to the topic of this thread: no, the syntax does not
    > allow one to select the type of the resulting sequence. It always
    > constructs a list.


    So by this argument, _every_ function that returns a list should take
    an optional argument to specify an alternative form of sequence.

    What, exactly, is so onerous about coercing your list to _whatever_
    type you want? You know, like everybody else has been.

    What does this _gain_ you other than one less line of code?
     
    alex23, Dec 13, 2011
    #5
  6. On Mon, 12 Dec 2011 16:58:33 -0500, Terry Reedy wrote:

    > On 12/12/2011 12:12 PM, Eelco wrote:


    >> Yes, we can cast the list to be whatever we want on the next line,

    >
    > Convert. For the very few cases one wants to do this, it is quite
    > adequate.


    +1

    with the small proviso that there's not much you can usefully do with a
    dict. If you convert to an ordered dict, the initial order is lost by the
    time you see it; if you want to allow duplicates, duplicates are likewise
    lost (or in a function call, an exception is raised).

    In Python 4000, I'd like to see some way to pass arbitrary parameters to
    functions, including duplicates. That is, emulating the richness of
    command line argument handling in functions.

    In Python 3.x, I'd like to see OrderedDict become a built-in, and then
    **kwargs can collect named arguments in an ordered dict without losing
    the order.



    --
    Steven
     
    Steven D'Aprano, Dec 13, 2011
    #6
  7. Terry Reedy

    Ian Kelly Guest

    On Mon, Dec 12, 2011 at 4:40 PM, Eelco <> wrote:
    >> For a linked list, no *target and no copying is needed:
    >>
    >> head, tail = llist

    >
    > I have no idea what this means.


    Each node of a linked list consists of a data member and a "next"
    member, that holds the next node in the list. The natural container
    for this and the way it is usually implemented in Python is a
    2-element tuple. For example:

    llist = ('one', ('two', ('three', ('four', None))))

    would be a linked list as typically constructed in Python, and it can
    be used in pretty much the same way that lists are used in Lisp. The
    result of the operation:

    head, tail = llist

    is:

    head = 'one'
    tail = ('two', ('three', ('four', None)))

    and as Terry pointed out, no copying is required to do this. I
    believe deque is also implemented as a doubly-linked list, which is
    probably why you keep referring to it as such, but that is an
    implementation detail. When you say "linked list" in relation to
    Python, the preceding is what comes to mind.

    >> >>>> head, deque(tail) = somedeque

    >>
    >> > Is better in every way I can think of (readability, consistence,
    >> > performance) than:
    >> >>>> head, *tail = somedeque
    >> >>>> tail = deque(tail)

    >>
    >> But your suggestion is much worse in each way than
    >>
    >> head = somedeque.popleft()

    >
    > No its not. First of all, its not semantically equivalent; popleft
    > mutates a collection type, and collection unpacking does not; it
    > creates a set of disjoint views of the collection's data. Whether its
    > worse in terms of readability is debatable, but in terms of the other
    > two stated metrics, your claim of it being any kind of worse is
    > objectively false.


    I definitely disagree on readability. Skimming this thread as I have
    been, it took me a while to get that your proposed syntax would create
    a second deque sharing internal state with the original, rather than
    creating a simple copy of the deque, which is what it looks like it
    should do.

    Incidentally, while that change is not really central to your syntax
    proposal, I think it would be very messy. For example, how do you
    propose keeping the length elements in sync? Inserting an item in one
    may or may not affect the length of the other. If I append an item to
    the end of one deque, should the other automatically be extended as
    well? What if the tail node of the second deque occurs after the tail
    node of the deque being appended? Does the appended element then get
    inserted into the middle of the second deque (I think it would have to
    be)? If I insert an element into the longer (second) deque that just
    happens to be immediately after the tail of the shorter deque, does
    *that* cause the shorter deque to be automatically extended? And
    likewise for operations at the head of the deque.

    None of these questions have obvious answers as to the "right" way to
    it, and for that reason I think this is probably best left to the user
    to implement a custom deque view class with whatever semantics they
    prefer.

    > Furthermore, this brings us back again to the point I raised several
    > times before. Yes, obviously you can already DO these things, but NOT
    > within the same uniform collection unpacking syntactic construct.
    > Again, you have failed to point out what is wrong with supplying a
    > type constrain to python and let it do the right thing based on that;
    > to reiterate:
    >
    > head, tail::deque = deque


    Python users generally follow the rule "explicit is better than
    implicit". Setting a general constraint and letting the language "do
    the right thing" is a kind of black magic that feels off because it
    tends to break that rule. But that's not to say that black magic
    never wins -- just look at super() and the MRO.

    > If you dont like the extra characters you have to type; there is of
    > course such a thing as defaults. You can choose:
    > head, tail:: = deque; if the type constraint is omitted, we could make
    > tail a list by default, or my preferred solution, infer it from the
    > right hand side type.


    I prefer the former. If it always creates a list by default, then the
    programmer reading this code three years later knows exactly what the
    type of tail is. If it instead infers it from the sequence being
    unpacked, then the programmer is forced to infer it as well, and it
    won't necessarily be obvious or even consistent.

    Cheers,
    Ian
     
    Ian Kelly, Dec 13, 2011
    #7
  8. Terry Reedy

    Eelco Guest

    On Dec 13, 1:27 am, alex23 <> wrote:
    > On Dec 13, 3:12 am, Eelco <> wrote:
    >
    > > But to relate it to the topic of this thread: no, the syntax does not
    > > allow one to select the type of the resulting sequence. It always
    > > constructs a list.

    >
    > So by this argument, _every_ function that returns a list should take
    > an optional argument to specify an alternative form of sequence.
    >
    > What, exactly, is so onerous about coercing your list to _whatever_
    > type you want? You know, like everybody else has been.
    >
    > What does this _gain_ you other than one less line of code?


    1) Turning two lines of code into a single more readable one is
    nothing to scoff at
    2) After-the-fact conversion is O(n), getting the view you want right
    away is O(1)

    Not every function needs this flexibility; many specialized functions
    could not care less. But collection unpacking is quite a general
    thing, and for the record; slicing a tuple returns a tuple. Would you
    rather have that return a list too?
     
    Eelco, Dec 13, 2011
    #8
  9. Terry Reedy

    Eelco Guest

    > > Python users generally follow the rule "explicit is better than
    > > implicit".  Setting a general constraint and letting the language "do
    > > the right thing" is a kind of black magic that feels off because it
    > > tends to break that rule.  But that's not to say that black magic
    > > never wins -- just look at super() and the MRO.

    >
    > We are not talking black magic here; we are talking about an EXPLICIT
    > type constraint provided on the very same line.


    To hammer that point home: would you say the following C# code
    performs 'black magic'?

    float x = 3;

    After all, exactly the same thing is going on; C# will 'do the right
    thing'; convert the integer literal 3 to a floating point value, based
    on the type constraint placed on x.
     
    Eelco, Dec 13, 2011
    #9
  10. On 13Dec2011 00:30, Eelco <> wrote:
    | On Dec 13, 1:27 am, alex23 <> wrote:
    | > On Dec 13, 3:12 am, Eelco <> wrote:
    | > > But to relate it to the topic of this thread: no, the syntax does not
    | > > allow one to select the type of the resulting sequence. It always
    | > > constructs a list.
    | >
    | > So by this argument, _every_ function that returns a list should take
    | > an optional argument to specify an alternative form of sequence.
    | >
    | > What, exactly, is so onerous about coercing your list to _whatever_
    | > type you want? You know, like everybody else has been.
    | >
    | > What does this _gain_ you other than one less line of code?
    |
    | 1) Turning two lines of code into a single more readable one is
    | nothing to scoff at
    | 2) After-the-fact conversion is O(n), getting the view you want right
    | away is O(1)

    Regarding (2), it has already cost you O(n) to get there. So your O(1)
    is a little ingenuous.
    --
    Cameron Simpson <> DoD#743
    http://www.cskk.ezoshosting.com/cs/

    I'm a volunteer EMT-I on our local ambulance service; one of our Paramedics
    calls squid style motorcycle accidents "ZoomSplats", as in "we had a good
    ZoomSplat the other night". ;-)
    - Scott <>
     
    Cameron Simpson, Dec 13, 2011
    #10
  11. Terry Reedy

    Eelco Guest

    On Dec 13, 8:11 pm, Cameron Simpson <> wrote:
    > On 13Dec2011 00:30, Eelco <> wrote:
    > | On Dec 13, 1:27 am, alex23 <> wrote:
    > | > On Dec 13, 3:12 am, Eelco <> wrote:
    > | > > But to relate it to the topic of this thread: no, the syntax does not
    > | > > allow one to select the type of the resulting sequence. It always
    > | > > constructs a list.
    > | >
    > | > So by this argument, _every_ function that returns a list should take
    > | > an optional argument to specify an alternative form of sequence.
    > | >
    > | > What, exactly, is so onerous about coercing your list to _whatever_
    > | > type you want? You know, like everybody else has been.
    > | >
    > | > What does this _gain_ you other than one less line of code?
    > |
    > | 1) Turning two lines of code into a single more readable one is
    > | nothing to scoff at
    > | 2) After-the-fact conversion is O(n), getting the view you want right
    > | away is O(1)
    >
    > Regarding (2), it has already cost you O(n) to get there. So your O(1)
    > is a little ingenuous.


    Well, yes, but if one takes a given sequence as input (at least O(n)
    complexity to obtain it in the first place, indeed), and then wants
    to, say, recursively unwind it, the cost of the total operation is
    O(n) versus O(n^2)

    And besides, O(n) < 2*O(n); perhaps of lesser concern than different
    orders, but still.
     
    Eelco, Dec 13, 2011
    #11
  12. Terry Reedy

    Eelco Guest

    On Dec 13, 7:15 pm, Ian Kelly <> wrote:
    > On Tue, Dec 13, 2011 at 1:44 AM, Eelco <> wrote:
    > > 'for i in llist' is not quite going to fly is it? Thats probably the
    > > reason noone ever uses that construct; its not a proper sequence type.

    >
    > Not really a problem, because fortunately Python makes it super-easy
    > to create custom iterators.
    >
    > def listiter(llist):
    >     while llist:
    >         head, llist = llist
    >         yield head
    >
    > And you're done.  If you like, you could also wrap this up in a class
    > with all the sequence-related magic methods you want, although you
    > lose the simplicity of the literal syntax by doing so.  If this is
    > rarely used, it is more likely because custom containers are going to
    > be less efficient than built-ins, but if you want to do functional
    > programming and are not overly concerned with the speed of iteration,
    > this is not a bad way to do it.


    Fair enough. Still, I dont readily see myself using the construct, and
    I do feel myself longing for something like that to be included as a
    builtin collection type. Im not sure why python is so stingy with the
    collections it provides.

    > > Good point. Copy-on-write semantics could be used, but really one
    > > should have several linked list types reflecting the underlying
    > > implementations. I would like to have an immutable singly linked list
    > > builtin of the standard functional type, which you can only unpack
    > > from one end and renders these issues moot, plus a builtin doubly
    > > linked list with copy-on-write or copy-on-unpacking semantics.

    >
    > Copy-on-write could be implemented with any type.  You don't need a
    > doubly linked list for that.


    True

    > > We are not talking black magic here; we are talking about an EXPLICIT
    > > type constraint provided on the very same line.

    >
    > An explicit type constraint with very different semantics depending on
    > what particular type you specify and what particular type you're
    > unpacking from, as I had understood it before.  Now you seem to be
    > saying that it would always be a copy, but sharing state with
    > copy-on-write possible, which is a different situation.


    Yes, I think consistent semantics over all sequence types is very
    important. Although, returning a view for immutable collections like
    tuples and 'functional lists' (for lack of a better term), and always
    returning a copy for mutable container types (lists and doubly-linked-
    lists / deques) is not so bad either, imo. Just one extra simple rule
    that is clear and obvious enough.

    > > Well perhaps, but not always knowing the type of your objects at write-
    > > time is inherent to weakly typed languages; this happens all the time.
    > > Not knowing the type of the sequence to be unpacked is in a sense an
    > > asset; I can use this construct in a function, and unpack any sequence
    > > type in a manner appropriate for it. About the result of the unpacking
    > > I will know just as much as about the input to it; that they are the
    > > same type.

    >
    > Just because the issue is inherent doesn't mean we should contribute
    > to it.


    How are we contributing to it? If we dont know the type of the RHS, we
    dont know the type of the LHS either, but its not like we lost any
    information. If we do know the type of the RHS, then we do know the
    type of the LHS as well; its conserved.

    > Anyway, the more I think about it, that concern is really more of an
    > issue for straight copying.  One of my pet peeves is that I prefer
    > list(x) for copying sequences rather than the more common x[::].  The
    > latter is fine if all I need is an immutable sequence of uncertain
    > type to iterate and index over -- but then why did I need to make a
    > copy?  Unpacking implies different use cases, though, and maybe a good
    > argument can be made for it to match type.


    Thanks for the constructive feedback; something to think about.
     
    Eelco, Dec 13, 2011
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Eelco Hoogendoorn

    Verbose and flexible args and kwargs syntax

    Eelco Hoogendoorn, Dec 11, 2011, in forum: Python
    Replies:
    1
    Views:
    243
    Steven D'Aprano
    Dec 11, 2011
  2. Eelco Hoogendoorn

    Verbose and flexible args and kwargs syntax

    Eelco Hoogendoorn, Dec 11, 2011, in forum: Python
    Replies:
    0
    Views:
    157
    Eelco Hoogendoorn
    Dec 11, 2011
  3. Eelco Hoogendoorn

    Verbose and flexible args and kwargs syntax

    Eelco Hoogendoorn, Dec 11, 2011, in forum: Python
    Replies:
    0
    Views:
    182
    Eelco Hoogendoorn
    Dec 11, 2011
  4. Eelco Hoogendoorn

    Verbose and flexible args and kwargs syntax

    Eelco Hoogendoorn, Dec 11, 2011, in forum: Python
    Replies:
    88
    Views:
    1,041
    Grant Edwards
    Dec 17, 2011
  5. Eelco Hoogendoorn

    Verbose and flexible args and kwargs syntax

    Eelco Hoogendoorn, Dec 11, 2011, in forum: Python
    Replies:
    0
    Views:
    157
    Eelco Hoogendoorn
    Dec 11, 2011
Loading...

Share This Page