Verbose and flexible args and kwargs syntax

[Jussi Piitulainen]

They might not be willing to define it, but as soon as we programmers
do, well, we did.

Having studied the contemporary philosophy of mathematics, their
concern is probably that in their minds, mathematics is whatever some
dead guy said it was, and they dont know of any dead guy ever talking
about a modulus operation, so therefore it 'does not exist'.

Whatever you want to call the concept we are talking about, or whether
you care to talk about it at all, it is most certainly a binary
operation, since there are two arguments involved. There is no way
around that.

Yes, I think you nailed it.

But I guess I'll still be confused the next time I meet one of them.
Happens to me.

 

[Eelco]

The dict constructor can receive either a sequence or a mapping, so if
I write this:

        def func(a, b, dict(c)):

what will I get?  Probably I would want the equivalent of:

        def func(a, b, **c):

but you seem to be saying that I would actually get the equivalent of this:

        def func(a, b, *c):
            c = dict(c)

Cheers,
Ian

Im not sure if I was clear on that, but I dont care what the
constructors accept; I meant to overload on the concept the underlying
type models. Dicts model a mapping, lists/tuples model a sequence. MI
deriving from both these models is illegal anyway, so one can
unambigiously overload on that trait.

The syntax only superficially resembles 'call the dict constructor
with the object passed into kwargs'. What its supposed to mean is
exactly the same as **kwargs, but with added flexibility.

 

[Ian Kelly]

Im not sure if I was clear on that, but I dont care what the
constructors accept; I meant to overload on the concept the underlying
type models. Dicts model a mapping, lists/tuples model a sequence. MI
deriving from both these models is illegal anyway, so one can
unambigiously overload on that trait.
False.

.... def __init__(self, items):
.... self._items = items
.... def __getitem__(self, item):
.... return self._items[item]
.... def __len__(self):
.... return len(self._items)
....

foo1 = Foo(range(5, 10))
foo2 = Foo({'one': 1, 'two': 2})
foo1[3] 8
foo2['one']

1

Or are you saying that only classes specifically derived from list,
tuple, or dict should be considered, and custom containers that are
not derived from any of those but implement the correct protocols
should be excluded? If so, that sounds less than ideal.

Cheers,
Ian

 

[Eelco]

False.

I stand corrected.

Or are you saying that only classes specifically derived from list,
tuple, or dict should be considered, and custom containers that are
not derived from any of those but implement the correct protocols
should be excluded?  If so, that sounds less than ideal.

That might be a desirable constraint from an implementational/
performance aspect anyway, but I agree, less than ideal.

Either way, its not hard to add some detail to the semantics to allow
all this. Even this function definition:

def func(Foo(args), Foo(kwargs))

....could even be defined unambigiously by overloading first on base
type, and if that does not uniquely determine the args and kwargs,
fall back on positionality, so that:

def func(Foo(args), dict(kwargs))
def func(list(args), Foo(kwargs))

would be uniquely defined as well.

 

[Eelco]

To get back on topic a little bit, lets get back to the syntax of all
this: I think we all agree that recycling the function call syntax is
less than ideal, since while it works in special contexts like a
function signature, its symmetric counterpart inside a function call
already has the meaning of a function call.

In general, we face the problem of specifying metadata about a
variable, or a limited form of type constraint.

What we want is similar to function annotations in python 3; in line
with that, we could have more general variable annotations. With an
important conceptual distinction; function annotations are meaningless
to python, but the annotations I have in mind should modify semantics
directly. However, its still conceptually close enough that we might
want to use the colon syntax here too. To distinguish it from function
annotations, we could use a double colon (double colon is an
annotation with non-void semantics; quite a simple rule); or to
maintain an historic link with the existing packing/unpacking syntax,
we could look at an augmented form of the asteriks notation.

For instance:

def func(list*args, dict*kwargs) <- list-of-args, dict-of-kwargs
def func(args::list, kwargs::dict) <- I like the readability of this
one even better; args-list and kwargs-dict

And:

head, deque*tail = somedeque
head, tail::deque = somedeque

Or some variants thereof

 

[Eelco]

To get back on topic a little bit, lets get back to the syntax of all
this: I think we all agree that recycling the function call syntax is
less than ideal, since while it works in special contexts like a
function signature, its symmetric counterpart inside a function call
already has the meaning of a function call.

In general, we face the problem of specifying metadata about a
variable, or a limited form of type constraint.

What we want is similar to function annotations in python 3; in line
with that, we could have more general variable annotations. With an
important conceptual distinction; function annotations are meaningless
to python, but the annotations I have in mind should modify semantics
directly. However, its still conceptually close enough that we might
want to use the colon syntax here too. To distinguish it from function
annotations, we could use a double colon (double colon is an
annotation with non-void semantics; quite a simple rule); or to
maintain an historic link with the existing packing/unpacking syntax,
we could look at an augmented form of the asteriks notation.

For instance:

def func(list*args, dict*kwargs) <- list-of-args, dict-of-kwargs
def func(args::list, kwargs::dict) <- I like the readability of this
one even better; args-list and kwargs-dict

And:

head, deque*tail = somedeque
head, tail::deque = somedeque

Or some variants thereof

As for calling functions; calling a function with the content of a
collection type rather than the collection as an object itself is a
rather weird special case operation I suppose, but we can cover it
with the same syntax:

def func(args::tuple, kwargs::dict):
funccall(args::, kwargs: <- void type constraint means
unpacking, for symmetry with args/kwargs aggregation
funccall:args, ::kwargs) <- I like this better, to emphasize it
being 'the other side' of the same coin, and quite close to ** syntax

Sequence and Mapping unpacking dont need their own symbols, if things
are done like this, since in the function declaration the meaning is
clear from the type of the annotations used, plus their position, and
in the call the meaning is clear from the type of the object
undergoing to unpacking operation.

 

[OKB (not okblacke)]

And you can blame C for the use of % instead of mod or modulo.

Anytime you can blame C for something, you can also blame a bunch
of other languages for foolishly perpetuating the inanities of C.

--
--OKB (not okblacke)
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is
no path, and leave a trail."
--author unknown

 

[alex23]

So is 'how much wood would a woodchucker chuck if a woodchucker could
chuck wood?'. But how often does that concept turn up in your code?

That comment right there? That's the moment every serious coder
stopped paying attention to a single word you say.

 

[Ian Kelly]

Either way, its not hard to add some detail to the semantics to allow
all this. Even this function definition:

def func(Foo(args), Foo(kwargs))

...could even be defined unambigiously by overloading first on base
type, and if that does not uniquely determine the args and kwargs,
fall back on positionality, so that:

def func(Foo(args), dict(kwargs))
def func(list(args), Foo(kwargs))

would be uniquely defined as well.

That solves some of the problems, but if I just have:

def func(SequenceOrMappingType(args)):

That's going to unpack positionally. If I want it to unpack keywords
instead, how would I change the definition to indicate that?

 

[Steven D'Aprano]

== may be taken to mean identity comparison; 'equals' can only mean one
thing.

Nonsense. "Equals" can be taken to mean anything the language designer
chooses, same as "==". There is no language police that enforces The One
True Meaning Of Equals. In fact, there is no one true meaning of equals.
Even my tiny Pocket Oxford dictionary lists five definitions.

It is ironic that the example you give, that of identity, is the standard
definition of equals in mathematics. 2*2 = 4 does not merely say that
"there is a thing, 2*2, which has the same value as a different thing,
4", but that both sides are the same thing. Two times two *is* four. All
numbers are, in some sense, singletons and equality implies identity.

A language designer might choose to define equals as an identity test, or
as a looser "values are the same" test where the value of an object or
variable is context dependent, *regardless* of how they are spelled: = ==
=== "is" "equals" or even "flibbertigibbet" if they wanted to be
whimsical. The design might allow types to define their own sense of
equality.

Triangle.equals(other_triangle) might be defined to treat any two
congruent triangles as equal; set equality could be defined as an
isomorphism relation; string equality could be defined as case-
insensitive, or to ignore leading and trailing whitespace. Regardless of
whether you or I *would* make those choices, we *could* make those
choices regardless of how our language spells the equality test.

Of course 'formally' these symbols are well defined, but so is
brainf*ck

I don't understand your point here.

So is 'how much wood would a woodchucker chuck if a woodchucker could
chuck wood?'. But how often does that concept turn up in your code?

You didn't make a statement about how often modulo turns up in code
(which is actually quite frequently, and possibly more frequently than
regular division), but about the obscurity of the operation. Taking the
remainder is not an obscure operation. The names "modulo" and "modulus"
may be obscure to those who haven't done a lot of mathematics, but the
concept of remainder is not. "How many pieces are left over after
dividing into equal portions" is something which even small children get.

I didnt know one of Python's design goals was backwards compatibility
with C.

Don't be silly. You know full well Python is not backwards compatible
with C, even if they do share some syntactical features.

C is merely one of many languages which have influenced Python, as are
Haskell, ABC, Pascal, Algol 68, Perl (mostly in the sense of "what not to
do" <wink>), Lisp, and probably many others. It merely happens that C's
use of the notation % for the remainder operation likely influenced
Python's choice of the same notation.

I note that the *semantics* of the operation differs in the two
languages, as I understand that the behaviour of % with negative
arguments is left undefined by the C standard, while Python does specify
the behaviour.

Yes, that was a hyperbole; but quite an often used construct, is it not?

It's hard, but not quite impossible, to write useful Python code without
it, so yes.

You cannot; only constructors modelling a sequence or a dict, and only
in that order. Is that rule clear enough?

But why limit yourself to those restrictive rules?

If I want to collect a sequence of arguments into a string, why shouldn't
I be allowed to write this?

def func(parg, str(args)): ...

If I want to sum a collection of arguments, why not write this?

def func(pargs, sum(args)): ...

Isn't that better than this?

def func(pargs, *args):
args = sum(args)
...


But no. I don't mean those examples to be taken seriously: when you
answer to your own satisfaction why they are bad ideas, you may be closer
to understanding why I believe your idea is also a bad idea.

I hope the above clears that up. It is as much about calling functions
as ** is about raising kwargs to the power of.

I don't understand this comment. Nobody has suggested that ** in function
parameter lists is the exponentiation operator.

As for "calling functions", how else do you expect to generate a type if
you don't call the type constructor? One of your early examples was
something like:

def func(parg, attrdict(kwargs)): ...

If you expect kwargs to be an attrdict, which is not a built-in,
presumably you will have needed to have defined attrdict as a type or
function, and this type or function will get called at run time to
collect the kwargs. That is all.

We dont strictly 'need' any language construct. Real men use assembler,
right?

"We're not using assembly" is not a reason to add a feature to a
language. Every feature adds cost to the language:

* harder to implement;
* harder to maintainer;
* larger code base;
* more documentation to be written;
* more tests to be written;
* more for users to learn

etc.

Yes, I know. How is that not a lot more verbose and worse than what I
have proposed in all possible ways?

That *specific* syntax, outside of function declarations, is something
I've often thought might be useful. But if I were to argue its case, I
would allow arbitrary functions, and treat it as syntactic sugar for:

head, *tail = iterable
tail = func(tail) # or possibly func(*tail)

But that's pie-in-the-sky musing. I certainly wouldn't put it in function
parameter lists. Param lists should be declarations, not code. Apart from
the most limited collation of args, code belongs inside the body of the
function, not the param list:

def foo(a, 2*b+1, c): # double the second arg and add 1

head, tail = somestring[0], somestring[1:]

Well yes, splendid; we can do that with lists too since the dawn of
Python. What you are saying here in effect is that you think the
head/tail syntax is superfluous; that youd rather see it eliminated than
generalized.

No.

It is not "head/tail" syntax, but sequence unpacking, and it has been
nicely generalised to allow things like this in Python 3:

a, b, c, d, *lump, x, y z = any_iterable

any_iterable isn't limited to a list, str, or other object which supports
slicing. It can be any object supporting the iteration protocol.

What I'm saying is that there is no need to OVER-generalise this to
specify the type of *lump within the packing operation. If you want lump
to be something other that Python's choice, perform the conversion
yourself.

 

[Chris Angelico]

It merely happens that C's
use of the notation % for the remainder operation likely influenced
Python's choice of the same notation.

Considering that Python also had the notion that "integer divided by
integer yields integer" until Py3, I would say it's extremely likely
that most of Python's division facilities were modelled off C. That's
not a bad thing; gives you a set of operations that a large number of
people will grok, and only a small number of oddities.

I note that the *semantics* of the operation differs in the two
languages, as I understand that the behaviour of % with negative
arguments is left undefined by the C standard, while Python does specify
the behaviour.

.... and there's the proof that "modelled off" does not mean "slavishly
follows". This lack of definition is a weakness in C.

def foo(a, 2*b+1, c):  # double the second arg and add 1

No, that should subtract 1 from the second arg and halve it. The
expression you give there has to match the value from the parameter
list.

This syntax would be a huge boon to Python. Imagine how much easier
this could make things:

def foo(sum(x)):
return x

print(foo(120)) # will print a list of numbers that sum to 120


ChrisA

 

[Ian Kelly]

If I want to collect a sequence of arguments into a string, why shouldn't
I be allowed to write this?

   def func(parg, str(args)): ...

Obviously, because the correct syntax would be:

def func(parg, ''.join(args)): ...

 

[Eelco]

That solves some of the problems, but if I just have:

        def func(SequenceOrMappingType(args)):

That's going to unpack positionally.  If I want it to unpack keywords
instead, how would I change the definition to indicate that?

That should raise an ambiguity error. But honestly, how often have you
worked with SequenceOrMappingType's? I think this is a rather
palatable constraint.

 

[Eelco]

To answer that question: for the same reasons. The conversion is

wasteful; allowing python to do the right thing based on a
typeconstraint is not. Plus, it is less code, and more readable code;
the only rule you have to learn is quite general, which is that :: is
a type constraint annotation; no need to remember specifics, like
'unpacking always returns lists for some arbitrary reason'.

Oh my bad; actually, that should be:

'collecting the remainder of an unpacked iterable using * will always
yield a list. That is, unless the construct appears inside a function
definition; then somehow a tuple is always the right choice'

 

[Arnaud Delobelle]

Oh my bad; actually, that should be:

'collecting the remainder of an unpacked iterable using * will always
yield a list. That is, unless the construct appears inside a function
definition; then somehow a tuple is always the right choice'

When you quote somebody (even yourself), it would be helpful if you
attributed your quote.

 

[Eelco]

When you quote somebody (even yourself), it would be helpful if you
attributed your quote.

Ah yes; im more used to proper forums, still getting used to these
mailing-list things. But point taken.

 

[Eelco]

With all this being said, I must say that the notion of indtroducing
type constraints into Python is quite a radical one*, and one that
should not be taken lightly, so I understand the general conservative
vibe the notion is getting. It probably has implications beyond just
collection types, and if youd introduce such a feature, you would like
to introduce it only once, and correctly the first time around.

Ill probably start a new thread soon, recapping the accumulated
insight, and capping all the OT threads that have spawned.

*even though the asteriks syntax is infact a limited form of exactly
that

 

[Steven D'Aprano]

[...]
We are not talking mathemathics, we are talking programming languages.

What *I* am talking about is your assertion that there is only one
possible meaning for "equals" in the context of a programing language.
This is *simply not correct*.

You don't have to believe me, just look at the facts. It's hard to find
languages that use the word "equals" (or very close to it) rather than
equals signs, but here are four languages which do:

(1) OpenXION:

Equals in OpenXION is weakly typed, like Perl:

1 + 1.0 equals "2"

true


(2) C# uses method notation: a.Equals(b) can be overridden, but for many
types it defaults to value equality, that is, the equivalent to Python's
a == b.

(3) Ruby uses a.equal?(b) for "reference equality", that is, the
equivalent of Python's "is" operator:

irb(main):001:0> a = "abc"
=> "abc"
irb(main):002:0> b = "abc"
=> "abc"
irb(main):003:0> a.equal?(b)
=> false
irb(main):004:0> a == b
=> true


(4) Mathematica's Equal[x, y] can test values and expressions for
equality. It may return true, false, or unevaluated (i.e. itself).


Four languages that use Equals (or close to it) with four different
behaviours.

Identity versus value equality is a well established concept within its
jargon. Within this context, 'equals' and 'is' have clearly defined
meanings.

Incorrect. Most programming languages do not even have a concept of
identity: identity is only(?) relevant to reference languages, like
Python, where variables are references to objects.

Even for languages that have a concept of identity, most don't don't call
it "is". Objective-C calls it "==", PHP calls it "===", C# calls it
object.ReferenceEquals. (Python, Algol 68, and VB .NET are three which do
call it "is".)

For stack-based languages like Forth, it doesn't even make sense to talk
about identity, since values aren't variables: they're just values on a
stack, not values in a fixed location, or bound to a known name.

Again, all this goes to demonstrate that the language designer is free to
choose any behaviour they like, and give it any name they like.


[...]

So 'frequency of use' is no valid interpretation of 'obscurity'? Im not
a native speaker, but im pretty sure it is.

No. Things which are obscure are used in language infrequently, because
if they were common they would not be obscure. But things which are used
infrequently are not necessarily obscure.

An example in common language: "Napoleon Bonaparte" does not come up in
conversation very frequently, but he is not an obscure historical figure.

An example from programming: very few people need to use the
trigonometric functions sin, cos, tan in their code. But they are not
obscure functions: most people remember them from school. People who have
forgotten almost everything about mathematics except basic arithmetic
probably remember sin, cos and tan. But they never use them.

Of course I was being silly; I know this use is following a historical
precedent; but id rather see it rectified in the previous version of
python rather than the next. My sillyness was prompted by the percieved
pointlessness of your remark. Of course python is not trying to be
backwards compatible with C; so why bring it up then?

Because you asked why Python uses the % operator for remainder.


[...]

They are bad ideas because they truely do not lead to the execution of
different code, but are merely a reordering, mixing statements in with a
function declaration. I am proposing no such thing; again, the type(arg)
notation I have dropped, and never was meant to have anything to do with
function calling; it is a way of supplying an optional type constraint,
so in analogy with function annotations, I changed that to arg::type.
Again, this has nothing to do with calling functions on arguments.

You have not thought about this carefully enough. Consider what happens
when this code gets called:

def f(*args): pass

f(a, b, c)


The Python virtual machine (interpreter, if you prefer) must take three
arguments a, b, c and create a tuple from them. This must happen at
runtime, because the value of the objects is not known at compile time.
So at some point between f(a, b, c) being called and the body of f being
entered, a tuple must be created, and the values of a, b, c must be
collated into a single tuple.

Now extend this reasoning to your proposal:

def f(args:FOO): pass

At runtime, the supplied arguments must be collated into a FOO, whatever
FOO happens to be. Hence, the function that creates FOO objects must be
called before the body of f can be entered. This doesn't happen for free.
Whether you do it manually, or have the Python interpreter do it, it
still needs to be done.

First off, type constraints must have some use; all those languages cant
be wrong.

But you're not talking about type constraints. You're not instructing the
function to reject arguments which have the wrong type, you are
instructing it to collate multiple arguments into a list (instead of a
tuple like Python currently does). def f(*args) *constructs* a tuple, it
doesn't perform a type-check.

 

[Steven D'Aprano]

With all this being said, I must say that the notion of indtroducing
type constraints into Python is quite a radical one*,

Not that radical. Here's the creator of Python musing about adding
optional type checks to Python:

http://www.artima.com/weblogs/viewpost.jsp?thread=85551
http://www.artima.com/weblogs/viewpost.jsp?thread=86641
http://www.artima.com/weblogs/viewpost.jsp?thread=87182


[...]

*even though the asteriks syntax is infact a limited form of exactly
that

It absolutely is not. def f(*args, **kwargs) constructs a tuple and a
dict, it does not type-check that the function is passed a tuple and a
dict as arguments. These are completely different things.

 

[Eelco]

Not that radical. Here's the creator of Python musing about adding
optional type checks to Python:

http://www.artima.com/weblogs/viewp....artima.com/weblogs/viewpost.jsp?thread=87182

Good find; but still radical enough that it hasnt been implemented.
Note that these musing are trying to adress a yet far more general
problem of specifying arbitrary types constraints on anything; I am
primarily interested in specifying container types in the special case
of collection packing/unpacking syntax, with further extensions
nothing but a welcome addon. The fact that the former was judged
infeasible does not mean the more modest goal of the latter might not
be attainable.

It absolutely is not. def f(*args, **kwargs) constructs a tuple and a
dict, it does not type-check that the function is passed a tuple and a
dict as arguments. These are completely different things.

Which is of course not something I ever proposed; I never said
anything about checking types of existing data; im talking about
coercing types of newly created data, like the target of a collection
packing. That is exactly what *args and **kwargs also do.