What could 'f(this:that=other):' mean?

[Jonathan Fine]

Giudo has suggested adding optional static typing to Python.
(I hope suggested is the correct word.)
http://www.artima.com/weblogs/viewpost.jsp?thread=85551

An example of the syntax he proposes is:

> def f(this:that=other):
> print this

This means that f() has a 'this' parameter, of type 'that'.
And 'other' is the default value.

I'm going to suggest a different use for a similar syntax.

In XML the syntax

> <element this:that='other'>

is used for name spaces.

Name spaces allow independent attributes to be applied to an
element. For example, 'fo' attributes for fonts and layout.
XSLT is of course a big user of namespaces in XML.

Namespaces seems to be a key idea in allow independent
applications to apply attributes to the same element.

For various reasons, I am interested in wrapping functions,
and supplying additional arguments. Namespaces would be
useful here. Decorators, by the way, are ways of wrapping
functions. Namespaces might make decorators a bit easier
to use.

Here's an example of how it might work. With f as above:

> f(this:that='value')

{'that': 'value'}

Do you see? The syntax of def makes 'this' a dictionary.
And the syntax of the call adds an item to 'this'.
This aligns nicely with XML syntax and semantics.

One could extend **kwargs similarly.

> def g(***nsargs):
> print ***nsargs
>
> g(this:that='other', that:this='more')

{'this': {'that': 'other'}; {'that': {'this': 'more'}}

All the namespace args are gathered into a dict of dicts.

Thus, this suggestion is mostly syntactic sugar for
f(this=dict(that='other), that=dict('this'=other))

(Have I got this right? - I'm only up to Python 2.2 at
home. This is how I remember 2.4.)


Back to optional static typing. A common idiom is

> def return_dict(data=None):
> if data is None:
> data = {}

# etc

This avoid the newbie gotcha in

> def return_dict(data={}:
> # etc

So to write this using the suggested syntax one has:

> def return_dict(data:dict=None):
> # oops!

So now some final comments.
1. I have no opinion yet about optional static typing.
2. Calls of function f() should outnumber definitions of f().
(I'm not totally convinced of this - for example __eq__ may
be defined in many classes, but called by few functions.
Frameworks often have functions as parameters.)
3. Granted (2), perhaps function calls are first in the
queue for syntactic sugar.

 

[Jeff Shannon]

Giudo has suggested adding optional static typing to Python.
(I hope suggested is the correct word.)
http://www.artima.com/weblogs/viewpost.jsp?thread=85551

An example of the syntax he proposes is:

This means that f() has a 'this' parameter, of type 'that'.
And 'other' is the default value.

Hm; so for a slightly more concrete example, one might have

def fib_sequence(length:int=9): ...

I'm going to suggest a different use for a similar syntax.

In XML the syntax

<element this:that='other'>

is used for name spaces.

Name spaces allow independent attributes to be applied to an
element. For example, 'fo' attributes for fonts and layout.
XSLT is of course a big user of namespaces in XML.

Namespaces seems to be a key idea in allow independent
applications to apply attributes to the same element.
[...]
Here's an example of how it might work. With f as above:

f(this:that='value')

{'that': 'value'}

I fail to see how this is a significant advantage over simply using
**kwargs. It allows you to have multiple dictionaries instead of just
one, that's all. And as you point out, it's trivial to construct your
own nested dicts.

Besides, Python already uses the concept of namespaces by mapping them
to object attributes. Module references are a namespace, exposed via
the attribute-lookup mechanism. This (IMO) fails the "there should be
one (and preferably only one) obvious way to do things" test. The
functionality already exists, so having yet-another way to spell it
will only result in more confusion. (The fact that we're borrowing
the spelling from XML does little to mollify that confusion.)

3. Granted (2), perhaps function calls are first in the
queue for syntactic sugar.

Huh? How much simpler of syntax do you want for calling a function?
I'm not sure what you'd want as "sugar" instead of funcname().

Jeff Shannon
Technician/Programmer
Credit International

 

[Jonathan Fine]

Giudo has suggested adding optional static typing to Python.
(I hope suggested is the correct word.)
http://www.artima.com/weblogs/viewpost.jsp?thread=85551

An example of the syntax he proposes is:

I'm going to suggest a different use for a similar syntax.

In XML the syntax

<element this:that='other'>

is used for name spaces.

Name spaces allow independent attributes to be applied to an
element. For example, 'fo' attributes for fonts and layout.
XSLT is of course a big user of namespaces in XML.

Namespaces seems to be a key idea in allow independent
applications to apply attributes to the same element.
[...]
Here's an example of how it might work. With f as above:

f(this:that='value')

{'that': 'value'}

I fail to see how this is a significant advantage over simply using
**kwargs. It allows you to have multiple dictionaries instead of just
one, that's all. And as you point out, it's trivial to construct your
own nested dicts.

This argument could be applied to **kwargs (and to *args). In other
words, **kwargs can be avoided using a trivial construction.
{'a': 3, 'b': 4}

(and in Python 2.3){'a': 3, 'b': 4}

f_1() is internally the same as f_2(), but the argument passing is
different.

Jeff, are you in favour of kwargs as a language feature? If so, you
may wish to refine your argument.

(One can be in favour of kwargs and against my proposal. That kwargs
is widely used, and my proposal would not be, is a good argument, IMO.)

I'll post some usage examples later today, I hope.

Besides, Python already uses the concept of namespaces by mapping them
to object attributes. Module references are a namespace, exposed via
the attribute-lookup mechanism. This (IMO) fails the "there should be
one (and preferably only one) obvious way to do things" test. The
functionality already exists, so having yet-another way to spell it will
only result in more confusion. (The fact that we're borrowing the
spelling from XML does little to mollify that confusion.)

Here, I don't understand. Could you give an example of two obvious ways
of doing the same thing, should my suggestion be adopted?

 

[Jeff Shannon]

Giudo has suggested adding optional static typing to Python.
(I hope suggested is the correct word.)
http://www.artima.com/weblogs/viewpost.jsp?thread=85551

An example of the syntax he proposes is:
def f(this:that=other):
print this

I'm going to suggest a different use for a similar syntax.

In XML the syntax
<element this:that='other'>
is used for name spaces.

Name spaces allow independent attributes to be applied to an
element. For example, 'fo' attributes for fonts and layout.
XSLT is of course a big user of namespaces in XML.

Namespaces seems to be a key idea in allow independent
applications to apply attributes to the same element.
[...]
Here's an example of how it might work. With f as above:
f(this:that='value')
{'that': 'value'}

I fail to see how this is a significant advantage over simply using
**kwargs. It allows you to have multiple dictionaries instead of just
one, that's all. And as you point out, it's trivial to construct your
own nested dicts.

This argument could be applied to **kwargs (and to *args). In other
words, **kwargs can be avoided using a trivial construction.

The use of *args and **kwargs allows functions to take a variable
number of arguments. The addition of ***nsargs does not add
significantly. Note that *args and **kwargs should always be used
together, because to do otherwise would require the function caller to
know details of the function's implementation (i.e. which arguments
are expected to be positional and which must be keywords). Since we
*want* the caller to not need to know this, then ***nsargs would
always need to be used together with *args and **kwargs. (A function
defined 'def f(***nsargs): ...' could not be called with 'f(1)'. This
means that all you're gaining is an extra bag to put variable numbers
of arguments in. The value here is that it maintains a parallel with
*args and **kwargs when one allows 'namespaced' arguments -- if one
allows that, then ***nsargs is required for consistency's sake, but it
does not simplify anything by itself.

So really, we need to look at what gains we'd get from having
'namespaced' arguments. What's the real advantage here?

When using 'namespace' arguments, instead of standard keyword
arguments, the function body would be given a dictionary instead of a
set of local variables, right? 'def f1(arg1, arg2, arg3, arg4)'
creates four names in locals(), where 'def f2(ns1:arg1, ns1:arg2,
ns1:arg3, ns1:arg4) creates a single dict named ns1 in locals(), which
contains four items (keyed on 'arg1', 'arg2', etc.), and 'def
f3(ns1:arg1, ns1:arg2, ns2:arg3, ns2:arg4)' creates two dicts (ns1 and
ns2) with two entries each.

Okay, now, let's take a look at how these functions will be used.

f1(1, 2, 3, 4)
f1(arg1=1, arg2=2, arg3=3, arg4=4)

Note that f1 doesn't care which of these methods of calling is
employed -- both result in the same situation inside of f1().

So what's the intended way of calling f2()? I'd presume that it
shouldn't care whether keywords or namespaces are specified, so that
the following should all be equivalent:

f2(1, 2, 3, 4)
f2(1, 2, arg3=3, arg4=4)
f2(1, 2, arg3=3, ns1:arg4=4)

Note that this doesn't *add* any utility. The function caller hasn't
gained anything. Since arg4 is unambiguous regardless of whether it's
referred to as arg4 or ns1:arg4, the only time that the caller has any
reason to specify the namespace is if argnames within different
namespaces clash -- that is, if we allow something like 'def
f4(ns1:arg1, ns1:arg2, ns2:arg1, ns2:arg2)'.

Now, though, we've lost the ability to specify *only* the argname and
not the namespace as well -- that is, you *cannot* call f4 with
keywords but not namespaces. From the caller's vantage point, this
means that they need to know the full namespace spec of the function,
which makes it no different than simply using longer (but unique)
keyword names.

So, we can see that allowing namespaces and ***nsargs doesn't add any
utility from the caller's perspective. How much does the callee gain
from it?

Well, the following functions would be equivalent:

def g1(arg1, arg2, arg3, arg4):
ns1 = {'arg1':arg1, 'arg2':arg2, 'arg3':arg3, 'arg4':arg4}
return ns1
def g2(ns1:arg1, ns1:arg2, ns1:arg3, ns1:arg4):
return ns1

You might say "Wow, look at all that repetetive typing I'm saving!"
But that *only* applies if you need to stuff all of your arguments
into dictionaries. I suspect that this is a rather small subset of
functions. In most cases, it will be more convenient to use your
arguments as local variables than as dictionary entries.

def gcd1(a, b):
while a:
a, b = b%a, a
return b

def gcd2(ns1:a, ns1:b):
while ns1['a']:
ns1['a'], ns1['b'] = ns1['b']%ns1['a'], ns1['a']
return ns1['b']

Speaking of repetetive typing....

Besides, another function equivalent to the above two would be:

def g3(arg1, arg2, arg3, arg4):
ns1 = locals()
return ns1

....which is quite a bit less repetetive typing than the 'namespace'
version.

So, you're looking at making the function-calling protocol
significantly more complex, both for caller and callee, for the rather
marginal gain of being able to get arguments prepackaged in a
dictionary or two, when there already exist easy ways of packaging
function arguments into a dict. Given the deliberate bias against
adding lots of new features to Python, one needs a *much* better
cost/benefit ratio for a feature to be considered worthwhile.

Besides, Python already uses the concept of namespaces by mapping them
to object attributes. [...]

Here, I don't understand. Could you give an example of two obvious ways
of doing the same thing, should my suggestion be adopted?

My main point here is that 'namespace' is a term/concept that is
already in use in Python, in different circumstances and using a
completely different mechanism. Functions already *have* namespaces,
whose contents can be inspected with locals() and modules have
namespaces that can be accessed through globals().

I'd note also that the usage you're drawing from, in XML/XSLT, isn't
really comparable to function parameters. It's a much closer parallel
to object attributes. Python *does* have this concept, but it's
spelled differently, using a '.' instead of a ':'. In other words,
the XML fragment you give,

<element this:that='other'>

.... would be more appropriate to render in Python as

e = Element()
e.this.that = 'other'

It's quite reasonable to suppose that some object of type Element may
have a set of font attributes and a set of image attributes, and that
some of these may have the same name. Python would use font objects
and image objects instead of 'namespaces' --

e.font.size = '14pt'
e.image.size = (640, 480)

So while these namespaces are probably a great thing for XML, XSLT,
they're not very useful in Python. Which, given the rather different
goals and design philosophies behind the languages, shouldn't really
be much of a surprise.

Jeff Shannon
Technician/Programmer
Credit International

 

[Jonathan Fine]

Jonathan Fine wrote:

I'll post some usage examples later today, I hope.

Well, here are some examples. A day later, I'm afraid.


** Pipelines and other composites

This is arising for me at work.
I produce Postscript by running TeX on a document.
And then running dvips on the output of TeX.

TeX as a program has parameters (command line options).
As does dvips.

For various reasons I wish to wrap TeX and dvips.

def tex_fn(filename, input_path=None, eps_path=None):
'''Run tex on filename.
Search for input files on input_path.
Search for EPS files on eps_path. '''

pass

def dvips_fn(filename, page_size=None, color_profile=None):
'''Run dvips on filename. etc.'''

pass

In reality, these tex and dvips have many options.
More parameters will be added later.

So now I wish to produce a composite function, that runs
both tex and dvips. And does other glue things.

def tex_and_dvips_fn(filename,
tex:input_path=xxx,
dvips:color_profile=yyy):

# glueing stuff
tex_fn(filename, **tex)
# more glueing stuff
dvips_fn(filename, **dvips)

To avoid a name clash, we use 'tex' for the parameter
space, and 'tex_fn' for the function that takes 'tex'
as parameter.

The point is that parameters can be added to tex_fn and
dvips_fn without our having to change tex_and_dvips_fn


** Wrapping functions

This is the problem that originally motivated my
suggestion.

We have coded a new function.

def adder(i, j): return i + j

We wish to test 'adder'.
But unittest is too verbose for us.

We wish to define a decorator (?) test_wrap to
work as follows.

orig_adder = adder
adder = test_wrap(adder)
new_adder = adder

orig_adder(i, j) and new_adder(i, j) to be
effectively identical - same return, same side
effects, raise same exceptions.

So far,
def test_wrap(fn): return fn
does the job.

But now we want
new_adder(2, 2, returns=4)
new_adder(2, '', raises=TypeError)
to be same as
orig_adder(2, 2)
orig_adder(2, '')
(which can be achieved by ignoring 'returns' and 'raises'.

The idea here is that we can call
adder = new(adder)
early on, and not break any working code.

And we also want
new_adder(2, 2, 5)
new_adder('', '', raises=TypeError)
to raise something like an AssertionError.

OK - so I have an informal specification of test_wrap.

Its clear, I think, that test_wrap must be something like

def test_wrap(fn):

def wrapped_fn(*args, **kwargs):

test_args = {}

# transfer entries from one dict to another
for key in ('returns', 'raises'):
if kwargs.has_key(key):
test_args[key] = kwargs[key]
del kwargs[key]

result = fn(args, kwargs)
if test_args.has_key(result):
assert test_args['result'] = result

(I've not coded testing for 'raises'.)

Now, the more parameters added by the test_wrap function,
the more the chance of a name clash.

So why not reduce the chances by using name spaces.

One possible namespace syntax is:
new_adder(2, 3, test=dict(returns=5))

Another such syntax is:
new_adder(2, 3, test:returns=5)

Each has its merits.

The first works with Python 2.4.
The second is, in my opinion, easier on the eye.

Anyway, that's my suggestion.


Jonathan

 

[Jonathan Fine]

Jonathan Fine wrote:

The use of *args and **kwargs allows functions to take a variable number
of arguments. The addition of ***nsargs does not add significantly.

I've posted usage examples elsewhere in this thread.
I think they show that ***nsargs do provide a benefit.
At least in these cases.

How significant is another matter. And how often do they occur.

Now, though, we've lost the ability to specify *only* the argname and
not the namespace as well -- that is, you *cannot* call f4 with keywords
but not namespaces. From the caller's vantage point, this means that
they need to know the full namespace spec of the function, which makes
it no different than simply using longer (but unique) keyword names.

This is a major point.

From the caller's point of view:
fn_1(x_aa=1, x_bb=2, y_aa=3)
fn_2(x:aa=1, x:bb=2, y:aa=3)
are very similar. Replace '_' by ':'.

So you are saying, I think, 'what does this buy the caller'.

Well, by using ':' the namespacing is explicit.
fn_3(my_path, my_file, any_file)
Is this an example of implicit namespacing? (Rhetorical question.)

Explicit is better than implicit, I'm told (smile).

So, we can see that allowing namespaces and ***nsargs doesn't add any
utility from the caller's perspective. How much does the callee gain
from it?

Well, the following functions would be equivalent:

def g1(arg1, arg2, arg3, arg4):
ns1 = {'arg1':arg1, 'arg2':arg2, 'arg3':arg3, 'arg4':arg4}
return ns1
def g2(ns1:arg1, ns1:arg2, ns1:arg3, ns1:arg4):
return ns1

You might say "Wow, look at all that repetetive typing I'm saving!" But
that *only* applies if you need to stuff all of your arguments into
dictionaries.

This is a key point. But there's more to it.

Namespace arguments are good if you have to divide the arguments into
two or more dictionaries. Based on the namespace prefix.

I suspect that this is a rather small subset of
functions. In most cases, it will be more convenient to use your
arguments as local variables than as dictionary entries.

This is a matter of usage. If the usage is very, very small, then
what's the point. If the usage is very, very large, then the case
is very strong.

def gcd1(a, b):
while a:
a, b = b%a, a
return b

def gcd2(ns1:a, ns1:b):
while ns1['a']:
ns1['a'], ns1['b'] = ns1['b']%ns1['a'], ns1['a']
return ns1['b']

Speaking of repetetive typing....

Besides, another function equivalent to the above two would be:

def g3(arg1, arg2, arg3, arg4):
ns1 = locals()
return ns1

....which is quite a bit less repetetive typing than the 'namespace'
version.

These are not good examples for the use of namespacing.
And the results are horrible.

So this is an argument _in favour_ of namespacing.

Namely, that it _passes_ "there should be one (and preferably only one)
obvious way to do things" test (quoted from your earlier message).

So, you're looking at making the function-calling protocol significantly
more complex, both for caller and callee, for the rather marginal gain
of being able to get arguments prepackaged in a dictionary or two, when
there already exist easy ways of packaging function arguments into a
dict. Given the deliberate bias against adding lots of new features to
Python, one needs a *much* better cost/benefit ratio for a feature to be
considered worthwhile.

I believe that we _agree_, that it is a matter of cost/benefit ratio.

My opinion is that studying usage examples is a good way of evaluating
this ratio. Which is why I have posted some favourable usage examples.

I'd note also that the usage you're drawing from, in XML/XSLT, isn't
really comparable to function parameters. It's a much closer parallel
to object attributes. Python *does* have this concept, but it's spelled
differently, using a '.' instead of a ':'. In other words, the XML
fragment you give,

<element this:that='other'>

.... would be more appropriate to render in Python as

e = Element()
e.this.that = 'other'

It's quite reasonable to suppose that some object of type Element may
have a set of font attributes and a set of image attributes, and that
some of these may have the same name. Python would use font objects and
image objects instead of 'namespaces' --

e.font.size = '14pt'
e.image.size = (640, 480)

So while these namespaces are probably a great thing for XML, XSLT,
they're not very useful in Python. Which, given the rather different
goals and design philosophies behind the languages, shouldn't really be
much of a surprise.

It may be better, in some cases, to write

fp = FontParams()
fp.size = 14
fp.slope = 0.1
f(this=this, font_params=fp)

But not, I think, if the definition of f is something like:

def f(this, font_params):

fpd = font_params.__dict__
font_function(**fpd)

(Assuming that font_function has got it right.)


Prompted by your post, the idea of writing
f(this.that='other')
instead of
f(this:that='other')
came to my mind.

At first I did not like it, but now it is growing on me a little.

However, that is a syntactic issue. Which falls if the costs do
not justify the benefits.


And here, I think, the key question is this:

How often do you have to divide the arguments to a function into
two or more dictionaries?

Which can be answered by studying usage.

 

[Nick Coghlan]

If the caller is meant to supply a namespace, get them to supply a namespace.

def f(ns1, ns2):
print ns1['a'], ns1['b'], ns2['a'], ns2['b']

f(ns1 = dict(a=1, b=2), ns2 = dict(a=3, b=4))

Hey, where's Steve? Maybe his generic objects should be called namespaces
instead of bunches. . .

def f(ns1, ns2):
print ns1.a, ns1.b, ns2.a, ns2.b

f(ns1 = namespace(a=1, b=2), ns2 = namespace(a=3, b=4))

Cheers,
Nick.

 

[Jonathan Fine]

If the caller is meant to supply a namespace, get them to supply a
namespace.

def f(ns1, ns2):
print ns1['a'], ns1['b'], ns2['a'], ns2['b']

f(ns1 = dict(a=1, b=2), ns2 = dict(a=3, b=4))

Hey, where's Steve? Maybe his generic objects should be called
namespaces instead of bunches. . .

def f(ns1, ns2):
print ns1.a, ns1.b, ns2.a, ns2.b

f(ns1 = namespace(a=1, b=2), ns2 = namespace(a=3, b=4))

Basically, there are three main possibilities.

f_1(ns1=dict(a=1, b=2), ns2=dict(a=3, b=4))
f_2(ns1_a=1m, ns1_b=2, ns2_a=3, ns2_b=4)
f_3(ns1:a=1m, ns1:b=2, ns2:a=3, ns2:b=4)

f_3 is the suggested extension to Python.
f_3 is similar to f_2 for the caller of f_3.
f_3 is similar to f_1 for the implementor of f_3.

Nick points out that a key issue is this: Is the user meant
to supply arguments belonging to a namespace?

I'm not, at this time, wishing to promote my suggestion.
If I were, I would be well advised to find usage cases.

Rather, I simply wish to point out that the
f(this:that=other)
syntax may have uses, other than optional static typing.

And this I've done. So for me the thread is closed.


Jonathan