determining the number of output arguments

D

Dave Brueck

Greg said:
While I suspect you may be largely right, I
find myself wondering why this should be so.

*Is* it largely right? I don't think so. As you said, there doesn't seem to be
anything "wrong" with passing multiple pieces of data _into_ a function, so why
should we assume that complex data passing should be so one way?

One of my biggest pet peeves about programming in C, for example, is that you
are often forced to wrap stuff up into a structure just to pass data back and
forth - you create the structure, populate it, send it, and then pull out the data.

In many, many cases this is nothing more than poor-man's tuple unpacking, and
you end up with lots of extra work and lots of one-off structures. Also annoying
is the use of "out" parameters, which is again basically manual tuple unpacking.

Python isn't too different from C wrt deciding when to move from "bare"
parameters to a structure or object - is the number of parameters becoming
cumbersome? are the interrelationships becoming complex? will I need to use
those same parameters as a group elsewhere? etc. The difference is that Python
facilitates more natural data passing on the return.

A function like divmod is a prime example; rather than having

div = 0
mod = 0
divmod(x,y, &div, &mod)

we instead have the much more elegant

div, mod = divmod(x,y)

We can definitely come up with some hints or guidelines ("if your function
returns 10 parameters, that's probably bad" or "if the parameters are tightly
coupled and/or are often used & passed along together to different functions,
you probably should wrap them into an object"), but I don't think returning
tuples is _generally_ a sign of anything good or bad.

-Dave
 
C

Carlos Ribeiro

While I suspect you may be largely right, I
find myself wondering why this should be so. We
don't seem to have any trouble with multiple inputs
to a function, so why should multiple outputs be
a bad thing? What is the reason for this asymmetry?

If there is an assymetry, it is in *favor* of returning tuples for
multiple outputs. Using a tuple removes some of the coupling between
the caller and the function. Note that the function signature defines
names for its internal use, and the caller is not required to know the
internal names (unless kw parameter passing is used). Forcing names on
the output introduces *more* coupling, and makes the design less
flexible.

In practical terms, it's important to note that generic utility
functions often do not return a "full class", but just a bunch of
results that may or not be stored into a single class. Requiring that
a class to be defined just to return a bunch of data is overkill.

Finally, I assume that it's considered standard practice in Python to
use tuples to store arbitrary collections of data, pretty much like C
structs. The only difference is that the members are not named (and
thus loosely coupled). This should not be a big deal for *most*
situation. The structure of the return tuple can be documented in the
doc string. For complex structures, returning a dict is good
possibility (although in this case the fact that the names are defined
introduces some degree of coupling). Of course, if the case is complex
enough, declaring a class is the way to go, but then, it's a design
decision.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)
 
D

Donn Cave

*Is* it largely right? I don't think so. As you said, there doesn't seem to be
anything "wrong" with passing multiple pieces of data _into_ a function, so why
should we assume that complex data passing should be so one way?

What's up with keyword arguments, then? I don't think you can argue
that on one hand keyword arguments are a valuable feature, but plain
by-order tuples are a totally satisfactory interface on the other hand.
One of my biggest pet peeves about programming in C, for example, is that you
are often forced to wrap stuff up into a structure just to pass data back and
forth - you create the structure, populate it, send it, and then pull out the
data.

In many, many cases this is nothing more than poor-man's tuple unpacking, and
you end up with lots of extra work and lots of one-off structures. Also
annoying
is the use of "out" parameters, which is again basically manual tuple
unpacking.

Take stat(), which for those who haven't used it reports a number
of characteristics of a file. It's not the ideal example, because
it's big (10 items) and it's already "fixed" (has attributes like
a class instance.) But its worst problem has nothing to do with
the size, or whether or not the items can be referred to by name.
As long as they can be referred to by position, we're stuck with
the exact same items on all platforms, in all situations, for the
rest of eternity. Whether some of them might not really be supported
on some platforms, or it omits some useful values on others. It's
not very flexible. C stat(2) doesn't have this problem.

I don't know if I'd go along with "largely right", but it could
fall somewhere between "some truth in it" and "completely whacked."

Donn Cave, (e-mail address removed)
 
N

Neil Hodgson

Greg Ewing:
Maybe things would be better if we had "dict unpacking":

a, c, b = {'a': 1, 'b': 2, 'c': 3}

would give a == 1, c == 3, b == 2. Then we could
accept outputs by keyword as well as inputs...

Tuple returns do seem trickier than multiple arguments, partly due to
being more novel for me and partly because the function definition does not
document the tuple in code, although there is often a comment. Perhaps
something like:

def Transform(filename) -> (share, permissions, lock):
....
return (s, p, l)
....
(s=share, p=permissions) = Transform(name)

Neil
 
C

Carlos Ribeiro

Maybe things would be better if we had "dict unpacking":

a, c, b = {'a': 1, 'b': 2, 'c': 3}

would give a == 1, c == 3, b == 2. Then we could
accept outputs by keyword as well as inputs...

Dicts are not the best way to make it, because they are not ordered
:) But perhaps we could have something like "named tuples"; immutable
objects, like tuples, with the property that a unique name can be
associated to every item. Couple it with a decorator, and it can be
written like this:

@returns('year','month','day')
def today():
...
return year, month, day

The return of this function would be a 'named tuple'. It could be
treated like a sequence (using __getitem__), or like a dict (using
__getattr__). This way, we could have the best of both worlds; the
simplicity of tuples and the convenience of named attribute access.

(warning: now I am going to give myself enough rope to hang everyday
in my life... and please, flames off :)

What if 'named tuples' were supported natively? Entering into pre-PEP
mode (a really dangerous thing to do), the idea is that a sequence
could have a __names__ property, that would contain a tuple of names.
On __set__, this property would automatically build a hash using the
names & and the tuple contents. On __get__, it would simply return a
tuple. (I think that it is better to expose the __names__ attribute as
a tuple, and not as a dict; it's clean, more tuple-like, and hides a
lot of the complexity).

(BTW, I still think that using tuples is convenient, flexible, and as
loosely coupled as it can get. But having named access is also
convenient sometimes).

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)
 
D

Dave Brueck

Donn said:
What's up with keyword arguments, then? I don't think you can argue
that on one hand keyword arguments are a valuable feature, but plain
by-order tuples are a totally satisfactory interface on the other hand.

I don't think anybody is putting forth the opinion that by-order tuples are
totally satisfactory. I just disagree that returning a tuple is generally a sign
of something wrong with your function.

-Dave
 
C

Carlos Ribeiro

The return of this function would be a 'named tuple'. It could be
treated like a sequence (using __getitem__), or like a dict (using
__getattr__). This way, we could have the best of both worlds; the
simplicity of tuples and the convenience of named attribute access.

Well.. I think I should have googled for it *before* hitting the send button :)

There are a few recipes in the Python Cookbook to deal with named
tuples. I've collected some pointers for those who may be interested
into this topic:

* Tuples with Named Elements via Spawning -- Derrick Wallace
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/303770

* Tuples with named elements -- Andrew Durdin
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/303439

* super tuples -- Gonçalo Rodrigues
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/218485

* Tuples with named elements - using metaclasses -- Andrew Durdin
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/303481

* Just van Rossum's NamedTuple
http://just.letterror.com/ltrwiki/JustVanRossum_2fNamedTuple

* Christos Georgiou -- an old reference to a similar problem (named
structs) on Google Groups
http://groups.google.com/[email protected]

* Another reference to the same code posted above, from the author's website
http://www.sil-tec.gr/~tzot/python/TupleStruct.py

One of the reasons I became interested on this is related to my
previous problems with 'unordered' dicts in some contexts, specially
when implementing some metaclass tricks. We've had some discussion
here about two months ago, and one of the comments was that it would
be useful if the locals() dict used in a class definition stored order
information. I see some parallels between the two problems (ordered
dicts and named tuples). The fact that named tuple are immutable by
definition should help in this case.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)
 
A

Alex Martelli

Carlos Ribeiro said:
But perhaps we could have something like "named tuples"; immutable
objects, like tuples, with the property that a unique name can be
associated to every item.

Of course we should -- I've lost count of how many recipes in the
cookbook I've merged that were implementing that idea in umpteen ways,
not counting several other ideas that only flew by in this NG. Since
standard modules time, os (for stat), resource (dunno if any others),
started returning this kind of supertuples, their convenience has been
obvious to all. I do believe they SHOULD _be_ tuples (that matters when
they're the only RHS argument of a % formatting operator) and it should
also be easy to get from them a name->value mapping (for % formatting
with named-items format style, and the like). Definitely PEP time...
who's gonna carry the torch?


Alex
 
B

Bengt Richter

ISTM that a tuple _is_ a class that wraps content for return as a _single value_,
just like a custom container class instance. What's the problem? The fact that you
can write

r,g,b = foo()

instead of

t = foo()
r = t[0]
g = t[1]
b = t[0]

(or some other verbose object-content-accessing code)
is very nice, if you ask me. There are lots of uses for
small ordered sets of values where no explicit naming
is required, any more than it is in a call to def foo(r,g,b): ...

I agree that unpacking long tuples to a series of local names
is bug-prone, just like a similar arg list for function call, but
it is efficient and sometimes that's worth the debugging of a
couple of lines of code. I don't think programmers should arbitrarily
be prevented from using tuples as they see fit.

Since the alternative of returning a dict of keyworded values is now
syntactically so simple (i.e., return dict(a=a_value, b=b_value, c=etc),
ISTM this is mostly a design/style issue, and I disagree with the idea
that returning small tuples is generally bad design.
While I suspect you may be largely right, I
I suspect that most tuples returned are small, and the too-broad
criticism of tuple returning is therefore largely inappropriate ;-)
find myself wondering why this should be so. We
don't seem to have any trouble with multiple inputs
to a function, so why should multiple outputs be
a bad thing? What is the reason for this asymmetry?
Actually, it is not multiple outputs. It is a single tuple.
Perhaps it has something to do with positional vs.
keyword arguments. If a function has too many input
arguments to remember what order they go in, we
always have the option of specifying them by
keyword. But we can't do that with the return-a-tuple-
and-unpack technique for output arguments -- it's
strictly positional.
We can (in current python) as easily return a dict as a tuple,
e.g., return dict(a=1, b=2, c=3) for your dict below.
Maybe things would be better if we had "dict unpacking":

a, c, b = {'a': 1, 'b': 2, 'c': 3}

Yes, that is an interesting idea. It's part of the larget issue
of programmatic creation of bindings in the local namespace. The
trouble is that locals() effectively generates a snapshot dict
of current local bindings, and it's possible to mutate this dict,
but it's not possible (w/o black magic) to get the mods back into
the actual local bindings. If locals() returned a proxy for the
local namespace that could rebind local names, we could write

localsproxy().update({'a': 1, 'b': 2, 'c': 3})

instead of your line above. Likewise if foo returned a dict,

localsproxy().update(foo())

Maybe keyword unpacking could spell that with a '**' assignment target,
e.g.,

** = foo() # update local bindings with all legal-name bindings in returned dict

Hm, maybe you could use the same sugar for attributes, e.g.,

obj.** = foo() # sugar for obj.__dict__.update(foo())

IWT it would be acceptable to limit binding to existing bindings, so that frame structure
would not have to be altered. Still, as Carlos pointed out, formal parameter names
are private to a function, and their raison d'etre is to decouple function code internal
naming from external naming. Returning named values (whether using dict per se or the
attribute dict of a custom object, etc) creates coupling of a kind again. Or course, so
does any custom object with named attributes or methods.
would give a == 1, c == 3, b == 2. Then we could
accept outputs by keyword as well as inputs...
I think

a, c, b = {'a': 1, 'b': 2, 'c': 3}

probably needs an unpacking indicator in the syntax, e.g.,

a, c, b = **{'a': 1, 'b': 2, 'c': 3}

And I think this should be sugar for the effect

a = {'a': 1, 'b': 2, 'c': 3}['a']
c = {'a': 1, 'b': 2, 'c': 3}['c']
b = {'a': 1, 'b': 2, 'c': 3}['b']

I.e., if the dict has additional content, it is not an error
Hm, do you want to go for a dict.get-like default value in that?

a, c, b = **(default_value){'a': 1, 'b': 2}

would be sugar for

a = {'a': 1, 'b': 2, 'c': 3}.get('a', default_value)
c = {'a': 1, 'b': 2, 'c': 3}.get('c', default_value)
b = {'a': 1, 'b': 2, 'c': 3}.get('b', default_value)

Hm, better stop ;-)

Regards,
Bengt Richter
 
C

Carlos Ribeiro

Of course we should -- I've lost count of how many recipes in the
cookbook I've merged that were implementing that idea in umpteen ways,
not counting several other ideas that only flew by in this NG. Since
standard modules time, os (for stat), resource (dunno if any others),
started returning this kind of supertuples, their convenience has been
obvious to all. I do believe they SHOULD _be_ tuples (that matters when
they're the only RHS argument of a % formatting operator) and it should
also be easy to get from them a name->value mapping (for % formatting
with named-items format style, and the like). Definitely PEP time...
who's gonna carry the torch?

*After* I posted that message, I *did* some research (wrong order, I
know). And you're right, the situation now is pretty much like
Tanenbaum's famous quote: "the nice thing about standards is that
there are so many of them to choose from".

I am tempted to start writing this PEP. I think that I have a pretty
good idea about what do I want (which by itself isn't worth very
much). Anyway, it's a starting point:

1. Do not change the syntax. This leaves out some of the fancy
proposals, but greatly improves the acceptance chances.

2. Named tuples should, for all practical purposes, be an extension of
standard tuples.

3. Conventional access returns a tuple. __getitem__ works as in a
tuple, and the object itself is represented (by repr() and str()) as a
tuple.

4. Named attribute access is supported by __getattr__. Names are
looked up on the magic __names__ attribute of the tuple.

5. On slicing, a named tuple should return another named tuple. This
means that the __names__ tuple has to be sliced also.

6. Although useful, a new named tuple *cannot* be built directly from
a dict, as in this example:

tuple({'a':1, 'b':2})

....because the dict isn't ordered, and there is no way to guarantee
that the tuple would be constructed in the correct order. This will
only be possible if the dict stores the ordering information (btw, the
'correct' order is the definition order; it's the only one that can't
be inferred later at runtime, and alphabetical ordering can be always
obtained by a simple sort).

(However, *if* ordered dicts ever make it into the language, then a
conversion between 'named tuples' and 'ordered dicts' would become
natural -- think about it as an 'adaptation' :)

7. Now for the controversial stuff. NamedTuples can be implemented as
a regular class -- there are many recipes to look at and choose from.
However, I think that is possible to implement named attribute access
as an "improvement" of regular sequence types, with no syntax changes.

The idea is that any tuple could be turned into a named tuple by
assigning a sequence of names to its __names__ magic attribute. If the
attribute isn't assigned (as it's the case with plain tuples), then
the named attribute is disabled. BTW, adding names to an unnamed tuple
*does not* violates the immutability of the tuple.

The usage would be as follows:

time = now()
time.__names__ = ('hour', 'minute', 'second')
print time.hour

The reasoning behind this proposal is as follows:

7.1. It's fully backwards-compatible.

7.2. It allow for easy annotation of existing tuples.

7.3. It allows for internal optimization. By using an internal mapping
directly wired into the sequence, it is possible to hide the mapping
mechanism, and also to make it work faster. This also adds a safety
layer, and helps to avoid direct 'peeking' at the internal mapping
structure that will be exposed by a native Python implementation.

The *biggest* problem of this proposal is that it's MUCH more complex
-- orders of magnitude, perhaps -- and also, that it may present
problems for others implementations besides CPython. Anyway, it's
something that I think deserves discussion.


--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)
 
J

Jeff Shannon

Carlos said:
4. Named attribute access is supported by __getattr__. Names are
looked up on the magic __names__ attribute of the tuple.

5. On slicing, a named tuple should return another named tuple. This
means that the __names__ tuple has to be sliced also.

Hm. If __names__ is a tuple, then does that tuple have a __names__
attribute as well?

(In practice, I can't imagine any reason why tuple.__names__.__names__
should ever be anything other than None, but the potential recursiveness
makes me nervous...)

Jeff Shannon
Technician/Programmer
Credit International
 
C

Carlos Ribeiro

Hm. If __names__ is a tuple, then does that tuple have a __names__
attribute as well?

(In practice, I can't imagine any reason why tuple.__names__.__names__
should ever be anything other than None, but the potential recursiveness
makes me nervous...)

Humm. The worst case is if it's done in a circular fashion, as in:

mytuple.__names__ = nametuple
nametuple.__names__ = mytuple

That's weird. The best that I can imagine now is that it would be
illegal to assign a named tuple to the __names__ member of another
tuple.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)
 
B

Bryan

Carlos said:
Humm. The worst case is if it's done in a circular fashion, as in:

mytuple.__names__ = nametuple
nametuple.__names__ = mytuple

That's weird. The best that I can imagine now is that it would be
illegal to assign a named tuple to the __names__ member of another
tuple.


it should be possible to avoid a recusive problem:
>>> a = ('1', '2')
>>> a.__names__ = ('ONE', 'TWO')
>>> a[0] '1'
>>> a.ONE '1'
>>> a[0] is a.ONE True
>>> b = (3, 4)
>>> b.__names__ = a
>>> b[0] 3
>>> b.1 3
>>> b.ONE
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
IndexError: tuple index out of range


>>> a = (1, 2)
>>> a.__names__ = ('ONE', 'TWO')
>>> a[0] 1
>>> a.ONE 1
>>> a[0] is a.ONE True
>>> b = (3, 4)
>>> b.__names__ = a
>>> b[0] 3
>>> b.1
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
TypeError: tuple must contain strings



or maybe __names__ could be a property method and do some validation on assignment.

>>> a = (1, 2)
>>> a.__names__ = ('ONE', 'TWO')
>>> a[0] 1
>>> a.ONE 1
>>> a[0] is a.ONE True
>>> b = (3, 4)
>>> b.__names__ = a
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
TypeError: tuple must contain strings



bryan
 
G

Greg Ewing

Carlos said:
@returns('year','month','day')
def today():
...
return year, month, day

I was also thinking of suggesting an extension to
the return statement:

return year = y, month = m, day = d

but I thought one radical idea per post would
be plenty. :)
 
B

Bengt Richter

Humm. The worst case is if it's done in a circular fashion, as in:

mytuple.__names__ = nametuple
nametuple.__names__ = mytuple

That's weird. The best that I can imagine now is that it would be
illegal to assign a named tuple to the __names__ member of another
tuple.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

Just to introduce a different perspective, a viewer for unnamed sequences,
rather than a new type with names:
... """create sequence viewer object"""
... def __init__(self, names=''):
... for i, name in enumerate(names.split()):
... setattr(self, name, i)
... def __call__(self, tup):
... """accept tup and return self for named access"""
... self.tup = tup; return self
... def __setitem__(self, i, tup):
... """provide assignment target for tuples to view"""
... self.tup = tup
... def __getattribute__(self, name):
... """get tuple item by name"""
... return object.__getattribute__(self,'tup')[
... object.__getattribute__(self, name)]
...
>>> by3names = SeqVu('zero one two')
>>> t = range(5)
>>> by3names(t).one 1
>>> by3names(t).two 2
>>> for by3names[:] in [(i, chr(i)) for i in xrange(65, 70)]:
... print by3names.one, by3names.zero
...
A 65
B 66
C 67
D 68
E 69

Think of this as compost for the thought garden, not a bouquet ;-)

Regards,
Bengt Richter
 
B

Bengt Richter

[/QUOTE]
I guess you are working with a patched version of python?
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: 'tuple' object has only read-only attributes (assign to .__names__)

Regards,
Bengt Richter
 
B

Bryan

Bengt said:
I guess you are working with a patched version of python?

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: 'tuple' object has only read-only attributes (assign to .__names__)

Regards,
Bengt Richter

we were discussing a feature that doesn't yet exist. i believe it's possible to
get around the recursion issue by only allowing the __names__ attribute to be
accessed in the immediate explicitly called tuple. i really do like the idea of
named tuple elements.

bryan
 
A

Alex Martelli

Bengt Richter said:
ISTM that a tuple _is_ a class that wraps content for return as a _single
value_, just like a custom container class instance. What's the problem?
The fact that you

None that I can see, no matter how many times Jeremy repeats
"generally".
is very nice, if you ask me. There are lots of uses for
small ordered sets of values where no explicit naming
is required, any more than it is in a call to def foo(r,g,b): ...

Yes, I agree.
IWT it would be acceptable to limit binding to existing bindings, so that
frame structure would not have to be altered.

Looks right to me.
Still, as Carlos pointed out, formal parameter names
are private to a function,

? No they're not -- a caller knows them, in order to be able to call
with named argument syntax.
I.e., if the dict has additional content, it is not an error

Agreed, by analogy with '%(foo)s %(bar)s' % somedict.
Hm, do you want to go for a dict.get-like default value in that?

No, enriching dicts to support an optional default value (or default
value factory) looks like a separate fight to me. As does syntax sugar,
IMHO. If we can get the semantics in 2.5 with a PEP, then maybe sugar
can follow once the usefulness is established to GvR's contents.


Alex
 
J

Jeremy Bowers

ISTM that a tuple _is_ a class that wraps content for return as a _single value_,
just like a custom container class instance. What's the problem? The fact that you
can write

Generally, generally, generally, I say again, generally.

You take every example mentioned to date, which is largely:

* OS code trying to be as thin as possible (stat, socket)
* math-type code where the tuple really is the class, as you suggest
* more-or-less contrived one- or two-liner code

and you've still got way, way, way less than 1% of the code of the vast
majority of programs.

Like any other code smell, there are times to use it. But I say it's a
*smell*, and those of you implicitly reading that to mean "Returning
tuples is *never* a good idea" are doing me a disservice; please go look
up "code smell" and the word "generally".

If you use it every once in a while where it is the right solution, great.
I do too. If you're using it every other function in a module, and it
isn't a thin wrapper around some other library, you've got a code smell.
(It's probably a class or two trying to get out.) You've got a *big* code
smell if you are unpacking those return values many times, because now
you've hard coded the size of the tuple into the code in many places.

Named tuples eliminate the latter, and make the former weaker. But I
wouldn't add named tuples because of this use case, I'd add them because
we could use immutable dictionaries, and this just happens to come along
for the ride (as long as you aren't required to fully consume the tuple on
the left side, otherwise the smell remains).
 
J

Jeremy Bowers

* math-type code where the tuple really is the class, as you suggest

By the way, Bengt, just to be explicit, except for this particular "you"
the parent message is addressed generally.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,774
Messages
2,569,599
Members
45,172
Latest member
NFTPRrAgenncy
Top