Proposal: reducing self.x=x; self.y=y; self.z=z boilerplate code

Discussion in 'Python' started by Ralf W. Grosse-Kunstleve, Jul 2, 2005.

  1. ******************************************************************************
    This posting is also available in HTML format:

    Hi fellow Python coders,

    I often find myself writing::

    class grouping:

    def __init__(self, x, y, z):
    self.x = x
    self.y = y
    self.z = z
    # real code, finally

    This becomes a serious nuisance in complex applications with long
    argument lists, especially if long variable names are essential for
    managing the complexity. Therefore I propose that Python includes
    built-in support for reducing the ``self.x=x`` clutter. Below are
    arguments for the following approach (*please* don't get too agitated
    about the syntax right here, it really is a secondary consideration)::

    class grouping:

    def __init__(self, .x, .y, .z):
    # real code right here

    Emulation using existing syntax::

    def __init__(self, x, y, z):
    self.x = x
    del x
    self.y = y
    del y
    self.z = z
    del z

    Is it really that important?

    For applications of non-trivial size, yes. Here is a real-world example
    (one of many in that source tree):

    Fragment from this file::

    class manager:

    def __init__(self,
    self.crystal_symmetry = crystal_symmetry
    self.model_indices = model_indices
    self.conformer_indices = conformer_indices
    self.site_symmetry_table = site_symmetry_table
    self.bond_params_table = bond_params_table
    self.shell_sym_tables = shell_sym_tables
    self.nonbonded_params = nonbonded_params
    self.nonbonded_types = nonbonded_types
    self.nonbonded_function = nonbonded_function
    self.nonbonded_distance_cutoff = nonbonded_distance_cutoff
    self.nonbonded_buffer = nonbonded_buffer
    self.angle_proxies = angle_proxies
    self.dihedral_proxies = dihedral_proxies
    self.chirality_proxies = chirality_proxies
    self.planarity_proxies = planarity_proxies
    self.plain_pairs_radius = plain_pairs_radius
    # real code, finally

    Not exactly what you want to see in a high-level language.

    Is there a way out with Python as-is?

    Yes. If you take the time to look at the file in the CVS you'll find
    that I was cheating a bit. To reduce the terrible clutter above, I am
    actually using a simple trick::

    adopt_init_args(self, locals())

    For completeness, the implementation of ``adopt_init_args()`` is here:

    While this obviously goes a long way, it has several disadvantages:

    - The solution doesn't come with Python -> everybody has to reinvent.

    - People are reluctant to use the trick since scripts become
    dependent on a non-standard feature.

    - It is difficult to remember which ``import`` to use for
    ``adopt_init_args`` (since everybody has a local version/variety).

    - The ``adopt_init_args(self, locals())`` incantation is hard to
    remember and difficult to explain to new-comers.

    - Inside the ``__init__()`` method, the same object has two names,
    e.g. ``x`` and ``self.x``. This lead to subtle bugs a few times
    when I accidentally assigned to ``x`` instead of ``self.x`` or vice
    versa in the wrong place (the bugs are typically introduced while

    - In some cases the ``adopt_init_args()`` overhead was found to
    introduce a significant performance penalty (in particular the
    enhanced version discussed below).

    - Remember where Python comes from: it goes back to a teaching
    language, enabling mere mortals to embrace programming.
    ``adopt_init_args(self, locals())`` definitely doesn't live up
    to this heritage.

    Minimal proposal

    My minimal proposal is to add an enhanced version of ``adopt_init_args()``
    as a standard Python built-in function (actual name secondary!)::

    class grouping:

    def __init__(self, x, y, z):
    # real code

    Here is a reference implementation:

    Implementation of this proposal would remove all the disadvantages
    listed above. However, there is another problem not mentioned before:
    It is cumbersome to disable adoption of selected variables. E.g.::

    class grouping:

    def __init__(self, keep_this, and_this, but_not_this, but_this_again):
    self.keep_this = keep_this
    self.and_this = and_this
    self.but_this_again = but_this_again
    # real code, finally

    would translate into::

    class grouping:

    def __init__(self, keep_this, and_this, but_not_this, but_this_again):
    # real code

    Enhanced syntax proposal

    The exclusion problem suggests these alternatives::

    class grouping:

    def __init__(self, self.keep_this, self.and_this, but_not_this,
    # real code right here

    This is conceptually similar to the existing automatic unpacking of tuples.

    A shorter alternative (my personal favorite since minimally redundant)::

    class grouping:

    def __init__(self, .keep_this, .and_this, but_not_this, .but_this_again):
    # real code right here

    I guess both versions could be implemented such that users don't incur
    a performance penalty compared to the ``self.x=x`` alternative. At the
    danger of being overly optimistic: I can imagine that my favorite
    alternative will actually be faster (and the fastest).

    Enhanced __slot__ semantics proposal

    When ``__slots__`` are used (cool feature!) the boilerplate problem
    becomes even worse::

    class grouping:

    __slots__ = ["keep_this", "and_this", "but_this_again"]

    def __init__(self, keep_this, and_this, but_not_this, but_this_again):
    self.keep_this = keep_this
    self.and_this = and_this
    self.but_this_again = but_this_again
    # real code, finally

    Each variable name appears four times! Imagine yourself having to
    do this exercise given the real-world example above. Ouch.

    Based on the "Enhanced syntax proposal" above I see this potential

    class grouping:

    __slots__ = True

    def __init__(self, .keep_this, .and_this, but_not_this, .but_this_again):
    # real code right here

    Each variable name appears only once. Phew!

    Author: , July 02, 2005

    P.S.: If you reply to this message, please clearly separate
    naming/syntax issues from the core issue of providing built-in support
    designed to reduce clutter.

    Yahoo! Sports
    Rekindle the Rivalries. Sign up for Fantasy Football
    Ralf W. Grosse-Kunstleve, Jul 2, 2005
    1. Advertisements

  2. Ralf W. Grosse-Kunstleve

    Roy Smith Guest

    I'm really torn about this. On the one hand, my first thought was "you
    shouldn't be writing constructors with arguments lists so long that this is
    a problem", but you showed a reasonable counter-example to that argument.
    I'm a big fan of DRY (Don't Repeat Yourself), sometimes expressed as "Once
    And Only Once", and your proposal lets you do that.

    It also has the nice feature that it doesn't break any existing code; the
    suggested syntax is not currently legal Python.

    What happens if I do:

    def __init__ (self, .x, .y, .z):
    x = 0

    what does the assignment x do? Does it automatically get promoted to an
    assignment to self.x? Does it generate an error?

    The big question in my mind is not "Is this useful" (since it clearly is),
    but "Does the utility justify the cost?". In other words, will it be used
    frequently enough to compensate for the added complexity to the language?
    I'm not convinced of that.

    There have been some proposals floating around to implement a "with"
    keyword which would create implicit namespaces. That's sort of what you're
    proposing here. I'm not convinced either is a good idea, but if they were
    to be adopted, I'd certainly want to see the them done in a uniform,
    logically consistent way.
    Roy Smith, Jul 2, 2005
    1. Advertisements

  3. I think this is a bad idea, for a subtle reason.

    In Python, unlike many other languages, the names of formal parameters are
    part of a function's interface. For example:

    def f(x, y):
    return x-y

    Now f(3, 4) is -1 and f(y=3,x=4) is 1.

    The names of instance variables are generally not part of a class'
    interface--they are part of its implementation.

    This proposed feature, whenever used, would tie a class' implementation to
    the interface of every method that uses the feature. As far as I can see,
    it is impossible to use the feature without constraining the implementation
    in this way.

    For this reason, I would much rather have the mapping between parameter
    names and instance variables be explicit.
    Andrew Koenig, Jul 2, 2005
  4. Ralf W. Grosse-Kunstleve

    Roy Smith Guest

    While I suppose that's true from a theoretical point of view, as a
    practical matter, I don't see it being much of a big deal. I don't think
    I've ever written an __init__ method which saved its parameters and used
    different names for the parameter and the corresponding instance variable.
    Doing so would just be confusing (at least for the kind of code I write).

    Also, it doesn't really tie it in any hard and fast way. Right now, I
    would write:

    def __init__ (self, x, y, z):
    self.x = x
    self.y = y
    self.z = z

    under the new proposal, I would write:

    def __init__ (self, .x, .y, .z):

    If at some time in the future, if I decided I need to change the name of
    the instance variable without changing the exposed interface, it would be
    easy enough to do:

    def __init__ (self, .x, .y, z):
    self.zeta = z

    I'm still not convinced we need this, but the exposed interface issue
    doesn't worry me much.
    Roy Smith, Jul 2, 2005
  5. Why not just update the local dictionary?

    class Grouping:
    def __init__(self,x,y,z):
    Walter Brunswick, Jul 2, 2005
  6. Ralf W. Grosse-Kunstleve

    Ron Adam Guest

    The way this would work seems a bit inconsistent to me. Args normally
    create local variables that receive references to the objects passed to

    In this case, a function/method is *creating* non local names in a scope
    outside it's own name space to receive it's arguments. I don't think
    that's a good idea.

    A better approach is to have a copy_var_to(dest, *var_list) function
    that can do it. You should be able to copy only selected arguments, and
    not all of them.


    Not exactly straight forward to do as it runs into the getting an
    objects name problem.

    The 'del's aren't needed as the references will be unbound as soon as
    __init__ is finished. That's one of the reasons you need to do self.x=x
    , the other is to share the objects with other methods.

    With argument lists that long it might be better to use a single
    dictionary to keep them in.

    class manager:
    def __init__(self, **args):
    defaults = {
    'plain_pairs_radius':None }
    defaults.update(args) = defaults

    # real code

    Ron Adam, Jul 2, 2005
  7. Ralf W. Grosse-Kunstleve

    Roy Smith Guest

    That's pretty clever. The only minor annoyance is that it creates a
    self.self. If that bothers you, you can fix it with:

    def __init__ (self, x, y, z):
    vars = locals()
    del vars["self"]

    or, perhaps:

    def __init__ (self, x, y, z):
    del self.self

    It doesn't give you all the flexibility of the original proposal (i.e.
    name-by-name selectivity of what gets imported into self), but it does
    solve the OP's OP (Original Poster's Original Problem).
    Roy Smith, Jul 2, 2005
  8. Ralf W. Grosse-Kunstleve

    jcarlson Guest

    There is also the variant which I proposed on python-dev:

    class grouping:
    def __init__(self, _keep_this, _and_this, but_not_this, _but this
    InitAttrs(self, locals())
    #real code goes here

    Essentially you replace the '.' with '_', which doesn't require a
    syntax change. Unfortunately, both are almost invisible. It does
    offer you what you want right now (without that whole waiting a year+
    for Python 2.5, PEP process, etc.).
    There is also the AutoSlots metaclass (which I have fixed) that does
    this as well.

    class grouping(object):
    __metaclass__ = AutoSlots
    def __init__(self, _keep_this, _and_this, but_not_this,
    InitAttrs(self, locals())
    #real code goes here

    Both AutoSlots and InitAttrs use leading underscores on the __init__
    method to discover which attributes are to be __slots__, and which
    should be automatically assigned.
    Because you don't seem to have listened in python-dev, I'll say it
    here. Not everything that reduces clutter should be syntax, and not
    every function, class, and module which reduces programming time should
    be builtin. Why?

    1. Expanding Python syntax bloats the language. It increases what you
    need to teach to new Python users. In my opinion, syntax additions
    should really only be considered when significant gains in readability
    and writability for many users are realized, that a lack of syntax
    cannot offer.

    2. Expanding the Python standard library bloats the distribution.
    Right now, Python is a relatively light download. But if one were to
    include the top packages in everything, the distribution would quickly
    bloat to 40+ megs. This is not an option. Generally, the requirements
    of getting code into the standard library is either a demonstrated need
    for the addition, or a known best-of-breed implementation for a
    commonly used module, or both.

    I believe your syntax change is a non-starter. Why? Because I've
    offered code that does essentially everything you want, without a
    syntax change. If someone happens to include AutoSlots and InitAttrs
    with their code, module, what have you, and it manages to make its way
    into standard Python, so be it (I place the code into the public

    The code for the InitAttrs and AutoSlots mechanism are available here:

    - Josiah
    jcarlson, Jul 2, 2005
  9. With some help from new-style classes you can get more than
    just removing the "self.x = x" clutter.

    I'm not an expert of these low-level python tricks, but
    you can download from a
    small example that allows you to write

    class MyClass(Object):
    x = Float(default = 0.0, max = 1E20)
    y = Float(min = 1.0)

    and you can get in addition of redudancy removal also
    parameter checking. You can also have an __init__
    method that gets called with attributes already set up.

    Andrea Griffini, Jul 3, 2005
  10. What if parameter name syntax were expanded to allow dotted names as binding
    targets in the local scope for the argument or default values? E.g.,

    def foometh(self, self.x=0, self.y=0): pass

    would have the same effect as

    def foometh(self, self.y=0, self.x=0): pass

    and there would be a persistent effect in the attributes of self
    (whatever object that was), even with a body of pass.

    I'm not sure about foo(self, **{'self.x':0, 'self.y':0}), but if
    you didn't capture the dict with a **kw formal parameter, IWT you'd
    have to be consistent and effect the attribute bindings implied.

    (Just a non-thought-out bf here, not too serious ;-)

    Bengt Richter
    Bengt Richter, Jul 3, 2005
  11. Ralf W. Grosse-Kunstleve

    Ron Adam Guest

    Well it works the other way around to some degree.

    def foo(self, x=x, y=y):pass

    x=x binds the class variables to the arguments without the self. if no
    value is given.

    Which is kind of strange, since x by it self gives an error if no value
    is given. The strange part is x=x is not the same as just x. I
    understand why, but it still looks odd.

    Why isn't there a dict method to get a sub dict from a key list?
    Fromkeys doesn't quite do it.

    sub-dict = dict.subdict(key_list)

    Or have dict.copy() take a key list. (?)

    <Just a thought.>

    The following works and doesn't seem too obscure, although the x=x,
    etc.. could be annoying if they were a lot of long names.

    Seems like mutable default arguments is what's needed to make it work,
    not that it's needed IMO. But it's an interesting problem.

    def subdict(dict, keys):
    d = {}
    for k in keys:
    d[k] = dict[k]
    return d

    class foo(object):
    x = 1
    y = 2
    z = 3
    def __init__(self,x=x,y=y,z=z):
    save_these = subdict(locals(),['x','y'])

    # rest of code
    print self.x, self.y, self.z

    f = foo()
    f = foo(5,6,7)
    Ron Adam, Jul 3, 2005
  12. ISTM you are comparing apples to oranges, execution-wise. The def in the
    context of a class definition is executed as soon as you get to the end
    of the class suite. The execution happens in a special name space similar
    to what happens when you execute a function suite by calling the function,
    except the class definition body is executed automatically.

    When you write def foo(self, x=x): ... the second x is looked up
    starting in the local excution namespace of the class body, so it finds
    class variables, if they are defined, otherwise it looks for an outer
    scope x for the value.

    Note that order counts:
    ... def foo(self, x=x): print x
    ... x = 123
    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    ... x = 123
    ... def foo(self, x=x): print x
    ... 123

    Note that self.x is a different access mechanism, and would access the
    same Foo.x that became a default value, but the value is the only thing
    they have in common. You could extract the function from the class,
    and the plain function would still have the default value:
    3 0 LOAD_FAST 1 (x)
    5 LOAD_CONST 0 (None)
    8 RETURN_VALUE (['self', 'x'], None, None, (123,))

    There's where the default value is. It's put there when x=x is evaluated
    during the def execution for the function, which happens to have happened
    courtesy of a class body execution defining the class, but that's
    the only connection the 123 has.
    sub_dict = dict([(key, srcdct[key]) for key in srcdct]) #untested, should work
    (if you don't use 'dict' for a variable ;-)
    self.__dict__.update({'x':x, 'y':y, 'z':z}) # should work without save_these
    Bengt Richter
    Bengt Richter, Jul 4, 2005
  13. oops, I see you only wanted x and y, so that should be
    self.__dict__.update({'x':x, 'y':y}) # should work without save_these
    Bengt Richter
    Bengt Richter, Jul 4, 2005
  14. Ralf W. Grosse-Kunstleve

    mcherm Guest

    Yes... indeed it does. This is so common that there is a standard
    idiom for handling it:

    def __init__(self, x, y, z):

    sometimes with modifications to avoid setting self.self.
    If all you were proposing was a built-in function to make this
    idiom clearer and more reliable, then I think I'd back such a feature
    because the need is SO common. However, the suggestion you actually
    is far too broad and introduces new syntax unnecessarily.

    You yourself are using a helper function (although I belive it could
    be done more easily than you did it):
    To which you raise the following objections:
    Good point. Particularly since people won't think of all the
    special cases (eg: classes with __slots__ defined).

    If the implementation is only 3-4 lines long (and a simpler
    can be), then is can simply be included inline with every script that
    to use it.
    A better name would help with this. The need for locals() is
    But for REAL beginners, I wouldn't even bother... writing out "self.x =
    is useful for beginners since it helps make it very clear and concrete
    them just what is happening.
    Hmm... I've never had that problem, myself.
    Again... a different code will help here. And if execution speed is
    REALLY a concern, then you can just write it out the long way!
    No, but "self.x = x" does. It's only when you have lots of variables
    or very long names that this approach becomes unwieldy.
    I'd alter the name and the implementation, but the basic idea seems
    sound to me.
    The VERY simple, VERY straightforward, VERY common behavior of
    "store all the arguments as like-named attributes of self" is
    worth having a standard idiom (and *perhaps* a built-in). But
    odd special cases like skipping some arguments... that calls
    for writing the whole thing out. I'm firmly -1 on any proposal
    to support skipping arguments.
    ** NO! **

    __slots__ is *NOT* to be used except for those times when you NEED
    the performance advantages (mostly memory use). The simple rule is
    that you should *NEVER* use __slots__ (if you are in a situation
    where you *do* need it, then you'll know enough to understand why
    this advice doesn't apply to you). There should NOT be any support
    for auto-setting __slots__ *anywhere* in the standard library,
    because it would make it FAR too tempting for people to mis-use

    Besides, a metaclass would be a better solution, and it can be done
    today with no modifications to Python.

    .. . .

    All in all, I think there's SOME merit to this idea, in that this
    is a common enough practice that it might be nice to make it easy
    to type (and read). But your proposal entangles the good idea with
    several ideas I rather dislike, and on the whole I think it sounds
    rather dangerous.

    -- Michael Chermside
    mcherm, Jul 5, 2005
  15. Ahem - sys._getframe()

    Thomas Heller, Jul 5, 2005
  16. Ralf W. Grosse-Kunstleve

    NickC Guest


    I'd be very interested to hear your opinion on the 'namespace' module,
    which looks at addressing some of these issues (the Record object, in
    particular). The URL is, and any
    comments should be directed to the
    discussion list.

    NickC, Jul 6, 2005
  17. IIRC, the self.__dict__.update(locals()) trick confuses psyco.

    But you can make a decorator to achieve the same result. There's not
    really a convincing case for extending python syntax.

    def attribute_decorator(f):
    import inspect
    argnames = inspect.getargspec(f)[0]
    def decorator(*args, **keywords):
    bound_instance = args[0]
    for name, value in zip(argnames[1:], args[1:]):
    setattr(bound_instance, name, value)
    return f(*args, **keywords)
    return decorator

    #--------- example use:

    class foo(object):
    def __init__(self, thing):
    print "init: self.thing is", repr(self.thing)

    f = foo('hello world')

    Lonnie Princehouse, Jul 11, 2005
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.