__slots__

[David Isaac]

1. "Without a __dict__ variable,
instances cannot be assigned new variables not listed in the __slots__
definition."

So this seemed an interesting restriction to impose in some instances,
but I've noticed that this behavior is being called by some a side effect
the reliance on which is considered unPythonic. Why?

2. What is a simple example where use of slots has caused "subtle" problems,
as some claim it will?

3. What is a simple example of a Pythonic use of __slots__ that does NOT
involved the creation of **many** instances.

Thanks,
Alan Isaac

 

[Ziga Seilnacht]

1. "Without a __dict__ variable,
instances cannot be assigned new variables not listed in the __slots__
definition."

So this seemed an interesting restriction to impose in some instances,
but I've noticed that this behavior is being called by some a side effect
the reliance on which is considered unPythonic. Why?

If you want to restrict attribute asignment, you should use the
__setattr__ special method, see:
http://docs.python.org/ref/attribute-access.html

2. What is a simple example where use of slots has caused "subtle" problems,
as some claim it will?

The first point is true only if all bases use __slots__:
.... pass
........ __slots__ = ('spam',)
....

1

3. What is a simple example of a Pythonic use of __slots__ that does NOT
involved the creation of **many** instances.

Thanks,
Alan Isaac

Ziga

 

[David Isaac]

If you want to restrict attribute asignment, you should use the
__setattr__ special method, see:
http://docs.python.org/ref/attribute-access.html

That "should" is what I am asking about. If I understand,
in the simplest case, you want me to say something like
def __setattr__(self,name,value):
if name in myattrlist:
object.__setattr__(self,name,value)
else:
raise AttributeError
instead just saying
__slots__ = myattrlist
I understand that this *is* the prevailing advice. But why?

... pass
...
... __slots__ = ('spam',)
...
1

A good example of something that could be easily
missed, and possibly an answer to my question above.
(Although not I think if I am subclassing object.)

Thanks,
Alan Isaac

 

[Aahz]

That "should" is what I am asking about. If I understand,
in the simplest case, you want me to say something like
def __setattr__(self,name,value):
if name in myattrlist:
object.__setattr__(self,name,value)
else:
raise AttributeError
instead just saying
__slots__ = myattrlist
I understand that this *is* the prevailing advice. But why?

Because __slots__ breaks with inheritance.

 

[David Isaac]

Because __slots__ breaks with inheritance.

I believe that was the point of Ziga's example,
which I acknowledged as a good one in my reply.
So there still appears to be this single reason, which
applies if your class may be subclassed.

Does this beg the question of whether __slots__
*should* break with inheritance?

One other question I did not get answered: is there any
simple example of a Pythonic use of __slots__ that does NOT
involve the creation of **many** instances.

Thanks,
Alan Isaac

 

[Alex Martelli]

Does this beg the question of whether __slots__
*should* break with inheritance?

How would you expect the following code to behave:

class Base(object):
def __init__(self): self.x = 23

class Derived(Base):
__slots__ = 'y',

? I would expect it to work (as it does in Python), therefore I think
it's fit and proper that __slots__ "breaks with inheritance", meaning it
basically has no effect unless every class in the inheritance DAG has
slots.

One other question I did not get answered: is there any
simple example of a Pythonic use of __slots__ that does NOT
involve the creation of **many** instances.

Since the only benefit of __slots__ is saving a few bytes per instance,
it's not worth the bother unless there are many instances -- so, the
answer is 'no'.


Alex

 

[Ron Garret]

Since the only benefit of __slots__ is saving a few bytes per instance,
it's not worth the bother unless there are many instances -- so, the
answer is 'no'.

I can think of at least two other benefits to using __slots__:

1. If you have a typo in an attribute assignment you get an exception
instead of a latent downstream bug.

2. Implicit documentation.

rg

 

[Alex Martelli]

Since the only benefit of __slots__ is saving a few bytes per instance,
it's not worth the bother unless there are many instances -- so, the
answer is 'no'.

I can think of at least two other benefits to using __slots__:

1. If you have a typo in an attribute assignment you get an exception
instead of a latent downstream bug.[/QUOTE]

If your unittests are so feeble that they won't catch such typos, you
have far bigger problems -- and you should be using pychecker or pylint
anyway, as they'll catch far more typos than __slots__ ever will (far
from all, of course -- a simple typo of + vs - can still kill you --
which is why they can't *substitute* for unittests in any case).

2. Implicit documentation.

If you don't document what the sundry variables are FOR, you're really
not documenting your code at all -- just listing the names of some
attributes is far too weak. If the existence of such listing can in any
way give the programmer an excuse to NOT do real documentation (as your
classifying it as "implicit documentation" strongly suggests), then the
net effet is not a benefit, but a serious detriment to code quality.


Alex

 

[Duncan Booth]

I can think of at least two other benefits to using __slots__:

1. If you have a typo in an attribute assignment you get an exception
instead of a latent downstream bug.

Only if all classes in the inheritance define __slots__, and don't include
__dict__ in their slots. In particular, it means you can never rely on
this in any code you write which inherits from a library class over which
you have no control.

In other words, it is such a risky thing to depend on that you would be
much better never to rely on it. Try writing some unit tests instead.

2. Implicit documentation.

Explicit is better than implicit.

 

[John J. Lee]

you should be using pychecker or pylint

[...]

I'm curious, as somebody who doesn't regularly use these tools: How do
they fit into your workflow? Do you run them every few hours, every
day, every time you run functional tests, every release, every so
often, on gut feeling about when it's likely to catch problems...?

How do you cope with spurious warnings? Does it involve tweaking code
to quell warnings? Keeping suppression lists? Maintaining special
invocations of pychecker / pylint per-project? Do they cope well with
other people's code who do not use these tools?


John

 

[Alex Martelli]

you should be using pychecker or pylint

[...]

I'm curious, as somebody who doesn't regularly use these tools: How do
they fit into your workflow? Do you run them every few hours, every
day, every time you run functional tests, every release, every so
often, on gut feeling about when it's likely to catch problems...?

I use them before mailing code off for review (which in our workflow is
mandatory before the code is submitted into the codebase) -- not quite
as often as unittests, which I run with just about every keystroke in my
editor;-). I'd rather have such a tool run automatically, as a presubmit
check, but that depends on convincing every colleague that it's worth it
(probably hopeless for pychecker as long as it needs to i

How do you cope with spurious warnings? Does it involve tweaking code
to quell warnings? Keeping suppression lists? Maintaining special
invocations of pychecker / pylint per-project? Do they cope well with
other people's code who do not use these tools?

If said "other people's code", for example, does pointless (but
hopefully innocuous, otherwise unittests would have caught that;-)
things such as importing modules it never uses, the warning tools
complain. I do not consider such complaints "spurious", and prefer to
fix those things (sending the brief codereview to the original author
with advice to start using the checking tools -- it usually works;-).

Very occasionally, there _are_ warnings which are indeed spurious: the
typical example is a function which, to comply with a certain API, MUST
take three arguments named exactly x, y, and z (no more, no less, not
with any different names) but only needs x; in such cases, you do need
to add, e.g.,
__pychecker__ = 'unusednames=y,z'
I pay the price gladly, since this will also serve to reassure human
readers that my code's anomalous behavior (accepting arguments y and z
but ignoring them) is by design, and not an accident or mistake. I do
just about the same thing with the lint variant we use for C++, btw.


Alex

 

[Ben Caradoc-Davies]

I'm curious, as somebody who doesn't regularly use these tools: How do
they fit into your workflow? Do you run them every few hours, every
day, every time you run functional tests, every release, every so
often, on gut feeling about when it's likely to catch problems...?

I use pychecker when I have finished a chunk of work that is supposed to
be syntactically correct, but before I try any unit testing. Think of it
as an optional step that takes the place of running the compiler in
statically-typed language development. I use pytchecker again before I
release anything, just to be sure.

How do you cope with spurious warnings? Does it involve tweaking code
to quell warnings? Keeping suppression lists?

Suppression lists work for me.

From http://pychecker.sourceforge.net/

*** begin quote ***

You can also define suppressions in your code by doing:

__pychecker__ = 'no-namedargs maxreturns=0 unusednames=foo,bar'

*** end quote ***

Do they cope well with other people's code who do not use these tools?

pychecker complains a lot when used on code that is poorly written and
includes bad practices such as using builtin function names as local
variable names. I consider pychecker's complaints in these cases to be a
*feature*.

 

[Antoon Pardon]

you should be using pychecker or pylint

[...]

I'm curious, as somebody who doesn't regularly use these tools: How do
they fit into your workflow? Do you run them every few hours, every
day, every time you run functional tests, every release, every so
often, on gut feeling about when it's likely to catch problems...?

How do you cope with spurious warnings? Does it involve tweaking code
to quell warnings? Keeping suppression lists? Maintaining special
invocations of pychecker / pylint per-project? Do they cope well with
other people's code who do not use these tools?

IMO they don't cope well unless you adapt to what they expect.

I just tested the two.

pylint gave me countless warnings because I use two space to indent
instead of four. It also didn't like that I used uppercase letters
in my identifiers. That there was no space after a comma or
around an operator was a problem too.

That I use the same name for the index variable of for loops in
different scopes is also warned about.


Pychecker also warns about these shadowed variable. Pychecker
goes even so far to warn you that a local variable in an imported
module is shadowing a global variable of yours. Like the following
line:

/usr/lib/python2.3/random.py:247: Local variable (i) shadows global
defined on line 43 in file demo1a.py

So it seems pychecker thinks I shouldn't use something like

for i in lst:
...

in my code at the global level because some module in the standard
library has a function with a local i.

Pychecker also froze on my system.


I don't recommend the use of these tools.

 

[Georg Brandl]

you should be using pychecker or pylint

[...]

I'm curious, as somebody who doesn't regularly use these tools: How do
they fit into your workflow? Do you run them every few hours, every
day, every time you run functional tests, every release, every so
often, on gut feeling about when it's likely to catch problems...?

How do you cope with spurious warnings? Does it involve tweaking code
to quell warnings? Keeping suppression lists? Maintaining special
invocations of pychecker / pylint per-project? Do they cope well with
other people's code who do not use these tools?

IMO they don't cope well unless you adapt to what they expect.

I just tested the two.

pylint gave me countless warnings because I use two space to indent
instead of four. It also didn't like that I used uppercase letters
in my identifiers. That there was no space after a comma or
around an operator was a problem too.

Pylint knows different types of issues, these are "code style" issues
and can be
- turned off
- configured to what you like.
They help getting your coding style consistent and default to PEP8 style.

That I use the same name for the index variable of for loops in
different scopes is also warned about.

Well, everything that _could_ be problematic _or_ confuse a reader is
warned about. Pretty good in my eyes.

Pychecker also warns about these shadowed variable. Pychecker
goes even so far to warn you that a local variable in an imported
module is shadowing a global variable of yours. Like the following
line:

/usr/lib/python2.3/random.py:247: Local variable (i) shadows global
defined on line 43 in file demo1a.py

That's odd. How are the two modules related?

So it seems pychecker thinks I shouldn't use something like

for i in lst:
...

in my code at the global level because some module in the standard
library has a function with a local i.

Pychecker also froze on my system.

Pychecker imports the modules. Thus these things can happen when a
module expects not to be imported as-is.

I don't recommend the use of these tools.

Well, then I don't recommend anyone reading your code <wink>

Georg

 

[Antoon Pardon]

(e-mail address removed) (Alex Martelli) writes:
[...]
you should be using pychecker or pylint
[...]

I'm curious, as somebody who doesn't regularly use these tools: How do
they fit into your workflow? Do you run them every few hours, every
day, every time you run functional tests, every release, every so
often, on gut feeling about when it's likely to catch problems...?

How do you cope with spurious warnings? Does it involve tweaking code
to quell warnings? Keeping suppression lists? Maintaining special
invocations of pychecker / pylint per-project? Do they cope well with
other people's code who do not use these tools?

IMO they don't cope well unless you adapt to what they expect.

I just tested the two.

pylint gave me countless warnings because I use two space to indent
instead of four. It also didn't like that I used uppercase letters
in my identifiers. That there was no space after a comma or
around an operator was a problem too.

Pylint knows different types of issues, these are "code style" issues
and can be
- turned off
- configured to what you like.
They help getting your coding style consistent and default to PEP8 style.

I have been on their website in the documentation section and found
nothing that could easily help me here.

Well, everything that _could_ be problematic _or_ confuse a reader is
warned about. Pretty good in my eyes.

No it isn't. If most of these line are in fact not problematic
or confusing at all then people will in general no longer
take notice of those kind of lines.

If I use a specific variable like "i" only as an index in for
loops, there is no reason to think this is problematic or confusing
even if those loops are in different scopes. And all those lines
reporting something that is not a problem will detract from the
few lines that may be a real problem.

That's odd. How are the two modules related?

the program demo1a.py contains the following line:

from random import Random, randint, sample

Pychecker imports the modules. Thus these things can happen when a
module expects not to be imported as-is.

I still find it unacceptable.

 

[Klaas]

3. What is a simple example of a Pythonic use of __slots__ that does NOT
involved the creation of **many** instances.

mu. Your question presupposes the existence of such an example.

-Mike