Problem of Readability of Python

[Steven D'Aprano]

Point 3 clearly is harmful.

No, it is DIFFERENT, not harmful. At worst, it's a "gotcha" -- a really
well-documented gotcha.

As is the fact that __slots__ gives you
troubles if you e.g. pass objects to code that tries to set arbitrary
attributes on an object.

You mean like this?
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute 'label'

I guess that means ints and floats and strings and tuples and lists are
all harmful too, yes?

The question is: what does a slot buy you for this kind of problem?

Simplicity and explicitness.

 

[Diez B. Roggisch]

No, it is DIFFERENT, not harmful. At worst, it's a "gotcha" -- a really
well-documented gotcha.

To the casual observer? I doubt it. I wasn't aware of that until a recent
discussion about slots. But then, I've so far _never_ felt the need to use
them...

You mean like this?

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute 'label'

I guess that means ints and floats and strings and tuples and lists are
all harmful too, yes?

You are very well aware that I was talking about complex objects. And I
didn't say that they are harmful, but that using __slots__ to constrain
object attribute will lead to surprising results here in comparison to
the "usual" behavior. And with usual I mean
most-of-the-classes-work-that-way. Which might be considered as reason to
_not_ do it. But you are free to limit yourself, be my guest.

Simplicity and explicitness.

Where is that more simple? Additional notation that will lead to
runtime-errors, the same way misspelled attribute-names do?

And yes, it is more explicit. As are interfaces, and type declarations.

Diez

 

[Steven Bethard]

You can use __slots__ [...]

Aaaugh! Don't use __slots__!

Seriously, __slots__ are for wizards writing applications with huuuge
numbers of object instances (like, millions of instances).

You clipped me saying that __slots__ are for performance tweaks:

You can use __slots__ to make objects consume less memory and have
slightly better attribute-access performance. Classes for objects
that need such performance tweaks should start like...

I fully agree that __slots__ are for applications with huge numbers of
instances. But if you have that situation, you really do want to be
using __slots__.

STeVe

 

[Hrvoje Niksic]

Well, I've read the thread, and I've read the thread it links to,
and for the life of me I'm still no clearer as to why __slots__
shouldn't be used except that: [...]
But is there actually anything *harmful* that can happen if I use
__slots__?

Here is one harmful consequence: __slots__ breaks multiple
inheritance:

class A(object):
__slots__ = ['a', 'b']

class B(object):
__slots__ = ['c']

class AB(A, B):
pass

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Error when calling the metaclass bases
multiple bases have instance lay-out conflict

Even if A and B had the exact same slots, for example ['a', 'b'], it
wouldn't make a difference. AB explicitly setting __slots__ to
something like ['a', 'b', 'c'] doesn't help either. But that is only
a technical answer to your technical question which misses the real
problem people like Aahz and Guido have with __slots__. (I don't
claim to represent them, of course, the following is my
interpretation.)

The backlash against __slots__ is a consequence of it being so easy to
misunderstand what __slots__ does and why it exists. Seeing __slots__
has led some people to recommend __slots__ to beginners as a way to
"catch spelling mistakes", or as a way to turn Python's classes into
member-declared structures, a la Java. For people coming from Java
background, catching mistakes as early as possible is almost a dogma,
and they are prone to accept the use of __slots__ (and living with the
shortcomings) as a rule.

Python power users scoff at that because it goes against everything
that makes Python Python. Use of __slots__ greatly reduces class
flexibility, by both disabling __dict__ and __weakref__ by default,
and by forcing a tight instance layout that cripples inheritance.
With people using __slots__ for the majority of their classes, it
becomes much harder for 3rd-party code to attach an unforeseen
attribute to an existing object. Even with single inheritance,
__slots__ has unintuitive semantics because subclasses automatically
get __dict__ and __weakref__, thereby easily breaking the "benefits"
of their use.

__slots__ is a low-level tool that allows creation of dict-less
objects without resorting to Python/C. As long as one understands it
as such, there is no problem with using it.

 

[Licheng Fang]

...

Python Tutorial says an empty class can be used to do this. But if
namespaces are implemented as dicts, wouldn't it incur much overhead
if one defines empty classes as such for some very frequently used
data structures of the program?

Just measure:

$ python -mtimeit -s'class A(object)ass' -s'a=A()' 'a.zop=23'
1000000 loops, best of 3: 0.241 usec per loop

$ python -mtimeit -s'a=[None]' 'a[0]=23'
10000000 loops, best of 3: 0.156 usec per loop

So, the difference, on my 18-months-old laptop, is about 85 nanoseconds
per write-access; if you have a million such accesses in a typical run
of your program, it will slow the program down by about 85 milliseconds.
Is that "much overhead"? If your program does nothing else except those
accesses, maybe, but then why are your writing that program AT ALL?-)

And yes, you CAN save about 1/3 of those 85 nanoseconds by having
'__slots__=["zop"]' in your class A(object)... but that's the kind of
thing one normally does only to tiny parts of one's program that have
been identified by profiling as dramatic bottlenecks, to shave off the
last few nanoseconds in the very last stages of micro-optimization of a
program that's ALMOST, but not QUITE, fast enough... knowing about such
"extreme last-ditch optimization tricks" is of very doubtful value (and
I think I'm qualified to say that, since I _do_ know many of them....
There ARE important performance things to know about Python, but those
worth a few nanoseconds don't matter much.

Alex

This is enlightening. Surely I shouldn't have worried too much about
performance before doing some measurement.

 

[Kevin]

Am I missing something, or am I the only one who explicitly declares
structs in python?

For example:
FileObject = {
"filename" : None,
"path" : None,
}

fobj = FileObject.copy()
fobj["filename"] = "passwd"
fobj["path"] = "/etc/"

Kevin Kelley

Python is supposed to be readable, but after programming in Python for
a while I find my Python programs can be more obfuscated than their C/C
++ counterparts sometimes. Part of the reason is that with
heterogeneous lists/tuples at hand, I tend to stuff many things into
the list and *assume* a structure of the list or tuple, instead of
declaring them explicitly as one will do with C structs. So, what used
to be

struct nameval {
char * name;
int val;
} a;

a.name = ...
a.val = ...

becomes cryptic

a[0] = ...
a[1] = ...

Python Tutorial says an empty class can be used to do this. But if
namespaces are implemented as dicts, wouldn't it incur much overhead
if one defines empty classes as such for some very frequently used
data structures of the program?

Any elegant solutions?

 

[Steven Bethard]

Am I missing something, or am I the only one who explicitly declares
structs in python?

For example:
FileObject = {
"filename" : None,
"path" : None,
}

fobj = FileObject.copy()
fobj["filename"] = "passwd"
fobj["path"] = "/etc/"

Yes, I think this is the only time I've ever seen that. I think the
normal way of doing this in Python is:

class FileObject(object):
def __init__(self, filename, path):
self.filename = filename
self.path = path

fobj = FileObject(filename='passwd', path='etc')

STeVe

 

[Joe Riopel]

Am I missing something, or am I the only one who explicitly declares
structs in python?
For example:
FileObject = {
"filename" : None,
"path" : None,
}

fobj = FileObject.copy()
fobj["filename"] = "passwd"
fobj["path"] = "/etc/"

I am pretty new to python, but isn't that just a dictionary?

 

[Bjoern Schliessmann]

Am I missing something, or am I the only one who explicitly
declares structs in python?

Yes -- you missed my posting

Regards,


Björn

 

[Steven Bethard]

Yes -- you missed my posting

Actually, your posting just used dicts normally.

Kevin is creating a prototype dict with a certain set of keys, and then
copying that dict and filling in the keys each time he creates a new
instance. It's basically a poor man's OOP.

STeVe

 

[Bjoern Schliessmann]

Actually, your posting just used dicts normally.

Kevin is creating a prototype dict with a certain set of keys, and
then copying that dict and filling in the keys each time he
creates a new instance. It's basically a poor man's OOP.

And operatively, IMHO, there is no difference.

Regards,


Björn

 

[Aahz]

You can use __slots__ [...]

Aaaugh! Don't use __slots__!

Seriously, __slots__ are for wizards writing applications with huuuge
numbers of object instances (like, millions of instances).

You clipped me saying that __slots__ are for performance tweaks:

You can use __slots__ to make objects consume less memory and have
slightly better attribute-access performance. Classes for objects
that need such performance tweaks should start like...

I fully agree that __slots__ are for applications with huge numbers of
instances. But if you have that situation, you really do want to be
using __slots__.

Well, then, just make sure to put big honking warnings up whenever you
mention __slots__. ;-)

 

[kiilerix]

$ python -mtimeit -s'class A(object)ass' -s'a=A()' 'a.zop=23'

When I know that all instances of classes inheriting from object have
a namespace, then I would expect either that all objects have a
namespace or that it was inherited from object. I would expect
instantiating object to be the simplest and best way to get a
namespace.

But ...
Traceback (most recent call last):
Traceback (most recent call last):

That object is kind of "pure virtual" seems to me to be a confusing
special case without any advantages.

Why can't object be instantiated directly? Is that explained and
documented anywhere?

/Mads

 

[Chris Mellon]

When I know that all instances of classes inheriting from object have
a namespace, then I would expect either that all objects have a
namespace or that it was inherited from object. I would expect
instantiating object to be the simplest and best way to get a
namespace.

But ...

Traceback (most recent call last):

Traceback (most recent call last):


That object is kind of "pure virtual" seems to me to be a confusing
special case without any advantages.

Why can't object be instantiated directly? Is that explained and
documented anywhere?

What makes you think it can't be instantiated directly? You just did
it. It's not, however, suitable for use as an arbitrary thing to stick
attributes on.

Which is a little sad, but a necessary requirement for things like
int() and str() to be small and fast.

 

[kiilerix]

What makes you think it can't be instantiated directly? You just did
it. It's not, however, suitable for use as an arbitrary thing to stick
attributes on.

Which is a little sad, but a necessary requirement for things like
int() and str() to be small and fast.

So it's an optimization with side effects, giving a special case where
the simple and otherwise "right" way to do it doesn't work? Too bad :-
(

Ok; I'll continue to create dummy classes inheriting from object. And
hope that one day it will be simpler.

Thanks,
Mads

 

[Marc 'BlackJack' Rintsch]

So it's an optimization with side effects, giving a special case where
the simple and otherwise "right" way to do it doesn't work? Too bad :-
(

Ok; I'll continue to create dummy classes inheriting from object. And
hope that one day it will be simpler.

I'm using the following "dummy" class with a little extra functionality:

def Bunch(object):
def __init__(self, **kwargs):
self.__dict__.update(kwargs)

person = Bunch(name='Eric', age=42)
print person.name
point = Bunch(x=4711, y=23)

Ciao,
Marc 'BlackJack' Rintsch