module not callable - why not?

[Diez B. Roggisch]

Hi,

I just thought about creating a module for quaternions (as an personal
exercise, so I'm not after pointers to classlibs here).

Now usually I'd create a file called "quaternion.py", define my quaternion
class in there and then import and create an instance like this:

import quaternion

q = quaternion.quaternion()

Thats a lot to type. doing a

from quaternion import quaternion

would solve that - but AFAIK thats considered bad for some reasons.

Now I thought about defining a __call__ operator at module level to allow
this:

import quaternion

q = quaternion()


where the call looks like this:

def __call__(*a, *kw):
return quaternion()

But that doesn't work.

Now my question is: Is there a way to make a module callable that way? And
wouldn't it make sense to allow the implementation of a call operator on
module level?

 

[djw]

Hi,

I just thought about creating a module for quaternions (as an personal
exercise, so I'm not after pointers to classlibs here).

Now usually I'd create a file called "quaternion.py", define my quaternion
class in there and then import and create an instance like this:

import quaternion

q = quaternion.quaternion()

Thats a lot to type. doing a

from quaternion import quaternion

would solve that - but AFAIK thats considered bad for some reasons.


Now I thought about defining a __call__ operator at module level to allow
this:

import quaternion

q = quaternion()


where the call looks like this:

def __call__(*a, *kw):
return quaternion()

But that doesn't work.

Now my question is: Is there a way to make a module callable that way? And
wouldn't it make sense to allow the implementation of a call operator on
module level?

I know this has been discussed before (using __main__() instead). Try
googling for some discussions.

-Don

 

[Peter Hansen]

I think what people consider dangerous is 'from <module> import *'. The
form you give here is OK, as far as I know.

Even "from xxx import yyy" can be dangerous if you don't know
what you're doing. More specifically, any time the yyy thing
might be dynamic (i.e. replace in the original module with
something else at a later time), you will end up with a situation
in which everyone who did the "from xxx import yyy" before the
change has a binding to something other than what xxx.yyy is
currently bound to.

Generally speaking it's safe to do (and so is "from xxx import *")
but considered poor form, extremely so in the latter case. And
it's not an arbitrary thing: doing this makes code less readable
and maintainable because it becomes unclear where names are
coming from without constant reference to the list of imports
at the top.

-Peter

 

[Terry Reedy]

Hi,

I just thought about creating a module for quaternions (as an personal
exercise, so I'm not after pointers to classlibs here).

Now usually I'd create a file called "quaternion.py", define my quaternion
class in there and then import and create an instance like this:

import quaternion
q = quaternion.quaternion()

Thats a lot to type. doing a

which is why I might add 'as q' to the import

from quaternion import quaternion
would solve that - but AFAIK thats considered bad for some reasons.

By who? and why would you let whoever override your preference ;-)
I have only read suggestions that one be careful with from x import *.

Now my question is: Is there a way to make a module callable that way?

No. And I believe Guido has said he doesn't want modules to be classes, or
more classlike.

>And wouldn't it make sense to allow the implementation
of a call operator on module level?

To you it seems to ;-)

Terry J. Reedy

 

[John Roth]

Hi,

I just thought about creating a module for quaternions (as an personal
exercise, so I'm not after pointers to classlibs here).

Now usually I'd create a file called "quaternion.py", define my quaternion
class in there and then import and create an instance like this:

import quaternion

q = quaternion.quaternion()

Thats a lot to type. doing a

from quaternion import quaternion

would solve that - but AFAIK thats considered bad for some reasons.

I don't know why that's bad. Its fairly common in fact.
What's usually bad is "from foobar import *" which loads
your module namespace with a bunch of identifiers that
are not documented in the source of that module, and that
may change if the imported module changes.

In fact, I'd probably do: "from quaternion import quaternion as q"
to minimize typing later.

Now I thought about defining a __call__ operator at module level to allow
this:

import quaternion

q = quaternion()


where the call looks like this:

def __call__(*a, *kw):
return quaternion()

But that doesn't work.

Now my question is: Is there a way to make a module callable that way? And
wouldn't it make sense to allow the implementation of a call operator on
module level?

No, and probably not.

There's no really earthshaking reason why a module object
couldn't be a callable, and I suspect it would be rather simple
to do. However, it would add some complexity to the model,
and IMO your use case isn't compelling enough.

Other people may disagree with me on that, though.

John Roth

 

[Peter Hansen]

Generally speaking it's safe to do (and so is "from xxx import *")
but considered poor form, extremely so in the latter case. And
it's not an arbitrary thing: doing this makes code less readable
and maintainable because it becomes unclear where names are
coming from without constant reference to the list of imports
at the top.

I think for the simple "from xxx import yyy" I've overstated
the case somewhat, especially given that this is something I
do a lot too.

Rather than calling it "poor form", I would say merely that
overuse of it can lead to code becoming less clear. For the
usual cases, including John Roth's example with "as q", it's
not poor form if used relatively sparingly, and can actually
increase readability in those cases.

-Peter

 

[John Roth]

I know this has been discussed before (using __main__() instead). Try
googling for some discussions.

The discussion on __main__() isn't quite the same thing. If I remember
correctly, the use case for __main__ was to get rid of the two line suffix
to a script to make it executable. This is rather obscure to novices, while
__main__() would be a bit more understandable. At least, it's similar
to what other languages do for a startup function. There was no intention
that __main__() be called on an import, or that it be accessable via the
normal call syntax on the module object. At least, I don't remember such.

John Roth

 

[Jack Diederich]

I don't know why that's bad. Its fairly common in fact.
What's usually bad is "from foobar import *" which loads
your module namespace with a bunch of identifiers that
are not documented in the source of that module, and that
may change if the imported module changes.

In fact, I'd probably do: "from quaternion import quaternion as q"
to minimize typing later.

Naming style and imports came up on python-dev recently.
There were some good suggestions to make life easier:

- Don't name the module and a class in it the same thing
import Queue # Queue is a module
from Queue import Queue # Queue is a class

.. 50 lines later ...

ob = Queue() # do I want to do this?
ob = Queue.Queue() # or this?

- Do name modules more generically than the first class you put in them.
from your example a good name may be 'quatmath' because there will
also be some methods to manipulate Quaternion instances in there too.
import quatmath
q1 = quatmath.Quaternion(a, bi, ck, dk)
q2 = quatmath.Quaternion(a*2, bi, ck, dk*2)
result = quatmath.concat(q1, q2)

As a personal preference I'd avoid overriding __add__ __mult__ and
other operators in the Quaternion class. It is easy to forget what
'q1 * q2' is doing but hard to miss 'quatmath.product(q1, q2)'.

-jackdied

 

[Hans Nowak]

Now I thought about defining a __call__ operator at module level to allow
this:

import quaternion

q = quaternion()


where the call looks like this:

def __call__(*a, *kw):
return quaternion()

But that doesn't work.

Now my question is: Is there a way to make a module callable that way? And
wouldn't it make sense to allow the implementation of a call operator on
module level?

Is is possible, if you use packages, and are willing to use some evil hackery.
Here's what to do:

1. Create a package 'quaternion'.

2. Create quaternion/__init__.py:

# quaternion/__init__.py

from types import ModuleType
import os, sys

class QuaternionModule(ModuleType):
def __init__(self):
import _quaternion
self.__name__ = "_quaternion proxy"
self.__file__ = __file__
self.__base = os.path.dirname(__file__)
self.__module = sys.modules['quaternion']

def __call__(self):
return _quaternion.quaternion()

sys.modules['quaternion'] = QuaternionModule()
del sys, os, ModuleType, QuaternionModule

3. Create quaternion/_quaternion.py:

# _quaternion.py

class quaternion:
def foo(self, x):
return x*2

(obviously a dummy implementation, but you get the idea)

4. Test it:

Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
To be fair, this is an adaptation of some code sent to me by Olivier Poyen. It
was originally a construct to add dynamic import to my Wax library (which
currently imports a bunch of modules in its __init__.py, taking up time and
space, whether you use them or not).

Now, whether this good coding practice is something else entirely...

Cheers,

 

[Hung Jung Lu]

- Don't name the module and a class in it the same thing
import Queue # Queue is a module
from Queue import Queue # Queue is a class
...

To be frank, this problem is just one of the symptoms of a sickened
programming language. Python runs into this problem (like
SimpleXMLRPCServer.SimpleXMLRPCServer) because:

(a) By historical accident, Python modules differ significantly from
regular classes. (Modules are like singleton without constructor for
multiple instantiation, and without property getters/setters.)

(b) Python uses class-based OOP, instead of prototype-based OOP.

Class-based OOP is a historical mistake. If time could be turned back,
I think people would rather start with prototype-based OOP, instead.

Because of this mistake of using class-based OOP, you have incomplete
objects like modules and classes: you usually can't use them directly,
at least not as comfortably/powerfully. Therefore, you find yourself
dealing with names like SimpleXMLPRCServer.SimpleXMLRPCServer, without
being able to collapse instance/class/module into one single object
and one single name.

regards,

Hung Jung

 

[Josiah Carlson]

- Don't name the module and a class in it the same thing

To be frank, this problem is just one of the symptoms of a sickened
programming language. Python runs into this problem (like
SimpleXMLRPCServer.SimpleXMLRPCServer) because:

(a) By historical accident, Python modules differ significantly from
regular classes. (Modules are like singleton without constructor for
multiple instantiation, and without property getters/setters.)

(b) Python uses class-based OOP, instead of prototype-based OOP.

Class-based OOP is a historical mistake. If time could be turned back,
I think people would rather start with prototype-based OOP, instead.

Because of this mistake of using class-based OOP, you have incomplete
objects like modules and classes: you usually can't use them directly,
at least not as comfortably/powerfully. Therefore, you find yourself
dealing with names like SimpleXMLPRCServer.SimpleXMLRPCServer, without
being able to collapse instance/class/module into one single object
and one single name.

I think of modules as a namespace. Why? Because that is what they are.
They contain a space of names, some of which may be useful to you,
otherwise you wouldn't have imported the module. One really nifty thing
about modules is that you can rename them if you want to, or even rebind
names within them. The user is allowed to do anything they want, even
foolish things like "from module_blah import *".

In my opinion, modules are an explicit (and correct) solution to the
namespace problem. You don't like them? Fine, write your own custom
import command and stick it in site.py. If you're good, you can enable
it per-module (so that you can still use the standard library), and only
have to deal with the "mistake" every once and a while.

- Josiah

 

[Paul Rubin]

I think of modules as a namespace. Why? Because that is what they
are. They contain a space of names, some of which may be useful to
you, otherwise you wouldn't have imported the module.

Is a class instance not also a namespace? After all, it contains a
space of names, some of which you must have wanted or else you
wouldn't have instantiated it.

And yet, you can call a class instance if it has a __call__ operation
defined. I don't see why modules shouldn't be the same.

 

[Heather Coppersmith]

Is a class instance not also a namespace? After all, it
contains a space of names, some of which you must have wanted or
else you wouldn't have instantiated it.

And yet, you can call a class instance if it has a __call__
operation defined. I don't see why modules shouldn't be the
same.

In that light, IMO, __call__ methods are the kluge. This argument
seems to go back to (1) the old programs vs. data debate, and (2)
the meaning of "functions are first class objects." Note that
modules are neither functions nor objects.

If I really want to call (the top-level code in) a module, I can
always reload it:

Regards,
Heather

 

[Jeff Epler]

Note that modules are neither functions nor objects.

Of course modules are objects. Everything is an object, and most things
are even instances of "object".
1

Jeff

 

[Hung Jung Lu]

Is a class instance not also a namespace? After all, it contains a
space of names, some of which you must have wanted or else you
wouldn't have instantiated it.

And yet, you can call a class instance if it has a __call__ operation
defined. I don't see why modules shouldn't be the same.

Exactly. Another thing people have complained about is the lack of
getters/setters for module properties. For classes, you can have
properties. For modules, you can't.

Look, here is a partial list of the symptoms of the illness of using
class-and-module-based OOP:

(a) You need metaclasses. You have special syntax for usage of
metaclasses: you need to use the __metaclass__ name tag, in specific
places.

(b) Singletons become a "design pattern", since classes usually cannot
be used as instances directly.

(c) You have to create staticmethod and/or classmethod to make a class
functional.

(d) Modules can't have properties, nor are callable.

(e) You often need to create three objects: module, class, instance,
to do the job of one single object. Hence you often see usage of the
same name for all three of them, weird as it may seem.

(f) Module inheritance ('import' for containment, 'from ... import'
for inheritance) differs in syntax with class inheritance ('class
A(B):')

(g) Module functions and class methods become two distinct types of
objects.

In prototype-based OOP, you don't have any of these problems. You can
see that the usage of modules and classes introduces a whole lot of
repeated/redundant concepts and/or inconsistencies. That's the price
you pay. It's just an unfortunate historical development that lead us
to where we are today.

regards,

Hung Jung

 

[Fredrik Lundh]

In prototype-based OOP, you don't have any of these problems.

most of those problems don't apply if you're using Python in the "old way".

it's not like you *need* metaclasses, singleton patterns,
staticmethods/classmethods,
or class/module properties to write rock-solid, easily maintained,
efficient, and scalable
Python programs. Python works just fine without them.

</F>

 

[Hung Jung Lu]

I forgot to mention one more inconsistency/redundancy:

(h) In module-level functions, you need to use the 'global' statement
to access module-level attributes. In class (instance) methods, you
use the 'self' parameter to access instance attributes. (In class'
classmethods, you use 'cls'.) (Notice that there is a big difference
between saying "class method" and "classmethod".)

regards,

Hung Jung

 

[Hung Jung Lu]

One more, I promise I'll stop.

(i) For getting attributes by string names, for attributes of
instances you do getattr(self, 'name'), for module attributes you do
globals()['name']. I mention this because every once a month or so a
newbie would ask how to get module level attributes by name.

I'm not saying Python made the mistake alone: everyone else in
C++/Java, etc. made the same mistakes and a lot more. I'm saying that
in hindsight, prototype-based OOP would have saved a lot of troubles.
Why split things into 3 or 4 concepts (instance, class, module,
metaclass, etc.), when you could just have one (object)??? When you
have 3 or 4 different concepts, you are bound to have to duplicate
your effort. That's why people asked for staticmethod/classmethod in
Python (and they were done.) That's why metaclass came to be. That's
why people ask to make modules callable, that's why people ask about
properties for modules. It's tiring and complicated to implement
duplicated features: too much work, too many syntax conflicts. But if
one has started with prototype-based OOP, it only needs to be done
once, instead of 2 or 3 times.

Why do I think it's a good idea to compile a list of these "symptoms"?
Because many people are unfamiliar with prototype-based OOP, and they
think there is nothing (or not much) wrong with class-based OOP like
Python. When they see a list like what I am showing, they will
hopefully understand they have been coping with these
redundancies/inconsistencies for a long long time. Problems that
needed not to be there, in the first place. It's just a way to help
people think "out of the box."

regards,

Hung Jung

 

[Donn Cave]

Quoth "Fredrik Lundh" <[email protected]>:
| Hung Jung Lu wrote:
|
| > In prototype-based OOP, you don't have any of these problems.
|
| most of those problems don't apply if you're using Python in the "old way".
|
| it's not like you *need* metaclasses, singleton patterns,
| staticmethods/classmethods, or class/module properties to
| write rock-solid, easily maintained, efficient, and scalable
| Python programs. Python works just fine without them.

For that matter, I would rather expect software written without them
to be more solid and easily maintained.

It seems to me that it is easier to reason about the code you're
looking at, when it works within a simply structured system and
relies on thoughtful design for its elegance, instead of gimmicks.

I doubt it's just historical accident that OOP languages more or
less all have the class concept, or the type concept. This is an
organizing principle, helpful for human brains.

Donn Cave, (e-mail address removed)

 

[Aahz]

One more, I promise I'll stop.

(i) For getting attributes by string names, for attributes of
instances you do getattr(self, 'name'), for module attributes you do
globals()['name']. I mention this because every once a month or so a
newbie would ask how to get module level attributes by name.

That's not the only answer, and it's not the one I use:

import foo
getattr(foo, 'name')