Creating classes and objects more than once?

D

Diez B. Roggisch

Viktor said:
Here is the situation:

$ ls
test
$ cd test
$ ls
__init__.py data.py
$ cat __init__.py

$ cat data.py
DATA = {}

$ cd ..
$ python
import os
from test.data import DATA
DATA['something'] = 33
os.chdir('test')
from data import DATA as NEW_DATA
DATA {'something': 33}
NEW_DATA
{}


Is this a bug?

No. You create an alias NEW_DATA in your local scope, but that alias still
points to the same object, a dictionary.

Diez
 
A

Arnaud Delobelle

Viktor Kerkez said:
Here is the situation:

$ ls
test
$ cd test
$ ls
__init__.py data.py
$ cat __init__.py

$ cat data.py
DATA = {}

$ cd ..
$ python
import os
from test.data import DATA
DATA['something'] = 33
os.chdir('test')
from data import DATA as NEW_DATA
DATA {'something': 33}
NEW_DATA
{}


Is this a bug?

No. It's one of the compelling features of Python modules.
 
V

Viktor Kerkez

But this means that there is no way to create a safe Singleton in
python, because the classes are also created twice.

This is the problem that I encountered. I created a complex
implementation of a Singleton pattern using metaclasses because I
needed the __init__ method to be called just once and I wanted to use
inheritance from the Singleton class. Here is the code:




class SingletonMetaclass(type):

'''Singleton Metaclass



This metaclass is used for creating singletons.

It changes the class __new__ method to maintain only one instance
of the

class, and tweaks the __init__ method to be executed only once
(when the

first instance of the class is created.



Usage:

... """Real singleton class.

...

... You have to set the __metaclass__ attribute to
SingletonMetaclass,

... and define the __init__ function. Everythin else will be
done by

... metaclass.

... """

... __metaclass__ = SingletonMetaclass

... def __init__(self, data):

... print 'Initializing'

... self.data = data

...
actually happen

Initializing

'First initialization'

'First initialization'

'''

def __new__(cls, name, bases, dct):

dct['__new__'] = SingletonMetaclass._dekorate_new()

dct['get_instance'] = SingletonMetaclass._decorate_get_instance
()

try:

dct['__init__'] = SingletonMetaclass._dekorate_init(dct
['__init__'])

except KeyError:

init_functions = [getattr(base, '__init__') for base in
bases if hasattr(base, '__init__')]

if init_functions:

dct['__init__'] = SingletonMetaclass._dekorate_init
(init_functions[0])

else:

raise Exception('Don\'t use SingletonMetaclass, use
inheritance from Singleton class!')

return type.__new__(cls, name, bases, dct)



@staticmethod

def _dekorate_init(function):

def wrapper(self, *args, **kwargs):

if not hasattr(self, '_singleton_initialized'):

function(self, *args, **kwargs)

setattr(self, '_singleton_initialized', True)

return wrapper



@staticmethod

def _dekorate_new():

def wrapper(cls, *p, **k):

if not hasattr(cls, '_instance'):

cls._instance = object.__new__(cls)

return cls._instance

return wrapper



@staticmethod

def _decorate_get_instance():

@classmethod

def wrapper(cls):

if not hasattr(cls, '_instance'):

return None

return cls._instance

return wrapper





class Singleton(object):

'''Singleton class



Inherit from this class if you want to have a singleton class.

Never use SingletonMetaclass!



Usage:

... """Singleton without __init__ method"""

... pass

...

... """Singleton with __init__ method"""

... def __init__(self, data):

... print 'Initializing'

... self.data = data

...
actually happen

Initializing

'First initialization'

'''

__metaclass__ = SingletonMetaclass

def __init__(self):

pass





if __name__ == '__main__':

import doctest

doctest.testmod()




The problem started when the class gets imported in two different
ways, and the class gets created twice!??!

Do You have any suggestions how to solve this problem.
 
C

Carl Banks

Here is the situation:

$ ls
test
$ cd test
$ ls
__init__.py data.py
$ cat __init__.py

$ cat data.py
DATA = {}

$ cd ..
$ python>>> import os
from test.data import DATA
DATA['something'] = 33
os.chdir('test')
from data import DATA as NEW_DATA
DATA {'something': 33}
NEW_DATA

{}

Is this a bug?

No, because you've actually imported two different modules (even
though it's the same file on disk, for the second inport statement
Python imports the file again because it was done from a different
absolute path, that is, the module "data" is always different from the
module "test.data" even if they refer to the same file).

However, I'm not so sure the effect of os.chdir() on the import path
is a good idea. It would seem that among people who use os.chdir() in
their programs, some will want the import path to change with it, some
will not. You can't please everyone, so I would suggest that we
should choose in favor of limiting context sensitivity. I like to
think that "import abc" always does the same thing regardless of any
seemingly unrelated state changes of my program, especially since, as
the OP pointed out, import is used as a means to ensure singleness.
Thus, if I were designing the language, I would have sys.path[0] be
the current working directory at program start.


To the OP: My first suggestion is to consider not using os.chdir() in
your programs, and instead construct pathnames with the directory
included. If you do use os.chdir(), then early in your script script,
add a line such as "sys.path[0] = os.getcwd()". Then, no matter where
you are, always import the file relative to the starting directory.
So always use "from test.data import DATA", even after you os.chdir().


Carl Banks
 
V

Viktor Kerkez

However, I'm not so sure the effect of os.chdir() on the import path
is a good idea.

I'm not actually using os.chidir(), I just used it here to create a
clearer example.

Here is the simplest representation of the problem:

http://www.ninjashare.com/904415


$ cd singleton_test
$ export PYTHONPATH=.
$ python application/main_script.py
Creating class Singleton
Creating class MySingleton
Creating class Singleton
Creating class MySingleton
$
 
C

Carl Banks

Carl Banks said:
I like to think that "import abc" always does the same thing
regardless of any seemingly unrelated state changes of my program,
especially since, as the OP pointed out, import is used as a means
to ensure singleness. Thus, if I were designing the language, I
would have sys.path[0] be the current working directory at program
start.

This is resolved in the Python 2.x series by implementing PEP 328
<URL:http://www.python.org/dev/peps/pep-0328/>, such that the search
path for ‘import’ does *not* contain the current directory, and
requiring relative-to-the-current-directory imports to be explicitly
requested.

PEP 328 doesn't say anything about that. Using Python 2.5, which PEP
328 claims implements this change, I see the same behavior that Victor
Kerkez sees. My sys.path[0] is still the empty string, meaning that
Python does start its import search from the current directory.

Perhaps you are confusing "current directory" with "directory that the
current module file is in", which PEP 328 does address?

(It's ok, the PEP itself used "current directory" in this way.)


Carl Banks
 
C

Carl Banks

I'm not actually using os.chidir(), I just used it here to create a
clearer example.

Here is the simplest representation of the problem:

http://www.ninjashare.com/904415

$ cd singleton_test
$ export PYTHONPATH=.
$ python application/main_script.py
Creating class Singleton
Creating class MySingleton
Creating class Singleton
Creating class MySingleton
$

Ah ha. That's a slightly different issue. Here's the problem: when
you type "python application/main_script.py", Python doens't call the
module it runs "application,main_script" like you'd expect. Instead
it calls the module "__main__". Thus, when you later import
"application.main_script", Python doesn't see any already-loaded
modules called application.main_script, so it imports the file again
(this time calling the module "application.main_script").

Try it this way:

python -c 'import application.main_script'

In this example, Python the string passed to "-c" is considered the
__main__ module, so when you import application.main_script it calls
the module it imports "application.main_script", so you will get the
behavior you expect.

The thing you have to remember is, "Never try to reimport the main
script; it just won't work." If you find yourself doing that, create
a tiny script whose sole job is to import the real main script (and
maybe set up sys.path), and run that. (Also, some people consider
reciprocal imports to be bad style. I'm not one of them per se but I
find that it's worth the effort to avoid them if it's easy to do so.)


Carl Banks
 
C

Carl Banks

PEP 328 doesn't say anything about that. Using Python 2.5, which PEP
328 claims implements this change, I see the same behavior that
Victor Kerkez sees. My sys.path[0] is still the empty string,
meaning that Python does start its import search from the current
directory.

In Python 2.5, the PEP is implemented to the point that the absolute
import behaviour is available by adding a ‘from __future__ import
absolute_import’. Without that, yes, you will see the same behaviour
as reported by Victor.

I see the same behavior as Viktor reports regardless of whether I
disable relative imports or not.

Absolute versus relative imports don't have anything to do with the
issue here. PEP 328 concerns itself with imports relative to the
executing module in package space. It has nothing to do with imports
relative to the current directory in filename space.

If the string "" is in sys.path, then imports relative to the current
directory (including changes to the current directory made by
os.chdir) will work, even if relative imports are disabled.

Try this test with Python 2.5. (sh command lines to set up the
environment shown.)

$ mkdir foo
$ mkdir bar
$ touch bar/baz.py
$ cat > foo/script.py <<EOF

from __future__ import absolute_import

import sys, os
sys.path.insert(0,"")
os.chdir('../bar')
import baz

EOF
$ cd foo
$ python2.5 script

If your claim that importing from the current directory is disabled
when "from __future__ import absolute_import" is issued were true,
then the import of baz module would raise an exception. It doesn't
for me.

On further investigation, I found was that sys.path[0] is "" whenever
it uses an interactive interpreter or the -c switch is used, but is
the directory of the script file whenever python is given a script
file. Which means, by default, importing from the current directory
is disabled for scripts, but enabled for interactive work and the -c
switch. This is regardless of whether relative imports are disabled
or not.


Carl Banks
 
C

Carl Banks

On Nov 28, 11:51 pm, Carl Banks
Absolute versus relative imports don't have anything to do with the
issue here.  PEP 328 concerns itself with imports relative to the
executing module in package space.  It has nothing to do with imports
relative to the current directory in filename space.


I thought of another way to put this to help explain things. Suppose
you have two files in your current directory, a Python script file
(app.py) which imports a Python module (work.py).

Near the top of the file app.py, there is line like this:

import work

What happens beneath the covers when this statement is excuted (and
from __future__ import absolute_import hasn't been run)? Many people
seem to think that the Python interpreter first considers whether this
is a relative import and starts by looking for "sister" modules in the
same "package" (i.e., directory). Python would thus see the file
work.py in the same "package" and complete this as a relative import.

Thus, they reason, if one were to add "from __future__ import
absolute_import" to the top of app.py, the import would no longer work
because implicit relative imports have been disabled. One would have
to use "from . import work" instead.

Well, this is not how it happens. No top level module, including
__main__, is in a package. Thus, when Python sees "import work" in a
top-level module, it doesn't consider it to be a relative import, even
when implicit relative imports have not been disabled.

The point of this is, disabling implicit relative imports has no
effect on imports from top-level modules since they were never
relative imports anyway. Python is able to find the "sisters" of top-
level modules and scripts not because it is doing a relative import,
but because those "sister" modules are in one of the directories
listed in sys.path.

In particular, if "" is listed in sys.path, the current working
directory--even when modified by os.chdir--will be searched for
"sisters" of top-level modules.


Carl Banks
 
G

Gabriel Genellina

On Nov 28, 11:51 pm, Carl Banks


I thought of another way to put this to help explain things. Suppose
you have two files in your current directory, a Python script file
(app.py) which imports a Python module (work.py).

Near the top of the file app.py, there is line like this:

import work

What happens beneath the covers when this statement is excuted (and
from __future__ import absolute_import hasn't been run)? Many people
seem to think that the Python interpreter first considers whether this
is a relative import and starts by looking for "sister" modules in the
same "package" (i.e., directory). Python would thus see the file
work.py in the same "package" and complete this as a relative import.

Thus, they reason, if one were to add "from __future__ import
absolute_import" to the top of app.py, the import would no longer work
because implicit relative imports have been disabled. One would have
to use "from . import work" instead.

Well, this is not how it happens. No top level module, including
__main__, is in a package. Thus, when Python sees "import work" in a
top-level module, it doesn't consider it to be a relative import, even
when implicit relative imports have not been disabled.

The point of this is, disabling implicit relative imports has no
effect on imports from top-level modules since they were never
relative imports anyway. Python is able to find the "sisters" of top-
level modules and scripts not because it is doing a relative import,
but because those "sister" modules are in one of the directories
listed in sys.path.

In particular, if "" is listed in sys.path, the current working
directory--even when modified by os.chdir--will be searched for
"sisters" of top-level modules.

While all the above explanation is true for scripts executed using "python
xxx.py", it's not the same when you execute a module using runpy.py, that
is, using "python -m xxx"
If you execute a module inside a package in that way, Python recognizes
the fact it's inside a package, and honors relative imports (starting from
version 2.6, when PEP366 was implemented).
So the statement "from __future__ import absolute_import" *does* change
how imports are handled on the top module.
A simple test showing the difference:

C:\temp\test366>dir /b
b.py
pkga

C:\temp\test366>type b.py
msg='this is the external b module'

C:\temp\test366>cd pkga

C:\temp\test366\pkga>dir /b
abs.py
b.py
rel.py
__init__.py

C:\temp\test366\pkga>type __init__.py

C:\temp\test366\pkga>type b.py
msg='this is the b module inside package pkga'

C:\temp\test366\pkga>type rel.py
print "this script uses 'traditional' import semantics"
from b import msg
print msg
print "using a bare import:"
import b
print b.msg

C:\temp\test366\pkga>type abs.py
from __future__ import absolute_import
print "this script sets absolute_import"
print "using an absolute import:"
from b import msg
print msg
print "using a bare import:"
import b
print b.msg
print "using an explicit relative import:"
from .b import msg
print msg

C:\temp\test366\pkga>cd ..

C:\temp\test366>python -m pkga.rel
this script uses 'traditional' import semantics
this is the b module inside package pkga
using a bare import:
this is the b module inside package pkga

C:\temp\test366>python -m pkga.abs
this script sets absolute_import
using an absolute import:
this is the external b module
using a bare import:
this is the external b module
using an explicit relative import:
this is the b module inside package pkga
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,768
Messages
2,569,574
Members
45,051
Latest member
CarleyMcCr

Latest Threads

Top