bsddb.dbshelve problem unpickling objects from another directory

[BG]

I am experimenting with dbshelve in Python 2.3.3 on Win2k. The problem is
that objects put into the dbshelve by a script in one directory cannot be
unpickled by script running in another directory. Why?

I am writing my own collection called testcollection to hold testitems. The
testcollection uses dbshelve to store its items to disk. I want design a
package called testmodules.

So I have the following:

a directory called testmodules that holds the following files:
__init__.py
testcollection.py
testitem.py

I have a file called test.py:
---start code---
from testcollection import testcollection
from testitem import testitem

tc = testcollection()

t1 = testitem("car")
t2 = testitem("sportscar")

tc["1"]=t1
tc["2"]=t2

print "iteritems()"
print "\n".join(["%s, %s" % (k, v.name) for k, v in tc.iteritems()])

print "itervalues()"
print "\n".join(["%s" % (v.name, ) for v in tc.itervalues()])

print "values()"
print "\n".join(["%s" % (v.name, ) for v in tc.values()])

print "info"
print tc.dbinfo()
--- end code ---
If I execute this script from within the directory testmodules it works
perfectly. I have got another script test.py that I execute outside the
directory test modules. This script looks like this:

---start code---
from testmodules.testcollection import testcollection
from testmodules.testitem import testitem

tc = testcollection()

t3 = testitem("SUV")

tc["3"]=t3

print "iteritems()"
print "\n".join(["%s, %s" % (k, v.name) for k, v in tc.iteritems()])

print "itervalues()"
print "\n".join(["%s" % (v.name, ) for v in tc.itervalues()])

print "values()"
print "\n".join(["%s" % (v.name, ) for v in tc.values()])

print "info"
print tc.dbinfo()
--- end code ---

When I run this script (after I ran testmodules/test.py) I will get the
following error message:
iteritems()
Traceback (most recent call last):
File "C:\DataCOnly2\Python\worldObjects\test\test.py", line 11, in ?
print "\n".join(["%s, %s" % (k, v.name) for k, v in tc.iteritems()])
File "C:\Python23\lib\UserDict.py", line 99, in iteritems
yield (k, self[k])
File "C:\Python23\lib\bsddb\dbshelve.py", line 107, in __getitem__
return cPickle.loads(data)
ImportError: No module named testitem

When I clear all database files and first run test.py and than
testmodules/test.py I will get the same error message.

So objects made by a the script in one directory cannot be unpickled by a
script in another directory. Why? Anyone?

--- code of testcollection ---
from bsddb import dbshelve, db
import sys, os, inspect, shelve

class testcollection:
def __init__(self):
#open a multi reader, one writer database
dir = os.path.dirname(inspect.getfile(testcollection)) + '/data/'
if not os.path.exists(dir):
os.mkdir(dir)
self._envdir = dir
self._dbfilename = 'testcollection.dbshelve'
self._env = db.DBEnv()
self._env.open(self._envdir,
db.DB_CREATE|db.DB_INIT_CDB|db.DB_INIT_MPOOL)
self.items = dbshelve.open(self._dbfilename, dbenv=self._env)

def __setitem__(self, key, value):
self.items[key]=value

def __getitem__(self, key):
return self.items[key]

def iteritems(self):
return self.items.iteritems()

def itervalues(self):
return self.items.itervalues()

def values(self):
return self.items.values()

def dbinfo(self):
return self._env.lock_stat()
---end testcollection---

---start of testitem---
class testitem:
def __init__(self, name="_none_"):
self.name = name
---end testitem---

---start __init__.py---
__all__ = ["testcollection", "testitem"]
---end __init__.py---

 

[Josiah Carlson]

I am experimenting with dbshelve in Python 2.3.3 on Win2k. The problem is
that objects put into the dbshelve by a script in one directory cannot be
unpickled by script running in another directory. Why?

I am writing my own collection called testcollection to hold testitems. The
testcollection uses dbshelve to store its items to disk. I want design a
package called testmodules.

[snip]

Pickled objects reference the module and the class from which they were
created from.
When running testmodules/test.py, testitem lives in in the module testitem.
When running test.py, testitem lives in the module testmodules.testitem.


- Josiah

 

[BG]

Josiah,

Thank you for your quick answer. But, this very much limits the way
persistent objects can be used. Objects can apparently only be used in the
context they were created. There is a logic to this, but it is also strange
to me. Is there a way that upickled objects can also be used outside there
original context?

Berry.

I am experimenting with dbshelve in Python 2.3.3 on Win2k. The problem is
that objects put into the dbshelve by a script in one directory cannot be
unpickled by script running in another directory. Why?

I am writing my own collection called testcollection to hold testitems. The
testcollection uses dbshelve to store its items to disk. I want design a
package called testmodules.

[snip]

Pickled objects reference the module and the class from which they were
created from.
When running testmodules/test.py, testitem lives in in the module testitem.
When running test.py, testitem lives in the module testmodules.testitem.


- Josiah

 

[Josiah Carlson]

Thank you for your quick answer. But, this very much limits the way

persistent objects can be used. Objects can apparently only be used in the
context they were created. There is a logic to this, but it is also strange
to me. Is there a way that upickled objects can also be used outside there
original context?

I don't know. If I were put into a situation like that, I'd probably
just go and do this where necessary:

import sys
sys.path.append('<path to database module>')

import testmodule
#etcetera


Sure, it mangles paths, but then everything has the same context and
will work.

- Josiah

 

[BG]

Josiah,

I tried what you suggested and indeed it works perfectly! Thank you very
much for your help!

Berry