shelve.open generates (22, 'Invalid argument') Os X 10.5 with Python2.5

[seth]

There is something you could possibly help me with.

We have a code that creates a simple Python shelve database. We are
able to serialize objects and store them in the dbm file. This seem to
work the same on Windows XP Python 2.5, Ubuntu 9.1 with Python 2.6,
but on Os X 10.5 with Python 2.5 the database filename is changed by
the operating system by attaching the .db extension to it. Does an one
know why is that?

What is worse, at the time of reading it we get the error below the
extracted code, on Os X 10.5:

if os.path.exists( tarname )and tarfile.is_tarfile( tarname ):
try:
datastore_tarfile=tarfile.open( tarname, 'r:gz' )
print "Successfully opened the tarball: %s"%tarname
members=datastore_tarfile.getnames()
for dbmFile in members:
datastore_tarfile.extract( dbmFile )
print "Extracting Realizations File: %s"%dbmFile

realizations=shelve.open( dbmFile, 'c', 2, writeback =
False )

Successfully opened the tarball: datastore.tar.gz
Extracting Realizations File: realizations.dbm.db
<class 'bsddb.error'>
(22, 'Invalid argument')

This does not happen on Windows XP with Python 2.5.
Does anyone know what is happening on Os X?
Any comments and suggestions will be greatly appreciated. Thank you
very much in advance.

 

[Aahz]

We have a code that creates a simple Python shelve database. We are
able to serialize objects and store them in the dbm file. This seem to
work the same on Windows XP Python 2.5, Ubuntu 9.1 with Python 2.6,
but on Os X 10.5 with Python 2.5 the database filename is changed by
the operating system by attaching the .db extension to it. Does an one
know why is that?

Nope -- any reason you can't change the filename?

 

[Ned Deily]

We have a code that creates a simple Python shelve database. We are
able to serialize objects and store them in the dbm file. This seem to
work the same on Windows XP Python 2.5, Ubuntu 9.1 with Python 2.6,
but on Os X 10.5 with Python 2.5 the database filename is changed by
the operating system by attaching the .db extension to it. Does an one
know why is that?

It's a documented "feature": it depends on the underlying database
package used by shelve.

http://docs.python.org/library/shelve.html

"As a side-effect, an extension may be added to the filename ..."

 

[seth]

Nope -- any reason you can't change the filename?
--

Os X 10.5 did not recognized the dbm extension.

But, I have been able to fix the problem (hope it helps somebody):
At http://docs.python.org/library/shelve.html it says:

"shelve.open(filename[, flag='c'[, protocol=None[,
writeback=False]]])¶

Open a persistent dictionary. The filename specified is the base
filename for the underlying database. As a side-effect, an extension
may be added to the filename and more than one file may be created. By
default, the underlying database file is opened for reading and
writing. "

Then, I went ahead and try to find out which type of db file was being
created:

print whichdb.whichdb(dbmFile)#prints bsddb185
kpfmac:xLPR kpf$ python
Python 2.5.1 (r251:54863, Feb 9 2009, 18:49:36)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.<bsddb.bsddb object at 0x12260>

I found that the dbm extension is generated by bsddb and bsddb3 so I
went ahead and installed it: sudo easy_install bsddb3.
Finally, in our code, I just went ahead and instead of using
shelve.open, I am calling shelve.BsdDbShelf(bsddb3.btopen(filename,
'c'))

#realizations=shelve.open( filename, 'c', 2, writeback =
False )
#hk 021010: need to use BsdDbShelf to ensure the usage of
bsddb3 on OsX 10.5
#shelve.open creates a bsddb185 file on OsX 10.5
#but shelve.open on OsX reads with bsddb3, who knows why...
#as a result the shelve.open throws
#<class 'bsddb.error'> (22, 'Invalid argument')
dbshelf=shelve.BsdDbShelf(bsddb3.btopen(filename, 'c'))
realizations=dbshelf

And that fixed on Os X 10.5. It also works fine on Windows, by
ensuring the proper import:
try:
import bsddb3
except:
import bsddb as bsddb3