shelve crashing under Win ME

[gluckj]

Hi,

I'm not a Win ME fan myself (I'm a Mac user), but I'm here in Thailand
developing software for special-needs kids, and the test PC in my home
office is a Win ME machine (sigh). So when I ported my Python program
today to the PC, it quickly crashed. Everything seems to work except
for shelve. Using the latest version of Python (2.5.1), when I do the
following in IDLE on the Win ME machine here's the result:

Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC
v.1310 32 bit (Intel)] on win32
IDLE 1.2.1
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
f=shelve.open("foo")
File "C:\PYTHON25\lib\shelve.py", line 225, in open
return DbfilenameShelf(filename, flag, protocol,
writeback)
File "C:\PYTHON25\lib\shelve.py", line 209, in
__init__
Shelf.__init__(self, anydbm.open(filename, flag),
protocol, writeback)
File "C:\PYTHON25\lib\anydbm.py", line 83, in open
return mod.open(file, flag, mode)
File "C:\PYTHON25\lib\dbhash.py", line 16, in open
return bsddb.hashopen(file, flag, mode)
File "C:\PYTHON25\lib\bsddb\__init__.py", line 306,
in hashopen
d.open(file, db.DB_HASH, flags, mode)
DBError: (5, 'Input/output error')
For those who know shelve, this should actually run fine (if the file
"foo" does not exist it should create it). Printing the value of "f"
should show an empty dictionary {}.

It does run fine on my iBook (under Python 2.4.4) and on my iBook's
Virtual PC running Win XP & Python 2.5.0. I do not think this is a
Python 2.5.1 problem, because my first attempt to run my program on
the Win ME machine was with a version of the program I ported using
Py2exe on the VPC under Python 2.5.0, and it crashed the same way when
run on the Win ME machine (I subsequently installed Python on the Win
ME PC to try to get to the root of the problem.)

Help! How do I get Win ME (or at least the misbehaving Win ME machine
in my office) to run shelve? (or more specifically run bsddb's
hashopen?)

Or should I trash shelve entirely and rewrite all my code to use a
simpler, homemade database scheme?

Thanks for any advice!

Warmly,
Joel
(e-mail address removed)

 

[jim-on-linux]

Hi,

I'm not a Win ME fan myself (I'm a Mac user),
but I'm here in Thailand developing software
for special-needs kids, and the test PC in my
home office is a Win ME machine (sigh). So when
I ported my Python program today to the PC, it
quickly crashed. Everything seems to work
except for shelve. Using the latest version of
Python (2.5.1), when I do the following in IDLE
on the Win ME machine here's the result:

Python 2.5.1 (r251:54863, Apr 18 2007,
08:51:08) [MSC v.1310 32 bit (Intel)] on win32
IDLE 1.2.1

Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
f=shelve.open("foo")
File "C:\PYTHON25\lib\shelve.py", line 225,
in open return DbfilenameShelf(filename, flag,
protocol, writeback)
File "C:\PYTHON25\lib\shelve.py", line 209,
in __init__
Shelf.__init__(self, anydbm.open(filename,
flag), protocol, writeback)
File "C:\PYTHON25\lib\anydbm.py", line 83, in
open return mod.open(file, flag, mode)
File "C:\PYTHON25\lib\dbhash.py", line 16, in
open return bsddb.hashopen(file, flag, mode)
File "C:\PYTHON25\lib\bsddb\__init__.py", line
306, in hashopen
d.open(file, db.DB_HASH, flags, mode)
DBError: (5, 'Input/output error')


For those who know shelve, this should actually
run fine (if the file "foo" does not exist it
should create it). Printing the value of "f"
should show an empty dictionary {}.

It does run fine on my iBook (under Python
2.4.4) and on my iBook's Virtual PC running Win
XP & Python 2.5.0. I do not think this is a
Python 2.5.1 problem, because my first attempt
to run my program on the Win ME machine was
with a version of the program I ported using
Py2exe on the VPC under Python 2.5.0, and it
crashed the same way when run on the Win ME
machine (I subsequently installed Python on the
Win ME PC to try to get to the root of the
problem.)

Help! How do I get Win ME (or at least the
misbehaving Win ME machine in my office) to run
shelve? (or more specifically run bsddb's
hashopen?)

Or should I trash shelve entirely and rewrite
all my code to use a simpler, homemade database
scheme?

Thanks for any advice!

Warmly,
Joel
(e-mail address removed)

what version of py2exe are you using?

I had a similar problem with an old version of
py2exe but it it is fixed now. If my memory is
correct I had to import dbhash into my setup for
py2exe back then to fix it..

Your code for shelve is correct, the problem is
someplace else.


jim-on-linux
http:\\www.inqvista.com

 

[jim-on-linux]

Hi,

I'm not a Win ME fan myself (I'm a Mac user),
but I'm here in Thailand developing software
for special-needs kids, and the test PC in my
home office is a Win ME machine (sigh). So
when I ported my Python program today to the
PC, it quickly crashed. Everything seems to
work except for shelve. Using the latest
version of Python (2.5.1), when I do the
following in IDLE on the Win ME machine
here's the result:

Python 2.5.1 (r251:54863, Apr 18 2007,
08:51:08) [MSC v.1310 32 bit (Intel)] on
win32 IDLE 1.2.1

import shelve
f=shelve.open("foo")

Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
f=shelve.open("foo")
File "C:\PYTHON25\lib\shelve.py", line 225,
in open return DbfilenameShelf(filename,
flag, protocol, writeback)
File "C:\PYTHON25\lib\shelve.py", line 209,
in __init__
Shelf.__init__(self,
anydbm.open(filename, flag), protocol,
writeback)
File "C:\PYTHON25\lib\anydbm.py", line 83,
in open return mod.open(file, flag, mode)
File "C:\PYTHON25\lib\dbhash.py", line 16, in
open return bsddb.hashopen(file, flag, mode)
File "C:\PYTHON25\lib\bsddb\__init__.py",
line 306, in hashopen
d.open(file, db.DB_HASH, flags, mode)
DBError: (5, 'Input/output error')


For those who know shelve, this should
actually run fine (if the file "foo" does not
exist it should create it). Printing the
value of "f" should show an empty dictionary
{}.

It does run fine on my iBook (under Python
2.4.4) and on my iBook's Virtual PC running
Win XP & Python 2.5.0. I do not think this is
a Python 2.5.1 problem, because my first
attempt to run my program on the Win ME
machine was with a version of the program I
ported using Py2exe on the VPC under Python
2.5.0, and it crashed the same way when run
on the Win ME machine (I subsequently
installed Python on the Win ME PC to try to
get to the root of the problem.)

Help! How do I get Win ME (or at least the
misbehaving Win ME machine in my office) to
run shelve? (or more specifically run bsddb's
hashopen?)

Or should I trash shelve entirely and rewrite
all my code to use a simpler, homemade
database scheme?

Thanks for any advice!

Warmly,
Joel
(e-mail address removed)

what version of py2exe are you using?

I had a similar problem with an old version of
py2exe but it it is fixed now. If my memory is
correct I had to import dbhash into my setup
for py2exe back then to fix it..

Your code for shelve is correct, the problem is
someplace else.


jim-on-linux
http:\\www.inqvista.com

Correction:

I had to import dbhash into my program to make
things work.

jim-on-linux
http:\\www.inqvista.com

 

[gluckj]

Hi Jim,

Thanks for your messages -- however, py2exe is NOT the issue -- as you
can see with my example above, shelve is crashing even just by calling
it from IDLE (latest PC version of IDLE & Python), running on a Win ME
machine.

Anyone else out there having access to a Win ME PC willing to try it?
Thanks for checking!

In the meantime, I may replace shelve with a simpler workaround using
cPickle & basic flat file operations.

-- Joel
p.s. -- Jim, I too had to import dbhash in my program to make it work
using py2exe, so you are right on, there.

 

[gluckj]

OK, here's my workaround for shelve -- it's primitive and obviously
much slower (it saves the entire dictionary each time), and you have
to manually save -- BUT: it works...even on the Win ME machine. And
it's possibly more universally portable in the long run than
shelve...Also, once you open the dictionary, it is just as fast
(perhaps faster?) than shelve for reading the contents (just not for
saving...)
--Joel

================

#! /usr/local/bin/python
"""
myshelve.py -- a simpler (safer though slower) approach to shelve
"""

#Import Modules
import os, cPickle

def msopen(path):
""" open file containing pickled dict & return dict; if no such file,
create it & return empty dict """
if os.access(path, os.R_OK):
f=open(path, "r")
mydict=cPickle.load(f)
f.close()
return(mydict)
else:
f=open(path, "w")
cPickle.dump({}, f)
f.close()
return({})

def mssave(mydict,path):
""" pickle and save the dict to the file, overwriting previous
contents """
f=open(path, "w")
cPickle.dump(mydict, f)
f.close()

def mstest():
f1="_myshelvetest_.txt"
print "Testing myshelve..."
print "1) New file:",f1
if os.access(f1, os.F_OK): os.remove(f1)
d1=msopen(f1)
print "Contents of new file:",d1
d2=msopen(f1)
print "Contents on re-read:",d2
print "2) Write/Read/Overwrite/Read..."
d1={1:"hello",2:"there",3:"folks"}
print "Test dict:",d1
mssave(d1,f1)
d2=msopen(f1)
print "Loaded:",d2
print "Equal?",d1==d2
d1={1:"goodbye",2:"now",3:"folks"}
print "Test dict:",d1
mssave(d1,f1)
d2=msopen(f1)
print "Loaded:",d2
print "Equal?",d1==d2
print "** DONE **"
os.remove(f1)

if __name__=="__main__": mstest()

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

OK, here's my workaround for shelve -- it's primitive and obviously
much slower (it saves the entire dictionary each time), and you have
to manually save -- BUT: it works...even on the Win ME machine. And
it's possibly more universally portable in the long run than
shelve...Also, once you open the dictionary, it is just as fast
(perhaps faster?) than shelve for reading the contents (just not for
saving...)

Notice that you don't need to use bsddb for shelve. Instead, you might
use dumbdbm instead, passing the dbm directly to shelve.Shelf. It's
probably slower than bsddb, however, updating a single key should be
faster than pickling an entire dictionary.

Regards,
Martin