Frustrating segfaults with Shelf

[Edward Grefenstette]

Dear Pythonistas,

Below is a simple script that reads relations from a generator
(they're just tuples of strings) and attempts to write them to a
database. It's simple as hell, and if I simply ignore the database and
have it print to stdout by replacing the line "depsDB[str(index)] =
rels" with "print rels" it works just fine. However every time I try
to run it just inexplicably segfaults. Help, anyone?

Best,
Edward

========================================

from WNvectorBuilder import relBuilder
import sys
import cPickle as pickle
import shelve

def main(argv):

## read filenames from command line arguments
depsFile = open(argv[1])
lemmaFile = open(argv[2])
depsDB = shelve.open(argv[3],"c")

## load lemma dictionary from lemma file
lemmaDict = pickle.load(lemmaFile)

## index of database entries
index = 0

## read relations from relation generator, write each relation to
database under new index
for rels in relBuilder(depsFile,lemmaDict):
index += 1
depsDB[str(index)] = rels
del rels

if __name__ == '__main__':
main(sys.argv)

========================================

 

[Emile van Sebille]

On 9/13/2010 10:05 AM Edward Grefenstette said...

Dear Pythonistas,

Below is a simple script that reads relations from a generator
(they're just tuples of strings) and attempts to write them to a
database. It's simple as hell,

Ecept of course that no one else can test it due to the dependencies.

and if I simply ignore the database and
have it print to stdout by replacing the line "depsDB[str(index)] =
rels" with "print rels" it works just fine. However every time I try
to run it just inexplicably segfaults. Help, anyone?

There are reported problems with the backend database in pre 2.3 python
versions. There are also known problems with shelves being accessed by
differing versions.

So, I'd start by updating as needed, creating a new repository, then
testing that. Once it works, migrate the data over if needed.

If it doesn't work, it'll be tough to help out if narrowing things down
to a specific reproducible test case is difficult. Describe the
debugging techniques you're trying with the results you're getting, and
we can help contribute with other things to try.

HTH,

Emile

 

[Edward Grefenstette]

Dear Emile,

Thank you for your response. Upgrading python and starting with a
blank database somehow did the trick!

Best,
Edward

On 9/13/2010 10:05 AM Edward Grefenstette said...

Dear Pythonistas,

Below is a simple script that reads relations from a generator
(they're just tuples of strings) and attempts to write them to a
database. It's simple as hell,

Ecept of course that no one else can test it due to the dependencies.

and if I simply ignore the database and
have it print to stdout by replacing the line "depsDB[str(index)] =
rels" with "print rels" it works just fine. However every time I try
to run it just inexplicably segfaults. Help, anyone?

There are reported problems with the backend database in pre 2.3 python
versions.  There are also known problems with shelves being accessed by
differing versions.

So, I'd start by updating as needed, creating a new repository, then
testing that.  Once it works, migrate the data over if needed.

If it doesn't work, it'll be tough to help out if narrowing things down
to a specific reproducible test case is difficult.  Describe the
debugging techniques you're trying with the results you're getting, and
we can help contribute with other things to try.

HTH,

Emile