Growing database?

[Jonathon McKitrick]

This might be a silly question, so I'm warning you ahead of time.

I have a persistence database that is saving a 'month' object out with its
data. A single month pickled is about 50k. My database of shelved objects
contains 4 months of data is over 3 megs. Does the db object journal, or do
some other recordkeeping that could explain the huge size difference between
pickling to a file and pickling to a 'shelve'?

jm

 

[David M. Cooke]

This might be a silly question, so I'm warning you ahead of time.

I have a persistence database that is saving a 'month' object out with its
data. A single month pickled is about 50k. My database of shelved objects
contains 4 months of data is over 3 megs. Does the db object journal, or do
some other recordkeeping that could explain the huge size difference between
pickling to a file and pickling to a 'shelve'?

Depends on what db module shelve uses for its backend. For instance,
gdbm won't shrink a database file unless you specifically tell it to,
so if you have a lot of deletions it will grow.

Easiest way to fix it is to copy all the objects from your shelve to a
new shelve.

 

[j_mckitrick]

Depends on what db module shelve uses for its backend. For instance,

gdbm won't shrink a database file unless you specifically tell it to,
so if you have a lot of deletions it will grow.

Easiest way to fix it is to copy all the objects from your shelve to a
new shelve.

That makes sense. I wasn't sure how deletions worked, but your idea
is a good one. I'm currently running on FreeBSD, so I am using the
bsddb backend.