Growing database?

Discussion in 'Python' started by Jonathon McKitrick, May 4, 2004.

  1. This might be a silly question, so I'm warning you ahead of time.

    I have a persistence database that is saving a 'month' object out with its
    data. A single month pickled is about 50k. My database of shelved objects
    contains 4 months of data is over 3 megs. Does the db object journal, or do
    some other recordkeeping that could explain the huge size difference between
    pickling to a file and pickling to a 'shelve'?

    jm
    --
    My other computer is your Windows box.
     
    Jonathon McKitrick, May 4, 2004
    #1
    1. Advertising

  2. At some point, Jonathon McKitrick <> wrote:

    > This might be a silly question, so I'm warning you ahead of time.
    >
    > I have a persistence database that is saving a 'month' object out with its
    > data. A single month pickled is about 50k. My database of shelved objects
    > contains 4 months of data is over 3 megs. Does the db object journal, or do
    > some other recordkeeping that could explain the huge size difference between
    > pickling to a file and pickling to a 'shelve'?


    Depends on what db module shelve uses for its backend. For instance,
    gdbm won't shrink a database file unless you specifically tell it to,
    so if you have a lot of deletions it will grow.

    Easiest way to fix it is to copy all the objects from your shelve to a
    new shelve.

    --
    |>|\/|<
    /--------------------------------------------------------------------------\
    |David M. Cooke
    |cookedm(at)physics(dot)mcmaster(dot)ca
     
    David M. Cooke, May 4, 2004
    #2
    1. Advertising

  3. Jonathon McKitrick

    j_mckitrick Guest

    > Depends on what db module shelve uses for its backend. For instance,
    > gdbm won't shrink a database file unless you specifically tell it to,
    > so if you have a lot of deletions it will grow.
    >
    > Easiest way to fix it is to copy all the objects from your shelve to a
    > new shelve.


    That makes sense. I wasn't sure how deletions worked, but your idea
    is a good one. I'm currently running on FreeBSD, so I am using the
    bsddb backend.
     
    j_mckitrick, May 4, 2004
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mahesh Prasad
    Replies:
    1
    Views:
    709
    Tom Wells
    Feb 22, 2004
  2. seabass

    reading a growing log file

    seabass, Oct 5, 2003, in forum: Perl
    Replies:
    1
    Views:
    484
    seabass
    Oct 5, 2003
  3. Clement
    Replies:
    7
    Views:
    1,622
    Kevin Spencer
    Jul 3, 2003
  4. Hoon
    Replies:
    1
    Views:
    488
    John Timney \(Microsoft MVP\)
    Jul 3, 2003
  5. V. Jenks
    Replies:
    5
    Views:
    367
Loading...

Share This Page