handeling very large dictionaries

M

mclovin

Hello all,

I need to have a dictionary of about 8 gigs (well the data it is
processing is around 4gb). so naturally i am running into memory
errors.

So i looked around and found bsddb which acts like a dictionary object
only offloads the data from the RAM to the HDD, however that only
supports strings.

my dictionaries hold my own class objects

Is there something like it that is more flexible?
 
A

Aahz

I need to have a dictionary of about 8 gigs (well the data it is
processing is around 4gb). so naturally i am running into memory
errors.

So i looked around and found bsddb which acts like a dictionary object
only offloads the data from the RAM to the HDD, however that only
supports strings.

Look at the pickle module. Personally, I'd use SQLite instead of bsddb.
You might also look into a full-blown ORM such as SQLAlchemy.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top