B
barbaros
Hello everybody,
I need some advice on the following problem.
I want to write a program (in C or C++) which will
build a huge dictionary (hash array, if you prefer).
The keys are strings, the values are floating point numbers.
The dictionary will soon become larger than the RAM memory
(I expect more than ten gygabytes). So I need to save it
(or parts of it) to the disk, while still being able to
read from and write to it (efficiently). A detail which
may be useful: no entries will ever be deleted. That is,
changes to the dictionary consist only in appending an
entry or modifiyng an existing value (no key will disappear).
One solution would be to have a very large swap partition,
under linux. But this is not a good idea in my case,
since I need to keep the dictionary for future analysis.
Except if I save it in the end to another partition ...
why not, actually ?
Another solution would be to use an external database
like SQLite or Metakit.
Which solution would you recommend ?
Speed of execution is very important.
Thanks in advance for any hint !
Yours, Cristian Barbarosie
http://cmaf.fc.ul.pt/~barbaros
I need some advice on the following problem.
I want to write a program (in C or C++) which will
build a huge dictionary (hash array, if you prefer).
The keys are strings, the values are floating point numbers.
The dictionary will soon become larger than the RAM memory
(I expect more than ten gygabytes). So I need to save it
(or parts of it) to the disk, while still being able to
read from and write to it (efficiently). A detail which
may be useful: no entries will ever be deleted. That is,
changes to the dictionary consist only in appending an
entry or modifiyng an existing value (no key will disappear).
One solution would be to have a very large swap partition,
under linux. But this is not a good idea in my case,
since I need to keep the dictionary for future analysis.
Except if I save it in the end to another partition ...
why not, actually ?
Another solution would be to use an external database
like SQLite or Metakit.
Which solution would you recommend ?
Speed of execution is very important.
Thanks in advance for any hint !
Yours, Cristian Barbarosie
http://cmaf.fc.ul.pt/~barbaros