initialize object permanently (only once)

[Keenlearner]

Hello, I am using Wordnet::QueryData which allow access to a very huge
dictionary data. The initialization of object
my $wn = WordNet::QueryData->new;

took
2 wallclock secs ( 2.36 usr + 0.07 sys = 2.43 CPU)

Then the subsequent request for the data is exetremely fast

For the lines below took
0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU)

print "Synset: ", join(", ", $wn->querySense("cat#n#7", "syns")),
"\n";
print "Hyponyms: ", join(", ", $wn->querySense("cat#n#1", "hypo")),
"\n";
print "Parts of Speech: ", join(", ", $wn->querySense("run")), "\n";
print "Senses: ", join(", ", $wn->querySense("run#v")), "\n";
print "Forms: ", join(", ", $wn->validForms("lay down#v")), "\n";
print "Noun count: ", scalar($wn->listAllWords("noun")), "\n";
print "Antonyms: ", join(", ", $wn->queryWord("dark#n#1", "ants")),
"\n";

I
am developing a web application, is there a way to make the
initialization of object permanently in memory ? I tried to use the
Storable module. But that only give me a little increase in
performance. Anybody's idea is very much appreciated, Thank you.


William

Send instant messages to your online friends http://uk.messenger.yahoo.com

 

[A. Sinan Unur]

I am developing a web application, is there a way to make the
initialization of object permanently in memory ? I tried to use the
Storable module. But that only give me a little increase in
performance. Anybody's idea is very much appreciated, Thank you.

Short of using mod_perl or FastCGI, I don't think so.

Incidentally, it looks like there is a bug in this module:

# Perform all initialization for new WordNet class instance
sub _initialize#
{
my $self = shift;
....
my $old_separator = $/;
$/ = "\n";
....
# Return setting of input record separator
$/ = $old_separator;
}

Shouldn't the setting and the re-setting of $/ be replaced with

local $/ = "\n";

I see potential problems with _initialize in a persistent environment.

Could someone please confirm?

Sinan

--
A. Sinan Unur <[email protected]>
(remove .invalid and reverse each component for email address)

comp.lang.perl.misc guidelines on the WWW:
http://www.rehabitation.com/clpmisc/

 

[xhoster]

Hello, I am using Wordnet::QueryData which allow access to a very huge
dictionary data. The initialization of object
my $wn = WordNet::QueryData->new;

took
2 wallclock secs ( 2.36 usr + 0.07 sys = 2.43 CPU)

Then the subsequent request for the data is exetremely fast
....
print "Noun count: ", scalar($wn->listAllWords("noun")), "\n";

This one is problematic (i.e. slow) with the method I will outline below.
So I commented it out and tested only the other requests.

I
am developing a web application, is there a way to make the
initialization of object permanently in memory ?

Under straightforward CGI, the process ends when the request is done, so
the object ends with it. So the real answer here is to use mod_perl
or one of the other ways of running CGI with a persistent perl process.

I tried to use the
Storable module. But that only give me a little increase in
performance. Anybody's idea is very much appreciated, Thank you.

Again, the best answer is almost certainly to use mod_perl or the like.
But just for kicks, I made it work with DBM:eep, version 0.983. This way
the data is stored on disk and only the parts actually used are read into
memory. As alluded to above, listAllWords defeats the purpose of this as
it goes through so much stuff, causing to be read into memory.

First one needs to prepare the database, to be held in "file.dbm". This
only needs to be done one time, or whenever the data files are updated:

perl -e 'use WordNet::QueryData; my $x= WordNet::QueryData->new(); \
delete $x->{data_fh}; use DBM:eep; my $h=DBM:eep->new("file.dbm"); \
$h->{foo}=$x;'

Once that is done, you can replace "my $wn = WordNet::QueryData->new;"
with:

use DBM:eep;
my $x=DBM:eep->new("file.dbm")->{foo};
my $wn={}; %$wn=%$x; #convert the top level to regular hash
#(so it can be blessed)
delete $wn->{data_fh}; # I don't know why this is necessary
# as this entry should not be here to begin with
bless $wn, 'WordNet::QueryData';
$wn->openData; ## restore contents of $wn->{data_fh)

And from here use $wn just as before. This is very similar to the
"Storable" method demonstrated a few months ago in comp.lang.perl.modules,
except using this method data is read from disk only when requested.

Here are timings on my machine for your "subsequent requests", except for
the listAllWords ones:

original:
0:01.73
Storable
0:00.50
DBM:eep
0:00.07

I have compared the output of all methods and they are identical, but that
doesn't constitute a rigorous test, so use with caution.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.