sharing a (tied) hash between processes

[Thomas Reat]

I have a perl script that maintains its state in a hash of hashes. It
runs all the time. I have another script that must access (read only)
this state. I'd also like to recover it in the event of a restart.

What I'm doing now is to dump it to a file (200kB) every time through
the main loop (every minute or so), despite the fact that the common
case is for there to be no changes. And any changes that are made are
tiny, to only a couple records. Locking is handled by writing to a
temporary file and renaming over the original.

What is an elegant solution for this? I've considered using a real
database, but it seems like overkill. MLDBM looks like it might help
me, but will I be able to have a dbm file opened for writing by one
process and reading by another?

 

[Andy Baxter]

At earth time Thu, 01 Jan 2004 12:49:07 -0800, the following transmission
was received from the entity known as Thomas Reat:

I have a perl script that maintains its state in a hash of hashes. It
runs all the time. I have another script that must access (read only)
this state. I'd also like to recover it in the event of a restart.

What I'm doing now is to dump it to a file (200kB) every time through
the main loop (every minute or so), despite the fact that the common
case is for there to be no changes. And any changes that are made are
tiny, to only a couple records. Locking is handled by writing to a
temporary file and renaming over the original.

What is an elegant solution for this? I've considered using a real
database, but it seems like overkill. MLDBM looks like it might help
me, but will I be able to have a dbm file opened for writing by one
process and reading by another?

The HTML::Template module I'm using can be set to use a module called
IPC::SharedCache to store the parsed templates between calls, so you could
have a look at that.

andy.

 

[R. KRause]

I have a perl script that maintains its state in a hash of hashes. It
runs all the time. I have another script that must access (read only)
this state. I'd also like to recover it in the event of a restart.

Hmm, this is a good question. I've been on the lookout for an elegant
solution to this for quite some time. In one of the modules I wrote
for a specialized chat daemon last year, I resorted to doing a process
fork, and then passing my read-only hash data to a sister process via
bidirectional interprocess pipes using an object request broker type
methodology. The primary advantage was that neither process had to
rely on disk storage operations, thus removing a slower middle layer,
and even more importantly it alleviated dependancy on the internal
data structures used by either process, thus providing a level of
transparency. The disadvantage, was that request operations were not
preemptive (they had to sit in a queue awaiting selection by the host
process, like a typical client-server).

There are other ways of accomplishing this bidirectional communication
technique in UNIX without forked processes, such as open2() and tty
device emulation -- and, if you really want to have fun, socket I/O.
However, I don't believe there is any way to directly share namespaces
or typeglobs between entirely disassociated perl processes. However, I
would be eager to learn that I am wrong in that regard!

--Randall

 

[ctcgag]

I have a perl script that maintains its state in a hash of hashes. It
runs all the time. I have another script that must access (read only)
this state. I'd also like to recover it in the event of a restart.

What I'm doing now is to dump it to a file (200kB) every time through
the main loop (every minute or so), despite the fact that the common
case is for there to be no changes. And any changes that are made are
tiny, to only a couple records. Locking is handled by writing to a
temporary file and renaming over the original.

What is an elegant solution for this? I've considered using a real
database, but it seems like overkill.

I would argue that MySQL is perhaps not overkill. (If in the right mood, I
may also argue it isn't a real database).

Xho

 

[pkent]

I would argue that MySQL is perhaps not overkill. (If in the right mood, I
may also argue it isn't a real database).

I'd second that - it's a database that I'm reliably informed is very
good in read-heavy operations, which sounds like the case here: usually
the state doesn't change, which means there's little writing going on
but much reading.

[ OTOH I have ranted to my coworker at discovering that our version[1]
of MySQL lacks certain features that I take for granted in Oracle, and
indeed that one clause was silently ignored... ]

Also, in my place of work we use MySQL for storing this kind of 'state
information that can be used by more than one program at a time' in all
sorts of applications and it seems to work really well.

Obviously if you've already got another database available then I'm not
saying you _have_ to use MySQL, just that it would probably be suitable
for your application.

P

[1] Your version may differ, read the fine manual, etc