Asynchronous Objects

B

Bryan Castillo

I've been looking through CPAN and the newsgroups for information on
calling on objects asynchronously from perl. Basically, I was
thinking about a Tk application I was working on that used DBI and had
some SQL statements that may take 10 to 30 seconds. I really didn't
want the screen to freeze for that long.

So I've been searching for methods of asynchronous operations on
objects. There are of course threads, however I seems that it is not
reccomended to use multiple database connections with threads, or to
even have more than 1 thread use the DBI module, even when protected
through synchronization.

What I've come up with is the start of a module that uses IPC::Open2
to start a new perl interpreter. The process is givin a package name
containing methods it should delegate out to. The process reads in
complex hashes that have been serialized with the Storable module, one
element in the hash contains the name of the method to call in the
delegate package. The method is called and the results are also
returned serialized with the storable module.

Ive then been building a client library build around select(4-arg
form) to try to send and received the data (method call and argumentS)
non-blocking.

# it kind of works like this....

my $timeout = 10;
my $server = AsyncObject::Server::IPCO2->new();

# Internally the line above is doing this to start a new process
my $command =
'use AsyncObject::Server; '.
'my $server = AsyncObject::Server->new(\\*STDIN,\\*STDOUT); '.
'$server->start();';
my $pid = open2(
$rdrfh, $wtrfh,
$Config{perlpath},
'-e',
$command
);
#-----------------------

my $request = $server->command('load', 'Async::Delegate::database');
my $reply;
while (!$reply = $request->getReply($timeout)) {
# whatever
}
if ($reply->{status} != 1) {
die $reply->{error};
}

$request = $server->call('connect', $user, $pass);
while (!$reply = $request->getReply($timeout)) {
# ....
}
if ($reply->{status} != 1) {
die $reply->{error};
}

Then i was planning on exposing the input and output handles, which
could be
used from Tk fileevent, or with select, (to handle more than one
asynchronous object call.)

# just pseudo - code.


$request = $server->call('exec_sql', 'select count(*) from blah');
my $in = $request->getInputHandle();

# use select on $in to determine when request can be sent to server
# perhaps in a fileevent
# can use more granular methods on callback from select
$request->send($timeout);

if ($request->sendComplete()) {
my $out = $request->getOutputHandle();
# use select with output handle or fileevent
on_callback:
$request->receive($timeout);
if ($request->complete()) {
my $result = $request->getResult();
}
}



Is there anything out there like this? I haven't been able to find
anything on CPAN. There is RPC::plServer and RPC::plClient, however I
was looking for something where the client would start the server on
demmand (theoreticaly this could still be done with
RPC::pl(Server|Client), but I didn't want to use sockets), and I was
looking for something that would allow asynchronous calls on the
objects.
 
U

Uri Guttman

BC" == Bryan Castillo said:
I've been looking through CPAN and the newsgroups for information on
calling on objects asynchronously from perl. Basically, I was
thinking about a Tk application I was working on that used DBI and
had some SQL statements that may take 10 to 30 seconds. I really
didn't want the screen to freeze for that long.
So I've been searching for methods of asynchronous operations on
objects. There are of course threads, however I seems that it is
not reccomended to use multiple database connections with threads,
or to even have more than 1 thread use the DBI module, even when
protected through synchronization.

check out stem in my author dir URI or from stemsystems.com. the soon to
be released .11 version is even better. it has exactly what you are
looking for in a services module called Stem::Cell::Flow (in .11). it
supports flow control with sync AND async operations on an object. in
fact the async operation could be in the same process and be sync for
development and later migrated to other stem processes for production
and no code will need to be changed.

What I've come up with is the start of a module that uses IPC::Open2
to start a new perl interpreter. The process is givin a package name
containing methods it should delegate out to. The process reads in
complex hashes that have been serialized with the Storable module, one
element in the hash contains the name of the method to call in the
delegate package. The method is called and the results are also
returned serialized with the storable module.

this is effectively what stem does but it is much more flexible and
powerful as this is only one feature of the system.
Ive then been building a client library build around select(4-arg
form) to try to send and received the data (method call and argumentS)
non-blocking.

again, stem is already based on event loops (and .11 will support both
Event.pm and a pure perl event loop that runs on winblows).

but stem goes beyond just event processing with its message passing
design. you can use message passing to emulate almost any other IPC
paradigm and with little effort. RPC, publish/subscribe, peer to peer,
client/server are all easily done under stem.

let me know if you have any questions or need help with this project. i
can get you a prerelease version of .11 with the stem flow support.

uri
 
S

Steve Lidie

Bryan Castillo said:
I've been looking through CPAN and the newsgroups for information on
calling on objects asynchronously from perl. Basically, I was
thinking about a Tk application I was working on that used DBI and had
some SQL statements that may take 10 to 30 seconds. I really didn't
want the screen to freeze for that long.

A traditional Perl/Tk approach is to have the Tk parent fork a network
helper task. The two talk via pipes or sockets and use fileevent() to
prevent blocking the Tk event loop. The Tk task could assemble SQL
statements, send them to the helper to do the actual DBI work. When
the query/task is complete the helper writes information to the GUI.

There are several examples floating around the net and in books and
magazines. Check out the IPC sample code in Chapter 19 here:
http://www.lehigh.edu/~sol0/ptk/mptk-code/mptk-code19.tar.gz
 
R

Rocco Caputo

I've been looking through CPAN and the newsgroups for information on
calling on objects asynchronously from perl. Basically, I was
thinking about a Tk application I was working on that used DBI and had
some SQL statements that may take 10 to 30 seconds. I really didn't
want the screen to freeze for that long.
[...]

Is there anything out there like this? I haven't been able to find
anything on CPAN. There is RPC::plServer and RPC::plClient, however I
was looking for something where the client would start the server on
demmand (theoreticaly this could still be done with
RPC::pl(Server|Client), but I didn't want to use sockets), and I was
looking for something that would allow asynchronous calls on the
objects.

POE cooperates with Tk, and the CPAN has perhaps too many asynchronous
DBI abstractions available for it. The POE DBI components usually do
what you outlined, but they differ in their approaches to executing SQL
and managing child processes.

Look for POE DBI on http://search.cpan.org/
 
B

Bryan Castillo

Uri Guttman said:
check out stem in my author dir URI or from stemsystems.com. the soon to
be released .11 version is even better. it has exactly what you are
looking for in a services module called Stem::Cell::Flow (in .11). it
supports flow control with sync AND async operations on an object. in
fact the async operation could be in the same process and be sync for
development and later migrated to other stem processes for production
and no code will need to be changed.



this is effectively what stem does but it is much more flexible and
powerful as this is only one feature of the system.


again, stem is already based on event loops (and .11 will support both
Event.pm and a pure perl event loop that runs on winblows).

but stem goes beyond just event processing with its message passing
design. you can use message passing to emulate almost any other IPC
paradigm and with little effort. RPC, publish/subscribe, peer to peer,
client/server are all easily done under stem.

let me know if you have any questions or need help with this project. i
can get you a prerelease version of .11 with the stem flow support.

uri

Thanks,

I downloaded .10, I think I would like to look at .11. I've been
looking to write something that works both on windows and unix. Will
..11 be available soon? You don't have to send it to me directly, my
application is just for fun and the love of perl.


(I'm generally trying to figure out how to perform
multithreaded/multiprocess operations in perl. This is something I've
done before using fork/exec/etc.... and I've used the threads in 5.8,
it just seems like it is alot harder than it should be. What i've
read, discourages use of DBI with threads, which keeps you from
performing multiple queries/operations at once. Someone else,
suggested looking at POE, which I will. Although I doubt it returns
control to your perl script while a query is still executing. In the
end I was hoping to come up with a module that allows various types of
asynchronous operations on objects, where the execution context,
carrying out the operations, is completly managed by the initial
execution context.)
 
B

Bryan Castillo

Thanks,

I downloaded .10, I think I would like to look at .11. I've been
looking to write something that works both on windows and unix. Will
.11 be available soon? You don't have to send it to me directly, my
application is just for fun and the love of perl.


(I'm generally trying to figure out how to perform
multithreaded/multiprocess operations in perl. This is something I've
done before using fork/exec/etc.... and I've used the threads in 5.8,
it just seems like it is alot harder than it should be. What i've
read, discourages use of DBI with threads, which keeps you from
performing multiple queries/operations at once. Someone else,
suggested looking at POE, which I will. Although I doubt it returns
control to your perl script while a query is still executing.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(I was wrong to say this before checking out POE
it does have some modules to run database queries in a new
process, where the original script will receive control
again during a long query. It was stupid of me to say something
about a technology I knew nothing about.)
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,770
Messages
2,569,584
Members
45,077
Latest member
SangMoor21

Latest Threads

Top