Generic ? regarding Threads

[Venks]

Hi,

I am planning to use Ruby threading to query 2 different databases
running on 2 different machines. I understand how Ruby threads are in
process and are NOT native. My requirement is simple. Send the query
request to different database servers running on completely separate
machines and then combine the output from both the queries as one.

I am wondering whether anybody has implemented such requirement and
would like to know your experiences, suggestions etc.

Thanks,

-Venks

 

[Carlos J. Hernandez]

Venks:

My experience on multiple sources is that sometimes
one source says "N/A" while another says "1.99" for the same item in a
record.
So you'll need to decide which source outranks the other, and
other heuristics to determine which data overwrites the other.
For example, "Nil" usually means "I don't know yet", so you
go with the source that claims knowledge.

Also, you may want a local cache to fall back on if
any of the sources are temporarily down and can't get the very latest
data.

Another trick is that sometimes on your updates,
you'll get a thread stuck on a query, a request, a particularly tuff
computation.....
On my dual/core machine I like having 4 threads running,
it's like a 4 lane highway that will keep flow going even is a lane is
closed.

OK, hope that helps.
-Carlos

 

[Venks]

I have much simpler requirements. I don't use Active Record which I
believe is not thread safe. I will be using the native MySQL driver
and also there are no DML, just queries. But the important thing is
that I need to pass a time out parameter based on which I need to kill
the all the threads if the entire process exceeds the time out
parameter.

 

[Carlos J. Hernandez]

I did some very minor experimentation back when the first Database
library came out for Ruby, so
probably this is as far as I can go on this.

When you create a thread
thread1 = Thread.new { your query #1 here }
thread2 = Thread.new { your query #2 here }
You can set up a timeout block waiting for thread1 and thread2 to join
the main thread.
You can use the Timeout library.
Looks like it be something like
begin
Timeout::timeout( @timeout ) do
thread1.join; thread2.join
end
rescue Timeout:ERROR
tread1.kill; thread2.kill;
end
I've not tested the code above, but definitely doable in Ruby.

-Carlos