DRb question

[Jos Backus]

I am trying to write a simple remote execution framework, and am running into a
DRb problem I can't seem to figure out. The system consists of a controller and
a number of clients; the controller sends commands to the clients (these
commands in turn will eventually talk to a server). The problem is that when I
make the same method call twice in a row I see the following error on the
controller side:

Terminal A: running the client:

lizzy:~/public_html/ruby/webrun% ./webrun-client.rb
Starting WebRun service on localhost:12345
WebRunClient.add(u1)
users=[u1]
cmd=load_file

Terminal B: running the controller:

lizzy:~/public_html/ruby/webrun% ./webrun-control.rb
assigning proxy for localhost
#<UserPool:0x8174ba8 @userlist={"u1"=>#<User:0x8174b30 @client_class="Client_u1", @name="u1", @filename=nil, @thread=nil>}>
#<UserPool:0x8174234 @userlist={"u1"=>#<User:0x81741bc @client_class="Client_u1", @name="u1", @filename=nil, @thread=nil>}>
user u1 -> client localhost:12345
execute cmd load_file on client localhost:12345 for users [u1]

2nd call fails:
#<DRb:RbUnknown:0x8172d44 @buf="\004\010o:\rUserPool\006:\016@userlist{\006\"\au1o:\tUser\n:\022@client_class\"\016Client_u1:\v@klassc\024User::Client_u1:\016@filename\"\fproc.rb:\f@thread0:\n@name\"\au1", @name="User::Client_u1">
#<DRb:RbUnknown:0x8172060 @buf="\004\010o:\rUserPool\006:\016@userlist{\006\"\au1o:\tUser\n:\022@client_class\"\016Client_u1:\v@klassc\024User::Client_u1:\016@filename\"\fproc.rb:\f@thread0:\n@name\"\au1", @name="User::Client_u1">
/webrun.rb:29:in `has_user?': undefined method `has_user?' for #<DRb:RbUnknown:0x81719d0> (NoMethodError)
from ./webrun.rb:82:in `where_is_user'
from ./webrun.rb:80:in `each'
from ./webrun.rb:75:in `each'
from ./webrun.rb:75:in `each'
from ./webrun.rb:87:in `where_is_user'
from ./webrun.rb:138:in `execute_each_user'
from ./webrun.rb:137:in `each'
from ./webrun.rb:137:in `execute_each_user'
from ./webrun-control.rb:17

The code is at:

http://lizzy.dyndns.org/~jos/ruby/webrun/webrun.rb
http://lizzy.dyndns.org/~jos/ruby/webrun/webrun-client.rb
http://lizzy.dyndns.org/~jos/ruby/webrun/webrun-control.rb
http://lizzy.dyndns.org/~jos/ruby/webrun/proc.rb

Does anybody have any idea what I could be doing wrong or how to debug this
problem?

Thanks,
--
Jos Backus _/ _/_/_/ Sunnyvale, CA
_/ _/ _/
_/ _/_/_/
_/ _/ _/ _/
jos at catnook.com _/_/ _/_/_/ require 'std/disclaimer'

 

[Shashank Date]

Hi Jos,

I am trying to write a simple remote execution framework, and am running

into a

Interesting ... keep us posted.

DRb problem I can't seem to figure out. The system consists of a controller and
a number of clients; the controller sends commands to the clients (these
commands in turn will eventually talk to a server).

Looking at the code makes me feel that the controller is a DRb client
and your client is actually a DRb server. Not a big issue at all ... just
tripped
me initially.

The problem is that when I
make the same method call twice in a row I see the following error on the
controller side:

Terminal A: running the client:

lizzy:~/public_html/ruby/webrun% ./webrun-client.rb
Starting WebRun service on localhost:12345
WebRunClient.add(u1)
users=[u1]
cmd=load_file

Terminal B: running the controller:

lizzy:~/public_html/ruby/webrun% ./webrun-control.rb
assigning proxy for localhost
#<UserPool:0x8174ba8 @userlist={"u1"=>#<User:0x8174b30

@client_class="Client_u1", @name="u1", @filename=nil, @thread=nil>}>

#<UserPool:0x8174234 @userlist={"u1"=>#<User:0x81741bc

@client_class="Client_u1", @name="u1", @filename=nil, @thread=nil>}>

user u1 -> client localhost:12345
execute cmd load_file on client localhost:12345 for users [u1]

2nd call fails:
#<DRb:RbUnknown:0x8172d44

@buf="\004\010o:\rUserPool\006:\016@userlist{\006
^^^^^^^^^^^^^^^
I tried hard to track down the source of this object ... but was not very
successful :-(

/webrun.rb:29:in `has_user?': undefined method `has_user?' for

#<DRb:RbUnknown:0x81719d0>

This was my starting point of investigation

NoMethodError)
from ./webrun.rb:82:in `where_is_user'
from ./webrun.rb:80:in `each'
from ./webrun.rb:75:in `each'
from ./webrun.rb:75:in `each'
from ./webrun.rb:87:in `where_is_user'
from ./webrun.rb:138:in `execute_each_user'
from ./webrun.rb:137:in `each'
from ./webrun.rb:137:in `execute_each_user'
from ./webrun-control.rb:17

Does anybody have any idea what I could be doing wrong or how to debug this
problem?

Well, I can point you to some obvious typos (see line 15 in webrun.rb):

@proxy = DRbObject.new(nil, "druby://#@name:#@port")
^^^^
^^^^
Don't you want this to be:

@proxy = DRbObject.new(nil, "druby://#{@name}:#{@port}")

Search for #@ in your code and you will find few more occurances

And line 17:

yield @proxy if block_given? and @proxy

I would rewrite as:

yield @proxy if (block_given? and @proxy)

just to be on the safe side...."and" has very low precedence.

Unfortunately, even after changing this and using ruby 1.8.1 on Win XP,
I got the same errors as before. I tried running in the debugger mode (with
-rdebug option) and I got even more errors, even for the first call.

All I can suggest is: try single-threaded, non-distributed version first, if
possible
and slowly introduce the more advanced features. Best of luck !

-- shanko

 

[Eric Hodel]

--abYdCjSRCBwcb+dP
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Well, I can point you to some obvious typos (see line 15 in webrun.rb):
=20
@proxy =3D DRbObject.new(nil, "druby://#@name:#@port")

Don't you want this to be:
=20
@proxy =3D DRbObject.new(nil, "druby://#{@name}:#{@port}")

#@varname and #$globalname get expanded to #{@varname} and
#{$globalname}. #$0 is seen sometimes in usage messages like this:

STDERR.puts "#$0: some usage message here about how to run this thing"

I tend not to use them, because the {} tell me where interpolation
begins/ends properly.

--=20
Eric Hodel - (e-mail address removed) - http://segment7.net
All messages signed with fingerprint:
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04


--abYdCjSRCBwcb+dP
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (FreeBSD)

iD8DBQE/97MOMypVHHlsnwQRAmlEAJ0ZSpfilyRbxhztYhyUmKjOecFpWACg27NT
lkE5FpjE21JSeN4dcd+dLbE=
=nv0Z
-----END PGP SIGNATURE-----

--abYdCjSRCBwcb+dP--

 

[Jos Backus]

Btw, this is with ruby-1.8.1 and dRuby 2.0.4 on a recent FreeBSD -current,
installed from ports. I should also add that what's weird is that the _second_
_identical_ call fails in the controller, i.e.:

cp.execute_each_userload_file, args, usernames)
puts "\n2nd call fails:"
cp.execute_each_userload_file, args, usernames)

into a

Interesting ... keep us posted.
Sure.


Looking at the code makes me feel that the controller is a DRb client
and your client is actually a DRb server. Not a big issue at all ... just
tripped
me initially.

Yeah, that is why I gave the short description. The clients will eventually be
making HTTP calls to a web server, which is why I called them clients, even
though from the controller's perspective they are really servers.

The problem is that when I
make the same method call twice in a row I see the following error on the
controller side:

Terminal A: running the client:

lizzy:~/public_html/ruby/webrun% ./webrun-client.rb
Starting WebRun service on localhost:12345
WebRunClient.add(u1)
users=[u1]
cmd=load_file

Terminal B: running the controller:

lizzy:~/public_html/ruby/webrun% ./webrun-control.rb
assigning proxy for localhost
#<UserPool:0x8174ba8 @userlist={"u1"=>#<User:0x8174b30

@client_class="Client_u1", @name="u1", @filename=nil, @thread=nil>}>

#<UserPool:0x8174234 @userlist={"u1"=>#<User:0x81741bc

@client_class="Client_u1", @name="u1", @filename=nil, @thread=nil>}>

user u1 -> client localhost:12345
execute cmd load_file on client localhost:12345 for users [u1]

2nd call fails:
#<DRb:RbUnknown:0x8172d44

@buf="\004\010o:\rUserPool\006:\016@userlist{\006
^^^^^^^^^^^^^^^
I tried hard to track down the source of this object ... but was not very
successful :-(

Same here :-(

#<DRb:RbUnknown:0x81719d0>

This was my starting point of investigation




Well, I can point you to some obvious typos (see line 15 in webrun.rb):

@proxy = DRbObject.new(nil, "druby://#@name:#@port")
^^^^
^^^^
Don't you want this to be:

@proxy = DRbObject.new(nil, "druby://#{@name}:#{@port}")

Search for #@ in your code and you will find few more occurances

This syntax is actually permissible, see

http://www.rubycentral.com/book/tut_classes.html

for some examples. It's smelly in some ways.

And line 17:

yield @proxy if block_given? and @proxy

I would rewrite as:

yield @proxy if (block_given? and @proxy)

just to be on the safe side...."and" has very low precedence.

Fortunately it doesn't matter in this case. Thanks for the reminder though.

Unfortunately, even after changing this and using ruby 1.8.1 on Win XP,
I got the same errors as before. I tried running in the debugger mode (with
-rdebug option) and I got even more errors, even for the first call.

All I can suggest is: try single-threaded, non-distributed version first, if
possible
and slowly introduce the more advanced features. Best of luck !

Thanks. I may have to simplify it some more. Taking out the threading stuff
didn't seem to make any difference though.

-- shanko

--
Jos Backus _/ _/_/_/ Sunnyvale, CA
_/ _/ _/
_/ _/_/_/
_/ _/ _/ _/
jos at catnook.com _/_/ _/_/_/ require 'std/disclaimer'

 

[Jos Backus]

#<DRb:RbUnknown:0x8172060 @buf="\004\010o:\rUserPool\006:\016@userlist{\006\"\au1o:\tUser\n:\022@client_class\"\016Client_u1:\v@klassc\024User::Client_u1:\016@filename\"\fproc.rb:\f@thread0:\n@name\"\au1", @name="User::Client_u1">
./webrun.rb:29:in `has_user?': undefined method `has_user?' for #<DRb:RbUnknown:0x81719d0> (NoMethodError)

Never mind, I found the problem in my code triggering this error after
studying drb.rb and adding some debugging code to it. The User::Client_u1
class apparently wasn't being defined properly in my code, causing the above
error. @name holds the name of the undefined class/module, which was the clue
I was missing.

DRb rulez

--
Jos Backus _/ _/_/_/ Sunnyvale, CA
_/ _/ _/
_/ _/_/_/
_/ _/ _/ _/
jos at catnook.com _/_/ _/_/_/ require 'std/disclaimer'

 

[Shashank Date]

Never mind, I found the problem in my code triggering this error after
studying drb.rb and adding some debugging code to it. The User::Client_u1
class apparently wasn't being defined properly in my code, causing the above
error. @name holds the name of the undefined class/module, which was the clue
I was missing.

Is the working code available for study?
Thanks,
-- shanko

 

[Jos Backus]

Is the working code available for study?

I just updated the code on the website. This is far from complete though, it
needs much more work to be useful As a first step, there's a file named
loadtest.rb that I need to integrate.

Jos

Thanks,
-- shanko

--
Jos Backus _/ _/_/_/ Sunnyvale, CA
_/ _/ _/
_/ _/_/_/
_/ _/ _/ _/
jos at catnook.com _/_/ _/_/_/ require 'std/disclaimer'

 

[Piergiuliano Bossi]

I am trying to write a simple remote execution framework,

Btw, do you know that Phil Tomson's TaskMaster
<http://raa.ruby-lang.org/list.rhtml?name=taskmaster> has exactly this
purpose?

(although I don't know which is its status, nor if it has been ported to
ruby 1.8)

Can you explain in which way your framework is different, please?

Ciao,
Giuliano

 

[Shashank Date]

I just updated the code on the website.

Thanks ! Where can I find apollo.rb ?

I searched on RAA and found a Delphi/Ruby project with that name.
Is it the same one?

-- Shashank

 

[Jos Backus]

Thanks ! Where can I find apollo.rb ?

Actually, you'll want common.rb, I just put it up.

I searched on RAA and found a Delphi/Ruby project with that name.
Is it the same one?
No.

-- Shashank

--
Jos Backus _/ _/_/_/ Sunnyvale, CA
_/ _/ _/
_/ _/_/_/
_/ _/ _/ _/
jos at catnook.com _/_/ _/_/_/ require 'std/disclaimer'

 

[Jos Backus]

Btw, do you know that Phil Tomson's TaskMaster
<http://raa.ruby-lang.org/list.rhtml?name=taskmaster> has exactly this
purpose?

No, thanks for the pointer! It looks very interesting.

(although I don't know which is its status, nor if it has been ported to
ruby 1.8)

Can you explain in which way your framework is different, please?

My "framework" needs to be able to execute multiple tasks/scripts in parallel
on each slave/client, to achieve better scalability. I suppose I could write a
TaskMaster task that handles the threading internally and still lets the
end-user write scripts without having to worry about the threading issue but
I'm not sure how I'd go about that. Sure would be cool though.

Ciao,
Giuliano

--
Jos Backus _/ _/_/_/ Sunnyvale, CA
_/ _/ _/
_/ _/_/_/
_/ _/ _/ _/
jos at catnook.com _/_/ _/_/_/ require 'std/disclaimer'