Which server component?

[Tim Bennen]

Hello!

I have the scenarion, that on the one hand, there are mobile devices
which are the clients, on the other hand there is a server with ruby.

The clients send the messages to the server witch JSON via a persistent
TCP connection (it must be persistent). The server (ruby scripts)
receive the messages and send a JSON answer string back to the client.

My ruby tcp server is build up with this snippet of source code:
http://segment7.net/projects/ruby/snippets/server.rb

Now I like to run it on port 80/443 as a webservice, because the tcp
server is not so scalable and I do not have any idea how to run it
secure via ssl/tls.
What´s the best way to do this? The ruby-side ist pure ruby (not rails)
with gems like Sequel, Validatable, JSON.

Should I use an apache webserver mit Phusion Passenger or there are
better ways that the server could make persistent tcp connections with
security (SSL) to send/receive JSON messages?

Thanks a lot in advance & Best Regards.

 

[Brian Candler]

The clients send the messages to the server witch JSON via a persistent

TCP connection (it must be persistent). The server (ruby scripts)
receive the messages and send a JSON answer string back to the client.

My ruby tcp server is build up with this snippet of source code:
http://segment7.net/projects/ruby/snippets/server.rb

Now I like to run it on port 80/443 as a webservice, because the tcp
server is not so scalable and I do not have any idea how to run it
secure via ssl/tls.

You are talking about two different things. Webservers talk HTTP(S). You
can have a persistent HTTP connection, but the messages sent over it
would be HTTP requests and responses.

At the moment, you are talking directly over TCP, and your message
structure is just JSON objects delimited by newline.

So really you have two options:

- scale your existing protocol. You could make it work very efficiently
with thousands of concurrent connections using 'eventmachine' or one of
the other reactor-pattern libraries. Adding SSL to your existing code is
also pretty easy.

- or you switch entirely to a new protocol, which would be JSON over
HTTP(S). This is more heavyweight (higher overhead), but does give you a
large number of scaling options and is arguably more 'standard' than
your existing custom protocol.

Should I use an apache webserver mit Phusion Passenger or there are
better ways that the server could make persistent tcp connections with
security (SSL) to send/receive JSON messages?

As this is a very small service and you want it to run as fast as
possible, I suggest you write it as a Rack application. There are then
lots of different ways you can actually run your Rack application (under
Phusion Passenger is just one of them)

Look at the examples which come with Rack to see how it works, it's
extremely simple.
http://rack.rubyforge.org/

 

[Tim Bennen]

Thank you very much for your reply!

You are talking about two different things. Webservers talk HTTP(S). You
can have a persistent HTTP connection, but the messages sent over it
would be HTTP requests and responses.

Okay, I understand what you mean.

At the moment, you are talking directly over TCP, and your message
structure is just JSON objects delimited by newline. Right.

So really you have two options:

- scale your existing protocol. You could make it work very efficiently
with thousands of concurrent connections using 'eventmachine' or one of
the other reactor-pattern libraries. Adding SSL to your existing code is
also pretty easy.

If I would use EventMachine, how would be the architecture? EventMachine
receives the messages and forward it to my ruby scripts? But
EventMachine would replace my TCP server? And with EventMachine it is
easy to set up a persistent SSL connection?

- or you switch entirely to a new protocol, which would be JSON over
HTTP(S). This is more heavyweight (higher overhead), but does give you a
large number of scaling options and is arguably more 'standard' than
your existing custom protocol. Okay.


As this is a very small service and you want it to run as fast as
possible, I suggest you write it as a Rack application. There are then
lots of different ways you can actually run your Rack application (under
Phusion Passenger is just one of them)

Okay, I could build up an apache webserver with for example phusion
passenger which enables the ssl security and the persistent connection.
And then I use Rack to get the requests and I can send the responses
back to the client. Is that right? But how do I call the server from the
client? Is it a POST HTTP Request with the JSON message in his body?

Look at the examples which come with Rack to see how it works, it's
extremely simple.
http://rack.rubyforge.org/

Ruby on Rails did also use this? I have to read a little bit...
Thanks a lot!

 

[Brian Candler]

If I would use EventMachine, how would be the architecture? EventMachine
receives the messages and forward it to my ruby scripts? But
EventMachine would replace my TCP server?

Yes, eventmachine would replace your TCP server. You would have to rejig
your application into an event-handling model, rather than a threaded
model. Because there are no threads, it scales very well.

I'm assuming here that the scaling you're looking for is large numbers
of concurrent TCP connections, more than large numbers of messages per
second. But you can also scale to multiple cores and/or servers by
putting a simple TCP load balancer in front of your server(s). 'pen' is
a suitable software load-balancer, or you can buy lots of different
hardware appliances.

And with EventMachine it is
easy to set up a persistent SSL connection?

I'm not sure, you'd have to look for samples in the eventmachine
distribution and/or ask on an EventMachine list. But I would strongly
expect it has been done before.

With a simple TCP socket server like you already have, it's easy to turn
it into an SSL server. For example, there is code in ruby-ldapserver
which does this (look at startssl in connection.rb and ssl_prepare in
server.rb)

Okay, I could build up an apache webserver with for example phusion
passenger which enables the ssl security and the persistent connection.
And then I use Rack to get the requests and I can send the responses
back to the client. Is that right?

Yes. And for initial testing, you can run the rack app directly under
mongrel or thin or even webrick (rackup will start these servers for
you)

But how do I call the server from the
client? Is it a POST HTTP Request with the JSON message in his body?

Yes. And you'd send back an application/json response with the JSON
message in the body.

Note that HTTP limits you to exactly one response per request. That is,
the server can't send back asynchronous unsolicited responses to the
client. If you want to do that with HTTP, you'd have to get the client
to send a request and block waiting for a response (aka 'COMET') or poll
periodically.

 

[Tim Bennen]

I'm assuming here that the scaling you're looking for is large numbers

of concurrent TCP connections, more than large numbers of messages per
second.

Yes, that´s right!

But you can also scale to multiple cores and/or servers by
putting a simple TCP load balancer in front of your server(s). 'pen' is
a suitable software load-balancer, or you can buy lots of different
hardware appliances.

Okay, but this seems to me not to be the "standard way". It would be
better if I will use standard technologies, so other people could easily
maintain it, like an Apache webserver. But this is a good alternative!

I'm not sure, you'd have to look for samples in the eventmachine
distribution and/or ask on an EventMachine list. But I would strongly
expect it has been done before.

Okay, I will look.

With a simple TCP socket server like you already have, it's easy to turn
it into an SSL server. For example, there is code in ruby-ldapserver
which does this (look at startssl in connection.rb and ssl_prepare in
server.rb)

That seems easy, easier than expected. That's all? Only 10-15 lines of
code to establish a SSl connection? Wow.

Yes. And for initial testing, you can run the rack app directly under
mongrel or thin or even webrick (rackup will start these servers for
you)

Okay, I read: "One popular configuration is to run Apache 2.2 as a load
balancer using mod_proxy_balancer in conjunction with several Mongrel
instances, with each Mongrel instance running on a separate port. This
is something that can be configured very easily using the
mongrel_cluster management utility. Apache can divide the incoming
requests among the available Mongrel processes, and, with careful
configuration, can even serve static content itself without having to
delegate to Mongrel.

For those who wish to avoid Apache altogether, it is possible to deploy
a Mongrel cluster with an alternative web server, such as nginx or
lighttpd, and a load balancer of some variety such as Pound or a
hardware-based solution." (wikipedia)

Do you think that is a good idea to do this?
Perhaps I will build up an Apache webserver with one Mongrel webserver
with Rack and my ruby scripts on one hardware server. This must be
possible?!
If I will need more webserver, then I buy a new hardware server, install
mongrel with my application and that's fine. But isn't it a problem with
the database if I only have one database and several webserver which
connect/read/write?

Yes. And you'd send back an application/json response with the JSON
message in the body.

Okay, that would be perfect.

Note that HTTP limits you to exactly one response per request. That is,
the server can't send back asynchronous unsolicited responses to the
client. If you want to do that with HTTP, you'd have to get the client
to send a request and block waiting for a response (aka 'COMET') or poll
periodically.

Does that mean, the client establish a connection and only(!) if the
client requests something, there will be one response from the server?
So, it is not possible that the server send a message without(!) a
request from the client?

Thanks a lot and best regards!

 

[Brian Candler]

Okay, I read: "One popular configuration is to run Apache 2.2 as a load
balancer using mod_proxy_balancer in conjunction with several Mongrel
instances,

...

Ignore that. Write your application as a rack application (just a single
Ruby object with a 'call' method). Then test it using the built-in
mongrel/thin/webrick handler. When you're happy with it, *then* look at
ways of deploying a rack application. At the moment, Phusion Passenger
(under either apache or nginx) and Unicorn (under nginx) look like good
options and both are much simpler than messing around with multiple
application instances started manually and bound to different ports.

Does that mean, the client establish a connection and only(!) if the
client requests something, there will be one response from the server?
So, it is not possible that the server send a message without(!) a
request from the client?

Correct. So the client must send a request GET /messages (say). If the
server responds immediately with 'no message available' then the client
must poll periodically. If the server blocks and doesn't send a response
until there is one, then you get COMET. If you're running COMET with
something like Apache or nginx in front, you need to beware that they
may time out idle connections, there may be some tweaking to do.

Or you can use a hybrid approach: poll, block for up to 2 minutes (say),
then reply with a message or 'no message available'.

 

[Tim Bennen]

Yes. And for initial testing, you can run the rack app directly under

...

Ignore that. Write your application as a rack application (just a single
Ruby object with a 'call' method). Then test it using the built-in
mongrel/thin/webrick handler. When you're happy with it, *then* look at
ways of deploying a rack application. At the moment, Phusion Passenger
(under either apache or nginx) and Unicorn (under nginx) look like good
options and both are much simpler than messing around with multiple
application instances started manually and bound to different ports.

Okay, I will do it that way. But I should prepare the load balancing, so
I think I will use one apache as a load balancer and one mongrel/rack
combination. Is the Mongrel webserver included into Rack? I don't think
so, only the "gateway" between a Mongrel and Rack?

Correct. So the client must send a request GET /messages (say). If the
server responds immediately with 'no message available' then the client
must poll periodically. If the server blocks and doesn't send a response
until there is one, then you get COMET. If you're running COMET with
something like Apache or nginx in front, you need to beware that they
may time out idle connections, there may be some tweaking to do.

Hm, that's a problem, because if something happened on the server, and
the user should be notified, I have to poll every 2-3 seconds and that's
a lot of data, because the clients could be mobile devices and that's
expensive.
Is there a Ruby Comet implementation?

Or you can use a hybrid approach: poll, block for up to 2 minutes (say),
then reply with a message or 'no message available'.

Okay, that´s also a possible solution...

Using Rack/Mongrel or Apache/Mongrel/Rack the connection could be
established as persistent?

Thanks a lot! Regards.

 

[Robert Klemme]

Hm, that's a problem, because if something happened on the server, and
the user should be notified, I have to poll every 2-3 seconds and that's
a lot of data, because the clients could be mobile devices and that's
expensive.

In that case it sounds like as if HTTP was not the ideal protocol for
the task. Maybe you should rather follow the other direction and think
about a solution with EventMachine as Brian has sketched. That way you
can also have better control over the protocol overhead - HTTP can be
quite verbose if your messages are really small (say 50 bytes or so):
the ratio of payload to protocol wouldn't be too good in that case.

Using Rack/Mongrel or Apache/Mongrel/Rack the connection could be
established as persistent?

AFAIK with HTTP even if the client indicates he wants to stay connected
the server is free to drop the connection if he feels like it (e.g.
running low on open file descriptors, too much idle time etc. Brian
hinted in the same direction). That, too, seems like a reason to look
at something other than HTTP. My 0.02 EUR anyway.

Kind regards

robert

 

[Brian Candler]

In that case it sounds like as if HTTP was not the ideal protocol for
the task. Maybe you should rather follow the other direction and think
about a solution with EventMachine as Brian has sketched.

Another option is to use XMPP (aka Jabber). This is asynchronous and
there are a variety of implementations to choose from. If you want
something that scales to tens of thousands of connections, look at
ejabberd. Even though that's written in erlang, you could still sit it
in between a large pool of clients and a server written in ruby +
xmpp4r.