nonblocking sockets, select, & OpenSSL

[rakaur]

I've been having issues using Ruby, select, and the OpenSSL library.
I've heard from a few people "use threads! Ruby breaks select because
threads rock!" I'm not going to use Ruby's threads, because they're not
real and I don't like them. So, neener.

I can implement simple TLS clients/servers (ie, proof of concepts) just
fine, but when I try to turn a plaintext XMPP stream into a TLS stream
I get errors.

I'm implementing an XMPP server in Ruby (or rather, trying to). XMPP
(aka Jabber) starts out plain text, and if the ability to do TLS is
advertised switches to that. Using the exact same code that works in
simple proof-of-concepts, I repeatedly get "no shared cipher" from the
server's side, and "wrong version number" from the client's side. Due
to the complete and utter lack of documentation excluding test/openssl/
in the Ruby source, I have no idea what these errors mean or how to go
about fixing them. I've tried dozens of things, including moving
methods around, using an unbuffered socket to make sure some weird
stuff wasn't happening, using external clients, using Ruby clients,
etc. I've been at this for nearly a week, and I've consulted with a
dozen people/websites/mailing lists/etc before coming to the general
Ruby community.

This is a blocker. If I can't resolve this, my project cannot be
implemented in Ruby. If it's some stupid side effect of using a main
select loop instead of threads, then I'll have to find a language that
correctly implements this.

Any help would be appreciated. I'm completely stuck.

 

[Bill Kelly]

Hi,

I've been having issues using Ruby, select, and the OpenSSL library.
I've heard from a few people "use threads! Ruby breaks select because
threads rock!" I'm not going to use Ruby's threads, because they're not
real and I don't like them. So, neener.

I myself look forward to the day when Ruby supports native OS threads.

However, there's nothing unreal about Ruby's green threads. Consider
this: You are already using ruby's threads, period. You may not
choose to create *additional* threads, but you're always running at
least one thread. And when you call select(), from your main
thread, ruby calls rb_thread_select() which in turn calls
rb_thread_wait_for() and rb_thread_schedule(), to handle it the
same as any number of ruby threads making select() calls. You're
always using Ruby threads, calling select() from any thread always
goes through the same mechanism. (So, neener ;D)

Search for rb_thread_select if you'd like to look at the implementation:
http://www.ruby-lang.org/cgi-bin/cvsweb.cgi/ruby/eval.c?rev=1.616.2.142
(1.616.2.142 is v1_8_4)

I can implement simple TLS clients/servers (ie, proof of concepts) just
fine, but when I try to turn a plaintext XMPP stream into a TLS stream
I get errors.

I'm implementing an XMPP server in Ruby (or rather, trying to). XMPP
(aka Jabber) starts out plain text, and if the ability to do TLS is
advertised switches to that. Using the exact same code that works in
simple proof-of-concepts, I repeatedly get "no shared cipher" from the
server's side, and "wrong version number" from the client's side. Due
to the complete and utter lack of documentation excluding test/openssl/
in the Ruby source, I have no idea what these errors mean or how to go
about fixing them.

Yeah I would definitely donate $$ toward a ruby OpenSSL cookbook
project.

I'm wondering, what does it mean to "turn a plaintext XMPP stream
into a TLS stream" in code? I mean, if your proof-of-concept
programs work, but this dynamic switching doesn't work - what does
the code look like? Could you post a sample program that reproduces
the error?

I doubt it's select()/thread related unless you've discovered a bug
in Ruby or the OpenSSL extension.


Regards,

Bill

 

[rakaur]

Actually, I'm starting to think it is a bug.

I decided to implement a more thorough proof-of-concept, and it fails
in the same way that my larger project fails in. The simple proof of
concepts, without any select calls, worked fine. But it seems the
problem comes when you try to throw an SSLSocket into a select() call.
It doesn't work as expected. It seems to always return that there's
something to read, and when you call SSLSocket#read (not recv,
apparently), it blocks. SSLSocket doesn't seem to incorporate
io/nonblock, so you can't SSLSocket.nonblock = true as you can with
normal sockets, so there's no chance of getting an Errno:EWOULDBLOCK.

This is either a bug in Ruby, or more likely a bug in OpenSSL/OpenSSL
Ruby bindings.

My (rather hacked up) code is at http://www.ericw.org/ruby/echo/.

 

[rakaur]

My code at the previous URL has been updated, as it seems I've found
the culprit.

SSLSocket#read doesn't behave as it should. If you specify a size (as I
did earlier, 8192) it blocks until that many bytes have been read,
instead of reading up to a maximum of that many bytes as TCPSocket#recv
does.

Is there any obvious way to get around this other than reading it in
one character at a time? This would use significantly more CPU, as it
results in one system call per byte instead of one system call per
maximum of 8192 bytes.

 

[Bill Kelly]

Hi,

My code at the previous URL has been updated, as it seems I've found
the culprit.

SSLSocket#read doesn't behave as it should. If you specify a size (as I
did earlier, 8192) it blocks until that many bytes have been read,
instead of reading up to a maximum of that many bytes as TCPSocket#recv
does.

Is there any obvious way to get around this other than reading it in
one character at a time? This would use significantly more CPU, as it
results in one system call per byte instead of one system call per
maximum of 8192 bytes.

It looks like SSLSocket#pending calls SSL_pending(), which, according to:
http://www.openssl.org/docs/ssl/SSL_pending.html
might be useful.


HTH,

Bill