Multi-threaded SSL

[Kris Kowal]

Dear Ophidians,

I'm attempting to create an SSL secured, AJAX chat server. I'm moving
on the hypothesis that I'll need to hang an XMLHttpRequest response
blocking on the server until a new message is ready to be dispatched.
This means that my server must be able to handle many open SSL sockets
in separate threads.

I started with Twisted, but, having looked as far as I can see, SSL is
either not implemented, or not documented for that library. There are
hints that it's in the works, but that's all. So, I've moved on.

I'm using PyOpenSSL on a Debian box, and I started with the ActiveState
Cookbook article,
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/442473 The SSL
server works very well as suggested in this article.

Starting with this code and adding threads, I've been trying to make
simultaneous HTTP requests operate in parallel on the server. To test,
I've added in turn busy and sleepy waiting to the GET processing
segment of the request handler. The threads work fine; every time the
server accepts a connection, it clearly starts accepting connections in
a new thread. However, the problem runs deeper than I can see. The
SSL listening socket blocks on accept in all threads until the one open
SSL connection finishes its waiting, responds, and closes. This means
that I can only have one client waiting for a response at a time.

Is there a limitation of SSL, or this SSL implementation, or something
else preventing me from having multiple connections waiting for
responses simultaneously?

Many thanks,
Kris Kowal

 

[Alex Martelli]

I started with Twisted, but, having looked as far as I can see, SSL is
either not implemented, or not documented for that library. There are
hints that it's in the works, but that's all. So, I've moved on.

??? SSL is fully implemented in Twisted, AFAIK. Is the example at
<http://twistedmatrix.com/projects/core/documentation/examples/echoserv_
ssl.py> at all unclear or mysterious, for example?


Alex

 

[cowbertvonmoo]

Thanks Alex. I hadn't noticed that example. I gave it a shot and
still have the synchronization problems. While this may be because of
a lack of understanding of twisted threads (again, perhaps I'm just not
looking in the right places, but even the the API reference is sparse
of explanation), I strongly suspect that there's something going on at
a lower level, like the Open SSL wrapper or Open SSL on the system
level. At this point, I'd just like to know if I should cut my losses
and try a very rapid polling solution (*cringes*). Here are my
attempts:

Twisted SSL Foray:
http://cixar.com/svn/mage/twisted_ssl_foray.py
http://cixar.com/websvn/filedetails.php?repname=Cixar&path=/mage/twisted_ssl_foray.py&rev=0&sc=0

Bare SSL Foray:
http://cixar.com/svn/mage/bare_ssl_foray.py
http://cixar.com/websvn/filedetails.php?repname=Cixar&path=/mage/bare_ssl_foray.py&rev=0&sc=0

In the twisted example, I'm logging before and after I start sleeping
in the process request section to note whether the pages are being
processed in parallel. They aren't. My log message order for two
asynchronous page requests should look like:

start (first)
start (second)
stop (first)
stop (second)

But I'm rather getting:

start (first)
stop (first)
start (second)
stop (second)

Again, grateful for your help,
Kris Kowal.

 

[Heikki Toivonen]

I started with Twisted, but, having looked as far as I can see, SSL is
either not implemented, or not documented for that library. There are
hints that it's in the works, but that's all. So, I've moved on.

I'm using PyOpenSSL on a Debian box, and I started with the ActiveState

Twisted actually supports SSL if you have PyOpenSSL installed. Alex
already pointed that out.

If you want to work with Twisted, alternative solutions for the SSL part
could be TLS Lite (see http://trevp.net/tlslite/) or M2Crypto
(http://wiki.osafoundation.org/bin/view/Projects/MeTooCrypto), both of
which provide an alternative SSL transport implementation for Twisted.