asyncore DoS vulnerability

B

billie

Hi all. I've just terminated a server application using asyncore /
asynchat frameworks.
I wrote a test script that performs a lot of connections to the server
app and I discovered that asyncore (or better, select()) can manage
only a limited number of file descriptors (aka simultaneous
connections).
When this number is reached select() raises an error but asyncore
doesn't handle this exception (a crash occurs).
If you want to try this I pasted two scripts below: a server and a
client.
On my Windows XP system server.py the crash occurs when 512
simultaneous connections are reached.
Here's the traceback:

Traceback (most recent call last):
File "C:\Documents and Settings\root\Desktop\test.py", line 31, in ?
asyncore.loop(timeout=1)
File "C:\Python24\lib\asyncore.py", line 192, in loop
poll_fun(timeout, map)
File "C:\Python24\lib\asyncore.py", line 122, in poll
r, w, e = select.select(r, w, e, timeout)
ValueError: too many file descriptors in select()


Why does this exception isn't handled inside asyncore.py?



----------------------------------------------------
# server.py
import asyncore, socket

class server(asyncore.dispatcher):
"""The base class for the backend."""

def __init__(self):
asyncore.dispatcher.__init__(self)
self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
self.bind(('', 8080))
self.listen(5)

def handle_accept(self):
sock_obj, addr = self.accept()
handler(sock_obj)

class handler(asyncore.dispatcher):

def __init__(self, sock_obj):
asyncore.dispatcher.__init__(self, sock=sock_obj)

def handle_write(self):
pass

def handle_read(self):
pass

server()
asyncore.loop(timeout=1)

-----------------------------------------------------

# client.py
import socket, threading, os

def client():
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
s.connect(("127.0.0.1", 8080))
except:
print x
os._exit(0)
while 1:
s.recv(1024)

x = 0
while 1:
x +=1
threading.Thread(target=client).start()
print x
 
W

William Heymann

Here's the traceback:

Traceback (most recent call last):
File "C:\Documents and Settings\root\Desktop\test.py", line 31, in ?
asyncore.loop(timeout=1)
File "C:\Python24\lib\asyncore.py", line 192, in loop
poll_fun(timeout, map)
File "C:\Python24\lib\asyncore.py", line 122, in poll
r, w, e = select.select(r, w, e, timeout)
ValueError: too many file descriptors in select()

I just tried this on 64bit kubuntu edgy and this is what I got

<snip lots of numbers>
1019
1020
Exception in thread Thread-1021:
Traceback (most recent call last):
File "threading.py", line 442, in __bootstrap
File "threading.py", line 422, in run
File "client.py", line 5, in client
File "socket.py", line 148, in __init__
error: (24, 'Too many open files')

1021
Exception in thread Thread-1022:
Traceback (most recent call last):
File "threading.py", line 442, in __bootstrap
File "threading.py", line 422, in run
File "client.py", line 5, in client
File "socket.py", line 148, in __init__
error: (24, 'Too many open files')

Exception in thread Thread-1020:
Traceback (most recent call last):
File "threading.py", line 442, in __bootstrap
File "threading.py", line 422, in run
File "client.py", line 12, in client
error: (104, 'Connection reset by peer')

1022
1023
 
A

aspineux

Did you take a look for "too many file descriptors in select()" on
google.

Hi all. I've just terminated a server application using asyncore /
asynchat frameworks.
I wrote a test script that performs a lot of connections to the server
app and I discovered that asyncore (or better, select()) can manage
only a limited number of file descriptors (aka simultaneous
connections).

I thing this is the system that limit the number of open sockets or
select() list
When this number is reached select() raises an error but asyncore
doesn't handle this exception (a crash occurs).

This is not a CRASH, It looks an exception with a "Traceback", this is
the normal way python report problems, nothing wrong with that.
You can handle it with a try: except:
If you want to try this I pasted two scripts below: a server and a
client.
On my Windows XP system server.py the crash occurs when 512
simultaneous connections are reached.

512 is probably a fixed limit into XP, win2k3 or win2k server will
accept more.
Maybe It's possible to increase this value somewhere in the registry.
If not this is how microsoft justify the difference between server and
workstation products :)
Here's the traceback:

Traceback (most recent call last):
File "C:\Documents and Settings\root\Desktop\test.py", line 31, in ?
asyncore.loop(timeout=1)
File "C:\Python24\lib\asyncore.py", line 192, in loop
poll_fun(timeout, map)
File "C:\Python24\lib\asyncore.py", line 122, in poll
r, w, e = select.select(r, w, e, timeout)
ValueError: too many file descriptors in select()

Why does this exception isn't handled inside asyncore.py?

To do what ? To raise a custom asyncore error ?
You can can probably run over this limit by starting multiple of your
server process (not thread, process) .
 
B

billie

This is not a CRASH, It looks an exception with a "Traceback", this is
the normal way python report problems, nothing wrong with that.
You can handle it with a try: except:

I think that such a thing should be handled by asyncore itself.
512 is probably a fixed limit into XP, win2k3 or win2k server will
accept more.
Maybe It's possible to increase this value somewhere in the registry.
If not this is how microsoft justify the difference between server and
workstation products :)

Yeah, maybe...
To do what ? To raise a custom asyncore error ?

asyncore aims to be a framework, right?
I think that when select() limit is reached asyncore should just drop
other connections. That's all.
You can can probably run over this limit by starting multiple of your
server process (not thread, process).

Hope you're joking...
Why should I have to run multiple processes / threads to avoid such a
problem?
And what if my system / inteprepter does not support multiple
processes / threads?
 
C

Chris Mellon

I think that such a thing should be handled by asyncore itself.

Handled by doing what, exactly?
Yeah, maybe...


asyncore aims to be a framework, right?
I think that when select() limit is reached asyncore should just drop
other connections. That's all.

Silently rejecting connections in a way that is invisible to the
application is clearly the wrong thing to do.
Hope you're joking...
Why should I have to run multiple processes / threads to avoid such a
problem?

Thats like asking why you should have to move your fingers to type or
why you should have to eat food in order to not starve. Windows is
placing a limit of 512 descriptors per process. Call Microsoft if you
want to go over that.

Or, you can volunteer to write a win32 port of asyncore that uses
native winsock functions instead of the BSD compatibility functions.
And what if my system / inteprepter does not support multiple
processes / threads?

What if it doesn't support sockets? This is a platform limitation, not
an arbitrary Python one. Nothing Python is going to do is going to
convince Windows to select() on more than 512 file descriptors per
process.
 
S

skip

billie> asyncore aims to be a framework, right? I think that when
billie> select() limit is reached asyncore should just drop other
billie> connections. That's all.

You're asking asyncore to make a policy decision on behalf the controlling
application. It has no idea what that application wants to do when the open
file limit is reached. Maybe it should close the oldest connection instead
of refusing all new ones. Maybe it should adjust the rate at which it
accepts new connections. asyncore doesn't know.

Skip
 
B

billie

Thats like asking why you should have to move your fingers to type or
why you should have to eat food in order to not starve. Windows is
placing a limit of 512 descriptors per process. Call Microsoft if you
want to go over that.

?
That's not a select() problem: that's an asyncore problem.
I'm just saying that asyncore should handle this event in some other
way than raising a not well defined "ValueError".
I've discovered this problem accidentally by writing a small test
script but personally I've never seen a paper describing it.
Not handling such a problem just means that an asyncore based server
is vulnerable to DoS attacks and I believe that a lot of servers out
there didn't used a try/except statement around "asyncore.loop()".
imho, such a problem should merit some attention.
Don't you agree?
 
A

aspineux

asyncore aims to be a framework, right?
I think that when select() limit is reached asyncore should just drop
other connections. That's all.

Nice idea.
It shoul be nice to be able to limit the number of connections
asyncore can manage, to say,
limit the load of the server, or in your case tell asyncore you are
working on a poor platform :)

Then asyncore could call a user defined function (a policy) when this
limit is reached.
This function could interact with asyncore to take an action: reject
the connection, close idle connections .....

You could try to persuade asyncore developers to include this feature.

Hope you're joking...
Why should I have to run multiple processes / threads to avoid such a
problem?
And what if my system / inteprepter does not support multiple
processes / threads?

I was just giving a possible workaround.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,764
Messages
2,569,565
Members
45,041
Latest member
RomeoFarnh

Latest Threads

Top