This is a followup with some more information. The program is written
in twisted and I posted this to twisted-python also. The freebsd list
wants a more detailed backtrace, which even with python compiled with
OPT -g and --with-pydebug isn't happening for some reason.
The script below calls a blocking method using deferToThread. When I
have 5 or so clients all hitting it at the same time I get a segfault
or bus error within minutes.
I recompiled python with --with-pydebug and set OPT to -g, but it
didn't give me any more information in the backtrace which is also
below. Then I ran gdb on the server process and something funny
happened. I didn't get a segfault, but the server started catching
exceptions and printing them to the screen. As soon as I detached gdb
the exceptions stop but it dumps core a minute later (or less). The
exception is pasted below also.
Python was compiled with pthreads, which is the freebsd kse threads as
far as I know. Freebsd version is 5.3-release-p2.
Server code:
-----------------------------------------------------------
from twisted.internet.protocol import Protocol, Factory
from twisted.internet import defer, reactor
from twisted.python import threadable
from twisted.internet import threads
threadable.init(1)
from otransact.otdirect import process
from twisted.python import log
class GetData:
# this calls my blocking method. THe method blocks when doing
database calls with psycopg and when using urllib2 to do an https
POST. Other than that everything is non blocking. psycopg is
supposed to be thread safe, and I would assume urllib2 is also.
def Do(self,data):
response = process(data)
return response
### Protocol Implementation
# This is just about the simplest possible protocol
class Echo(Protocol):
def dataReceived(self, data):
"""As soon as any data is received, write it back."""
reactor.callLater(0, self.Start,data)
def PrintData(self,data):
self.transport.write("%s\r\n" % data)
self.transport.loseConnection()
#reactor.stop()
def Start(self,data):
c = GetData()
d = threads.deferToThread(c.Do,data)
d.addCallback(self.PrintData)
d.addErrback(log.err)
def main():
f = Factory()
f.protocol = Echo
reactor.listenTCP(8000, f)
reactor.run()
if __name__ == '__main__':
main()
Exception raised by server when gdb was running.
-----------------------------------------------------------------------
File "otssl.py", line 42, in ?
reactor.run()
File "/usr/local/lib/python2.4/site-packages/twisted/internet/default.py",
line 126, in run
self.mainLoop()
File "/usr/local/lib/python2.4/site-packages/twisted/internet/default.py",
line 134, in mainLoop
self.runUntilCurrent()
--- <exception caught here> ---
File "/usr/local/lib/python2.4/site-packages/twisted/internet/base.py",
line 423, in runUntilCurrent
call.func(*call.args, **call.kw)
File "/usr/home/otransact/ssl/ots.py", line 72, in Start
d = threads.deferToThread(c.Do,data)
File "/usr/local/lib/python2.4/site-packages/twisted/internet/threads.py",
line 48, in deferToThread
reactor.callInThread(_putResultInDeferred, d, f, args, kwargs)
File "/usr/local/lib/python2.4/site-packages/twisted/internet/base.py",
line 453, in callInThread
self.threadpool.callInThread(_callable, *args, **kwargs)
File "/usr/local/lib/python2.4/site-packages/twisted/python/threadpool.py",
line 130, in callInThread
self.q.put(o)
File "/usr/local/lib/python2.4/Queue.py", line 91, in put
self.not_full.release()
thread.error: release unlocked lock
Backtrace:
