Threading server exits ERESTARTNOHAND

R

Rune Hansen

select(0, NULL, NULL, NULL, {0, 8000}) = 0 (Timeout)
futex(0x822f530, FUTEX_WAKE, 1) = 0
gettimeofday({1075823054, 603000}, NULL) = 0
futex(0x822f530, FUTEX_WAKE, 1) = 0
select(0, NULL, NULL, NULL, {0, 16000}) = 0 (Timeout)
futex(0x822f530, FUTEX_WAKE, 1) = 0
gettimeofday({1075823054, 623210}, NULL) = 0
futex(0x822f530, FUTEX_WAKE, 1) = 0
select(0, NULL, NULL, NULL, {0, 30370}) = ? ERESTARTNOHAND (To be restarted)
+++ killed by SIGSEGV +++

This is the output from strace -o xx python2.3 Spider.py, Spider.py
being a threaded web crawler. The server segfaults at no specific
interval, running for 30 secs and up to 10-15 minutes.

I can't figure out what's going on. In my years of threaded programming
this is the first time I've seen it (if it makes me a novice, please
don't rub it in :) I've found some information about "ERESTARTNOHAND"
(one hint was /etc/security/limits.conf) and gotten the feeling that it
has more to do with RedHat 9.0 than python2.3.3, but I'd really like to
hear from someone who perhaps has got some experience with this.

Server and setup:
Compiled Python2.3.3 on RedHat 9.0 with default Athlon kernel.

The server starts a XMLRPCServer and Controller. The Controller starts
15 threads doing urllib2.urlopen with data, using queue.get_nowait(),
received on the XMLRPCServer . Each thread has it's own MySQLdb
connection doing INSER DELAYED.
mxTidy is used before data is fed to an HTMLParser instance.

regards

/rune
 
R

Rune Hansen

Rune said:
select(0, NULL, NULL, NULL, {0, 8000}) = 0 (Timeout)
futex(0x822f530, FUTEX_WAKE, 1) = 0
gettimeofday({1075823054, 603000}, NULL) = 0
futex(0x822f530, FUTEX_WAKE, 1) = 0
select(0, NULL, NULL, NULL, {0, 16000}) = 0 (Timeout)
futex(0x822f530, FUTEX_WAKE, 1) = 0
gettimeofday({1075823054, 623210}, NULL) = 0
futex(0x822f530, FUTEX_WAKE, 1) = 0
select(0, NULL, NULL, NULL, {0, 30370}) = ? ERESTARTNOHAND (To be
restarted)
+++ killed by SIGSEGV +++

This is the output from strace -o xx python2.3 Spider.py, Spider.py
being a threaded web crawler. The server segfaults at no specific
interval, running for 30 secs and up to 10-15 minutes.

I can't figure out what's going on. In my years of threaded programming
this is the first time I've seen it (if it makes me a novice, please
don't rub it in :) I've found some information about "ERESTARTNOHAND"
(one hint was /etc/security/limits.conf) and gotten the feeling that it
has more to do with RedHat 9.0 than python2.3.3, but I'd really like to
hear from someone who perhaps has got some experience with this.

Server and setup:
Compiled Python2.3.3 on RedHat 9.0 with default Athlon kernel.

The server starts a XMLRPCServer and Controller. The Controller starts
15 threads doing urllib2.urlopen with data, using queue.get_nowait(),
received on the XMLRPCServer . Each thread has it's own MySQLdb
connection doing INSER DELAYED.
mxTidy is used before data is fed to an HTMLParser instance.

regards

/rune

In answer to my own question I'd like to quote our benevolent dictator:

"In any project that is multi-threaded, most bugs will come from
threading issues.
This is regardless of programming language -- it's a deep, as yet
ununderstood property of threads."
-- Guido van Rossum

So, there you are. Lesson learned.

regards

/rune
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,743
Messages
2,569,478
Members
44,899
Latest member
RodneyMcAu

Latest Threads

Top