SegFault using deque in 2.4b3

[Stefan Behnel]

Hi!

In Python 2.4b3, the deque is causing a segfault on two different machines I tested on.

With deque, my program runs fine for a while (at least some tens of seconds up to minutes) and then suddenly segfaults.

I'm sorry I can't tell exactly when, but I'm running an application that uses a few hundred deques where elements are appendleft()ed and pop()ed (simple queue). As the application is rather complex (and I didn't read about this error before anywhere), I guess it could be quite a bit of work for me to come up with a test case.

At least I couldn't reproduce it with simple loops running interleaved pop/appendleft and my machine has enough memory, so that's not the problem. I'm not using things like Psyco either, which BTW (in 2.4) segfaults as well (1.2 and CVS).

Anyway, when I replace the deque with this:

class deque(list):
def appendleft(self, item):
self.insert(0, item)

the segfault disappears, so it's definitely deque's fault.

Has anyone else observed this problem?

Stefan

 

[Stefan Behnel]

In Python 2.4-a-3, the deque is causing a segfault on two different

 

[George Yoshida]

In Python 2.4b3, the deque is causing a segfault on two different
machines I tested on.

With deque, my program runs fine for a while (at least some tens of
seconds up to minutes) and then suddenly segfaults.

I'm sorry I can't tell exactly when, but I'm running an application that
uses a few hundred deques where elements are appendleft()ed and pop()ed
(simple queue). As the application is rather complex (and I didn't read
about this error before anywhere), I guess it could be quite a bit of
work for me to come up with a test case.

Has anyone else observed this problem?

With trials and errors, I could come up with a deque program which
splits out a Segmentation Fault. (This program is attached at the
end of this mail.) With this program, I get segfault one or two
times out of three on average.

It uses a poor programming style,like appending myself or creating
unnecessary numbers of threads. Anyway it *does* generate
segfault, so I hope that's OK.

FYI, I've tested on:

Python 2.4a3 (#1, Sep 24 2004, 22:35:05)
[GCC 3.3.3 (SuSE Linux)] on linux2

On Win 2K(Python 2.4a3), it generates an Application Error.


- George

## START OF PROGRAM

import threading
import random
from collections import deque

MAX = 10**5

class Deq(threading.Thread):
def __init__(self, N):
threading.Thread.__init__(self)
self.d = deque()
self.N = N

def run(self):
for i in xrange(self.N):
n = random.random()
if n < 0.5:
self.d.appendleft(n)
elif 0.80 < n:
self.d.append(self)
elif self.d:
self.d.pop()
else:
print ".",

def main():
for i in xrange(40):
deq = Deq(random.randint(0, MAX))
deq.start()

if __name__ == '__main__':
main()
## END OF PROGRAM

 

[George Yoshida]

A bit simpler version(narrow the problem).

## START OF PROGRAM
import threading
import random
from collections import deque

class Deq(threading.Thread):
N = 10 ** 4
def __init__(self):
threading.Thread.__init__(self)
self.d = deque()

def run(self):
for i in xrange(self.N):
n = random.random()
if n < 0.5:
self.d.append(self)
else:
print ".",

def main():
for i in xrange(40):
deq = Deq()
deq.start()

if __name__ == '__main__':
main()
## END OF PROGRAM

- George

 

[Tim Peters]

[George Yoshida]

A bit simpler version(narrow the problem).

....

Thanks! It's more efficient to open a bug report. Running your
program in a debug build triggers an assertion failure in deque's C
implementation, which enabled finding a simple
fails-every-time-at-the-same-place one-thread test case:

from collections import deque
import gc

d = deque()
for i in xrange(100):
print i,
d.append(1)
gc.collect()

deque isn't setting up its garbage-collection correctly; that should
be fixed soon.

 

[George Yoshida]

...

Thanks! It's more efficient to open a bug report.

OK, I'll try to report to sourceforge once a bug is confirmed.


George

 

[Tim Peters]

[George Yoshida]

OK, I'll try to report to sourceforge once a bug is confirmed.

No need this time! I already checked in a fix and a new test.

 

[Stefan Behnel]

Ok, I checked the current CVS-Version of deque and the bug is gone.

Thanks!

Stefan