cPickle segfault with nested dicts in threaded env

Discussion in 'Python' started by Kenneth Dombrowski, Sep 8, 2010.

  1. Hi all,

    Before creating an issue @ bugs.python.org I wanted to run the
    following by everyone

    We are having a difficult time with what looks like a cPickle issue
    when given a data structure containing >15 nested dictionaries, but
    only when threading is involved

    Environment is FreeBSD 8, Python 2.5.5

    The following diff contains an addition for
    /usr/ports/lang/python25/work/Python-2.5.5/Lib/test/test_cpickle.py
    which reproduces the issue

    kenneth@kenneth0 Python-2.5.5 $ cat
    /tmp/test_cpickle.nested_dicts_in_threaded_env.diff
    1a2
    > import copy

    2a4
    > import threading

    93a96,105
    > def test_nested_dicts_in_threaded_env(self):
    > # segfaults on python2.5.5/FreeBSD 8.0-RELEASE-p2
    > def threadloop():
    > x = {}
    > for i in range(16):
    > x = {i:copy.copy(x)}
    > cPickle.dumps(x, 2)
    > t = threading.Thread(target=threadloop)
    > t.start()
    >


    Any thoughts will be appreciated, thanks for looking,
    Kenneth
    Kenneth Dombrowski, Sep 8, 2010
    #1
    1. Advertising

  2. Kenneth Dombrowski

    Carl Banks Guest

    On Sep 8, 11:58 am, Kenneth Dombrowski <> wrote:
    > Hi all,
    >
    > Before creating an issue @ bugs.python.org I wanted to run the
    > following by everyone
    >
    > We are having a difficult time with what looks like a cPickle issue
    > when given a data structure containing >15 nested dictionaries, but
    > only when threading is involved
    >
    > Environment is FreeBSD 8, Python 2.5.5


    Since Python 2.7 is released, Python 2.5 is no longer accepting bug
    fixes, only security fixes. So be aware.


    > The following diff contains an addition for
    > /usr/ports/lang/python25/work/Python-2.5.5/Lib/test/test_cpickle.py
    > which reproduces the issue
    >
    > kenneth@kenneth0 Python-2.5.5 $ cat
    > /tmp/test_cpickle.nested_dicts_in_threaded_env.diff
    > 1a2
    >
    > > import copy

    > 2a4
    > > import threading

    > 93a96,105
    > >     def test_nested_dicts_in_threaded_env(self):
    > >         # segfaults on python2.5.5/FreeBSD 8.0-RELEASE-p2
    > >         def threadloop():
    > >             x = {}
    > >             for i in range(16):
    > >                 x = {i:copy.copy(x)}
    > >             cPickle.dumps(x, 2)
    > >         t = threading.Thread(target=threadloop)
    > >         t.start()

    >
    > Any thoughts will be appreciated, thanks for looking,


    Bug. Python should never segfault (unless you're doing bad things
    with ctypes). Even if threading isn't supported in this case, it
    should fail with an exception or have undefined behavior, not
    segfault. I would check to see if the same problem exists on 2.7, and
    file a bug report if so. If they fix it in 2.7 maybe you could apply
    the same fix to your own 2.5.


    Carl Banks
    Carl Banks, Sep 8, 2010
    #2
    1. Advertising

  3. Kenneth Dombrowski

    Paul Rubin Guest

    Carl Banks <> writes:
    > Since Python 2.7 is released, Python 2.5 is no longer accepting bug
    > fixes, only security fixes. So be aware.


    Segfaults should be treated as security holes unless there's convincing
    reasons that no exploit is possible. So the bug should be reported
    against 2.5 as well as later versions.
    Paul Rubin, Sep 8, 2010
    #3
  4. On Wed, Sep 8, 2010 at 6:55 PM, Paul Rubin <> wrote:
    > Carl Banks <> writes:
    >> Since Python 2.7 is released, Python 2.5 is no longer accepting bug
    >> fixes, only security fixes.  So be aware.

    >
    > Segfaults should be treated as security holes unless there's convincing
    > reasons that no exploit is possible.  So the bug should be reported
    > against 2.5 as well as later versions.


    Hi Paul,

    Thanks for the input, it sounds reasonable to me. I reported it & the
    maintainers can decide what to do with it:
    http://bugs.python.org/issue9812

    Thanks again everyone,
    Kenneth
    Kenneth Dombrowski, Sep 9, 2010
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ankit Mehta
    Replies:
    1
    Views:
    1,396
    Simon Brooke
    Sep 29, 2006
  2. Zac Burns
    Replies:
    0
    Views:
    215
    Zac Burns
    Nov 10, 2009
  3. bruce
    Replies:
    0
    Views:
    234
    bruce
    Jan 10, 2012
  4. TDR
    Replies:
    3
    Views:
    164
    Daniel Berger
    Aug 31, 2007
  5. Terry Reedy
    Replies:
    0
    Views:
    143
    Terry Reedy
    Nov 27, 2012
Loading...

Share This Page