make test segfaults with "--enable-shared" on Python 2.3.3

Discussion in 'Python' started by Berthold Hoellmann, Dec 28, 2003.

  1. Hello,

    When I use

    ./configure --with-thread --with-fpectl --with-signal-module \
    --with-pymalloc --enable-shared --with-cxx=g++

    make test

    on 2.3.3 I get

    ....
    test_queue
    test_quopri
    test_random
    test_re
    make: *** [test] Speicherzugriffsfehler (Speicherauszug erstellt)

    gdb ./python core.29964
    GNU gdb 5.3.93
    Copyright 2003 Free Software Foundation, Inc.
    ....
    Reading symbols from /home/devel/compile/Python-2.3.3/build/lib.linux-i686-2.3/pyexpat.so...done.
    Loaded symbols for /home/devel/compile/Python-2.3.3/build/lib.linux-i686-2.3/pyexpat.so
    #0 0x400f126c in sre_match (state=0xbfffcaf0, pattern=0x41126a0c, level=6840)
    at Modules/_sre.c:750
    750 {
    (gdb) bt
    #0 0x400f126c in sre_match (state=0xbfffcaf0, pattern=0x41126a0c, level=6840)
    at Modules/_sre.c:750
    #1 0x400f2084 in sre_match (state=0xbfffcaf0, pattern=0x41126a0c, level=6839)
    at Modules/_sre.c:1257
    #2 0x400f20ef in sre_match (state=0xbfffcaf0, pattern=0x41126a0c, level=6838)
    at Modules/_sre.c:1271
    #3 0x400f20ef in sre_match (state=0xbfffcaf0, pattern=0x41126a0c, level=6837)
    at Modules/_sre.c:1271
    #4 0x400f20ef in sre_match (state=0xbfffcaf0, pattern=0x41126a0c, level=6836)
    at Modules/_sre.c:1271
    #5 0x400f20ef in sre_match (state=0xbfffcaf0, pattern=0x41126a0c, level=6835)
    at Modules/_sre.c:1271
    #6 0x400f20ef in sre_match (state=0xbfffcaf0, pattern=0x41126a0c, level=6834)
    at Modules/_sre.c:1271
    #7 0x400f20ef in sre_match (state=0xbfffcaf0, pattern=0x41126a0c, level=6833)
    at Modules/_sre.c:1271
    #8 0x400f20ef in sre_match (state=0xbfffcaf0, pattern=0x41126a0c, level=6832)
    at Modules/_sre.c:1271
    ....
    #6838 0x400f20ef in sre_match (state=0xbfffcaf0, pattern=0x41126a0c, level=2)
    at Modules/_sre.c:1271
    #6839 0x400f157c in sre_match (state=0xbfffcaf0, pattern=0x41126a0c, level=1)
    at Modules/_sre.c:1161
    ---Type <return> to continue, or q <return> to quit---
    #6840 0x400f8a36 in sre_search (state=0xbfffcaf0, pattern=0x411269ee)
    at Modules/_sre.c:1411
    #6841 0x400f63bf in pattern_search (self=0x411269c0, args=0x40e785ac, kw=0x0)
    at Modules/_sre.c:1869
    #6842 0x40068e74 in PyCFunction_Call (func=0x410610cc, arg=0x40e785ac, kw=0x0)
    at Objects/methodobject.c:93

    #6843 0x400a29e6 in call_function (pp_stack=0xbfffcf6c, oparg=6840)
    at Python/ceval.c:3439
    #6844 0x400a4115 in eval_frame (f=0x84f2ddc) at Python/ceval.c:2116
    #6845 0x400a181f in PyEval_EvalCodeEx (co=0x40429c60, globals=0x0, locals=0x0,
    args=0x84f2ddc, argcount=2, kws=0x84d3468, kwcount=0, defs=0x40436c18,
    defcount=1, closure=0x0) at Python/ceval.c:2663
    #6846 0x400540d5 in function_call (func=0x4043ba74, arg=0x40f54c6c, kw=0x41134e84)
    at Objects/funcobject.c:504

    #6847 0x4003a0aa in PyObject_Call (func=0x41126a0c, arg=0x40f54c6c, kw=0x41134e84)
    at Objects/abstract.c:1755
    #6848 0x400a2cad in ext_do_call (func=0x4043ba74, pp_stack=0xbfffd13c,
    flags=1089817708, na=0, nk=0) at Python/ceval.c:3713
    #6849 0x400a3fbb in eval_frame (f=0x84fa644) at Python/ceval.c:2151
    #6850 0x400a181f in PyEval_EvalCodeEx (co=0x40461e20, globals=0x0, locals=0x0,
    args=0x84fa644, argcount=5, kws=0x8423550, kwcount=0, defs=0x0, defcount=0,
    closure=0x0) at Python/ceval.c:2663

    #6851 0x400a76b5 in fast_function (func=0x41126a0c, pp_stack=0xbfffd2ec, n=5, na=5,
    nk=20) at Python/ceval.c:3529
    #6852 0x400a2aa7 in call_function (pp_stack=0xbfffd2ec, oparg=6840)
    at Python/ceval.c:3458
    #6853 0x400a4115 in eval_frame (f=0x84233ec) at Python/ceval.c:2116
    #6854 0x400a773e in fast_function (func=0x41126a0c, pp_stack=0xbfffd41c, n=1, na=1,
    nk=138556396) at Python/ceval.c:3518
    #6855 0x400a2aa7 in call_function (pp_stack=0xbfffd41c, oparg=6840)
    ---Type <return> to continue, or q <return> to quit---
    at Python/ceval.c:3458

    #6856 0x400a4115 in eval_frame (f=0x84f328c) at Python/ceval.c:2116
    #6857 0x400a181f in PyEval_EvalCodeEx (co=0x40461c20, globals=0x0, locals=0x0,
    args=0x84f328c, argcount=2, kws=0x0, kwcount=0, defs=0x404a7a58, defcount=1,
    closure=0x0) at Python/ceval.c:2663
    #6858 0x4005401b in function_call (func=0x404a695c, arg=0x40f5462c, kw=0x0)
    at Objects/funcobject.c:504
    #6859 0x4003a0aa in PyObject_Call (func=0x41126a0c, arg=0x40f5462c, kw=0x0)
    at Objects/abstract.c:1755

    #6860 0x400448a4 in instancemethod_call (func=0x1ab8, arg=0x40f5462c, kw=0x0)
    at Objects/classobject.c:2433
    #6861 0x4003a0aa in PyObject_Call (func=0x41126a0c, arg=0x40e78fac, kw=0x0)
    at Objects/abstract.c:1755

    #6862 0x40082bc8 in slot_tp_call (self=0x40e78fac, args=0x40e78fac, kwds=0x0)
    at Objects/typeobject.c:4473
    #6863 0x4003a0aa in PyObject_Call (func=0x41126a0c, arg=0x40e78fac, kw=0x0)
    at Objects/abstract.c:1755
    #6864 0x400a7806 in do_call (func=0x4113c36c, pp_stack=0x40e78fac, na=-1, nk=6840)
    at Python/ceval.c:3644
    #6865 0x400a29ac in call_function (pp_stack=0xbfffd8cc, oparg=6840)
    at Python/ceval.c:3460

    #6866 0x400a4115 in eval_frame (f=0x84237fc) at Python/ceval.c:2116
    #6867 0x400a181f in PyEval_EvalCodeEx (co=0x404a41e0, globals=0x0, locals=0x0,
    args=0x84237fc, argcount=2, kws=0x0, kwcount=0, defs=0x0, defcount=0,
    closure=0x0) at Python/ceval.c:2663
    #6868 0x4005401b in function_call (func=0x404a6d14, arg=0x40f543ac, kw=0x0)
    at Objects/funcobject.c:504
    #6869 0x4003a0aa in PyObject_Call (func=0x41126a0c, arg=0x40f543ac, kw=0x0)
    at Objects/abstract.c:1755
    #6870 0x400448a4 in instancemethod_call (func=0x1ab8, arg=0x40f543ac, kw=0x0)
    ---Type <return> to continue, or q <return> to quit---
    at Objects/classobject.c:2433

    #6871 0x4003a0aa in PyObject_Call (func=0x41126a0c, arg=0x40e78f8c, kw=0x0)
    at Objects/abstract.c:1755
    #6872 0x40082bc8 in slot_tp_call (self=0x40e78f8c, args=0x40e78f8c, kwds=0x0)
    at Objects/typeobject.c:4473
    #6873 0x4003a0aa in PyObject_Call (func=0x41126a0c, arg=0x40e78f8c, kw=0x0)
    at Objects/abstract.c:1755
    #6874 0x400a7806 in do_call (func=0x4113ca8c, pp_stack=0x40e78f8c, na=-1, nk=6840)
    at Python/ceval.c:3644

    #6875 0x400a29ac in call_function (pp_stack=0xbfffdd7c, oparg=6840)
    at Python/ceval.c:3460
    #6876 0x400a4115 in eval_frame (f=0x8364c9c) at Python/ceval.c:2116
    #6877 0x400a181f in PyEval_EvalCodeEx (co=0x404a41e0, globals=0x0, locals=0x0,
    args=0x8364c9c, argcount=2, kws=0x0, kwcount=0, defs=0x0, defcount=0,
    closure=0x0) at Python/ceval.c:2663
    #6878 0x4005401b in function_call (func=0x404a6d14, arg=0x40f5460c, kw=0x0)
    at Objects/funcobject.c:504
    #6879 0x4003a0aa in PyObject_Call (func=0x41126a0c, arg=0x40f5460c, kw=0x0)
    at Objects/abstract.c:1755
    #6880 0x400448a4 in instancemethod_call (func=0x1ab8, arg=0x40f5460c, kw=0x0)
    at Objects/classobject.c:2433
    #6881 0x4003a0aa in PyObject_Call (func=0x41126a0c, arg=0x40e7854c, kw=0x0)
    at Objects/abstract.c:1755
    #6882 0x40082bc8 in slot_tp_call (self=0x40e7854c, args=0x40e7854c, kwds=0x0)
    at Objects/typeobject.c:4473
    #6883 0x4003a0aa in PyObject_Call (func=0x41126a0c, arg=0x40e7854c, kw=0x0)
    at Objects/abstract.c:1755
    #6884 0x400a7806 in do_call (func=0x410e6f8c, pp_stack=0x40e7854c, na=-1, nk=6840)
    at Python/ceval.c:3644
    #6885 0x400a29ac in call_function (pp_stack=0xbfffe22c, oparg=6840)
    ---Type <return> to continue, or q <return> to quit---
    at Python/ceval.c:3460
    #6886 0x400a4115 in eval_frame (f=0x829a7a4) at Python/ceval.c:2116
    #6887 0x400a773e in fast_function (func=0x41126a0c, pp_stack=0xbfffe35c, n=2, na=2,
    nk=136947620) at Python/ceval.c:3518
    #6888 0x400a2aa7 in call_function (pp_stack=0xbfffe35c, oparg=6840)
    at Python/ceval.c:3458
    #6889 0x400a4115 in eval_frame (f=0x41b3a014) at Python/ceval.c:2116
    #6890 0x400a181f in PyEval_EvalCodeEx (co=0x40461420, globals=0x0, locals=0x0,
    args=0x41b3a014, argcount=2, kws=0x84376fc, kwcount=0, defs=0x4049e938,
    defcount=1, closure=0x0) at Python/ceval.c:2663
    #6891 0x400a76b5 in fast_function (func=0x41126a0c, pp_stack=0xbfffe50c, n=2, na=2,
    nk=8) at Python/ceval.c:3529
    #6892 0x400a2aa7 in call_function (pp_stack=0xbfffe50c, oparg=6840)
    at Python/ceval.c:3458
    #6893 0x400a4115 in eval_frame (f=0x8437594) at Python/ceval.c:2116
    #6894 0x400a181f in PyEval_EvalCodeEx (co=0x40461460, globals=0x0, locals=0x0,
    args=0x8437594, argcount=1, kws=0x814b440, kwcount=0, defs=0x0, defcount=0,
    closure=0x0) at Python/ceval.c:2663
    #6895 0x400a76b5 in fast_function (func=0x41126a0c, pp_stack=0xbfffe6bc, n=1, na=1,
    nk=4) at Python/ceval.c:3529
    #6896 0x400a2aa7 in call_function (pp_stack=0xbfffe6bc, oparg=6840)
    at Python/ceval.c:3458
    #6897 0x400a4115 in eval_frame (f=0x814b2ec) at Python/ceval.c:2116
    #6898 0x400a773e in fast_function (func=0x41126a0c, pp_stack=0xbfffe7ec, n=0, na=0,
    nk=135574252) at Python/ceval.c:3518
    #6899 0x400a2aa7 in call_function (pp_stack=0xbfffe7ec, oparg=6840)
    at Python/ceval.c:3458
    #6900 0x400a4115 in eval_frame (f=0x831ca34) at Python/ceval.c:2116
    #6901 0x400a181f in PyEval_EvalCodeEx (co=0x4044e5a0, globals=0x0, locals=0x0,
    args=0x831ca34, argcount=5, kws=0x80d6870, kwcount=0, defs=0x40459938,
    ---Type <return> to continue, or q <return> to quit---
    defcount=1, closure=0x0) at Python/ceval.c:2663
    #6902 0x400a76b5 in fast_function (func=0x41126a0c, pp_stack=0xbfffe99c, n=5, na=5,
    nk=20) at Python/ceval.c:3529
    #6903 0x400a2aa7 in call_function (pp_stack=0xbfffe99c, oparg=6840)
    at Python/ceval.c:3458
    #6904 0x400a4115 in eval_frame (f=0x80d6654) at Python/ceval.c:2116
    #6905 0x400a181f in PyEval_EvalCodeEx (co=0x4044e520, globals=0x0, locals=0x0,
    args=0x80d6654, argcount=0, kws=0x80baac4, kwcount=0, defs=0x4045e428,
    defcount=11, closure=0x0) at Python/ceval.c:2663
    #6906 0x400a76b5 in fast_function (func=0x41126a0c, pp_stack=0xbfffeb4c, n=0, na=0,
    nk=0) at Python/ceval.c:3529
    #6907 0x400a2aa7 in call_function (pp_stack=0xbfffeb4c, oparg=6840)
    at Python/ceval.c:3458
    #6908 0x400a4115 in eval_frame (f=0x80ba974) at Python/ceval.c:2116
    #6909 0x400a181f in PyEval_EvalCodeEx (co=0x4044e8a0, globals=0x0,
    locals=0x403fe79c, args=0x80ba974, argcount=0, kws=0x0, kwcount=0, defs=0x0,
    defcount=0, closure=0x0) at Python/ceval.c:2663
    #6910 0x400a1152 in PyEval_EvalCode (co=0x4044e8a0, globals=0x403fe79c,
    locals=0x403fe79c) at Python/ceval.c:537
    #6911 0x400dcc2d in run_err_node (n=0x403fe79c,
    filename=0xbfffefad "./Lib/test/regrtest.py", globals=0x403fe79c,
    locals=0x403fe79c, flags=0xbfffecf8) at Python/pythonrun.c:1265
    #6912 0x400db668 in PyRun_SimpleFileExFlags (fp=0x8049a00,
    filename=0xbfffefad "./Lib/test/regrtest.py", closeit=1, flags=0xbfffecf8)
    at Python/pythonrun.c:862
    #6913 0x400daf9e in PyRun_AnyFileExFlags (fp=0x8049a00,
    filename=0xbfffefad "./Lib/test/regrtest.py", closeit=1, flags=0xbfffecf8)
    at Python/pythonrun.c:659
    #6914 0x400e3994 in Py_Main (argc=2, argv=0xbfffed74) at Modules/main.c:415
    #6915 0x080486f3 in main (argc=5, argv=0xbfffed74) at Modules/ccpython.cc:10
    Current language: auto; currently c
    (gdb)

    Some kind of "infinite" recursion?

    Regards

    Berthold
    --
    / http://starship.python.net/crew/bhoel/
    It is unlawful to use this email address for unsolicited ads
    (USC Title 47 Sec.227). I will assess a US$500 charge for
    reviewing and deleting each unsolicited ad.
     
    Berthold Hoellmann, Dec 28, 2003
    #1
    1. Advertising

  2. (Berthold Hoellmann) writes:

    > Hello,
    >
    > When I use
    >
    > ./configure --with-thread --with-fpectl --with-signal-module \
    > --with-pymalloc --enable-shared --with-cxx=g++
    >
    > make test
    >
    > on 2.3.3 I get
    >
    > ...
    > test_queue
    > test_quopri
    > test_random
    > test_re
    > make: *** [test] Speicherzugriffsfehler (Speicherauszug erstellt)


    Everything works fine if I remove the "--enable-shared" flag from
    configure.

    Regards

    Berthold

    --
    / http://starship.python.net/crew/bhoel/
    It is unlawful to use this email address for unsolicited ads
    (USC Title 47 Sec.227). I will assess a US$500 charge for
    reviewing and deleting each unsolicited ad.
     
    Berthold Hoellmann, Dec 28, 2003
    #2
    1. Advertising

  3. (Berthold Hoellmann) writes:

    > Hello,
    >
    > When I use
    >
    > ./configure --with-thread --with-fpectl --with-signal-module \
    > --with-pymalloc --enable-shared --with-cxx=g++
    >
    > make test


    What platform, compiler, and versions thereof?

    >
    > on 2.3.3 I get
    >
    > ...
    > test_queue
    > test_quopri
    > test_random
    > test_re
    > make: *** [test] Speicherzugriffsfehler (Speicherauszug erstellt)
    >
    > gdb ./python core.29964
    > GNU gdb 5.3.93
    > Copyright 2003 Free Software Foundation, Inc.
    > ...
    > Reading symbols from /home/devel/compile/Python-2.3.3/build/lib.linux-i686-2.3/pyexpat.so...done.
    > Loaded symbols for /home/devel/compile/Python-2.3.3/build/lib.linux-i686-2.3/pyexpat.so


    [snippety]

    > Some kind of "infinite" recursion?


    Well, there's a test in tes_sre that uses quite a lot of C stack. I
    guess it's possible that something about shared library code uses more
    stack. Can you run 'ulimit -s <something larger>' and try again?

    Cheers,
    mwh

    --
    Lisp does badly because we refuse to lie. When people ask us if
    we can solve insoluble problems we say that we can't, and because
    they expect us to lie to them, they find some other language
    where the truth is less respected. -- Tim Bradshaw, comp.lang.lisp
     
    Michael Hudson, Dec 28, 2003
    #3
  4. Michael Hudson <> writes:

    > (Berthold Hoellmann) writes:
    >
    >> Hello,
    >>
    >> When I use
    >>
    >> ./configure --with-thread --with-fpectl --with-signal-module \
    >> --with-pymalloc --enable-shared --with-cxx=g++
    >>
    >> make test

    >
    > What platform, compiler, and versions thereof?


    Sorry, of course:

    SuSE Linux 9.0,
    gcc (GCC) 3.3.1 (SuSE Linux)

    I always forget about my *FLAGS:

    CXXFLAGS=-O3 -fstrict-aliasing -funroll-loops -fschedule-insns2 \
    -ffast-math -fno-math-errno -funsafe-math-optimizations \
    -fno-trapping-math -march=athlon-xp -mfpmath=sse -msse -mmmx -m3dnow

    CFLAGS=-O3 -fstrict-aliasing -funroll-loops -fschedule-insns2 \
    -ffast-math -fno-math-errno -funsafe-math-optimizations \
    -fno-trapping-math -march=athlon-xp -mfpmath=sse -msse -mmmx -m3dnow

    (My Processor is a AMD Athlon(tm) XP 2000+, 512MB RAM)

    > ...
    > Well, there's a test in tes_sre that uses quite a lot of C stack. I
    > guess it's possible that something about shared library code uses more
    > stack. Can you run 'ulimit -s <something larger>' and try again?
    >


    Well, stacksize is unlimited (somehow)

    >limit

    cputime unlimited
    filesize unlimited
    datasize unlimited
    stacksize unlimited
    coredumpsize unlimited
    memoryuse unlimited
    vmemoryuse unlimited
    descriptors 1024
    memorylocked unlimited
    maxproc 4095

    Regards
    Berthold
    --
    / http://starship.python.net/crew/bhoel/
    It is unlawful to use this email address for unsolicited ads
    (USC Title 47 Sec.227). I will assess a US$500 charge for
    reviewing and deleting each unsolicited ad.
     
    Berthold Hoellmann, Dec 28, 2003
    #4
  5. On Sun, 28 Dec 2003, Berthold Hoellmann wrote:

    > > When I use
    > >
    > > ./configure --with-thread --with-fpectl --with-signal-module \
    > > --with-pymalloc --enable-shared --with-cxx=g++
    > >
    > > make test
    > >
    > > on 2.3.3 I get
    > >
    > > ...
    > > test_queue
    > > test_quopri
    > > test_random
    > > test_re
    > > make: *** [test] Speicherzugriffsfehler (Speicherauszug erstellt)

    >
    > Everything works fine if I remove the "--enable-shared" flag from
    > configure.


    You don't mention what platform you are seeing this on. The output of a
    verbose test run (python Lib/test/regrtest.py -v test_re), preferably with
    error messages translated to English, may help diagnose the issue.

    I know that there are platforms where the amount of stack space available
    to a threaded process is not easily controlled, and recent versions of gcc
    are creating much larger stack frames than earlier versions. The sre
    module in 2.3.x (and earlier) is recursive and thus sensitive to stack
    space availability. A core dump is a likely indicator of this. Read the
    Modules/_sre.c source file for more info. sre in 2.4 will be
    significantly improved in this regard.

    FYI, since 2.3 PyMalloc is a default option so you don't need
    --with-pymalloc, and most recent Linux and BSD systems will default to
    building with --with-thread and --with-signal-module.

    --
    Andrew I MacIntyre "These thoughts are mine alone..."
    E-mail: (pref) | Snail: PO Box 370
    (alt) | Belconnen ACT 2616
    Web: http://www.andymac.org/ | Australia
     
    Andrew MacIntyre, Dec 28, 2003
    #5
  6. On Sun, 28 Dec 2003, Berthold Hoellmann wrote:

    > CXXFLAGS=-O3 -fstrict-aliasing -funroll-loops -fschedule-insns2 \
    > -ffast-math -fno-math-errno -funsafe-math-optimizations \
    > -fno-trapping-math -march=athlon-xp -mfpmath=sse -msse -mmmx -m3dnow
    >
    > CFLAGS=-O3 -fstrict-aliasing -funroll-loops -fschedule-insns2 \
    > -ffast-math -fno-math-errno -funsafe-math-optimizations \
    > -fno-trapping-math -march=athlon-xp -mfpmath=sse -msse -mmmx -m3dnow


    Build Python with the configure defaults before you start getting creative
    with optimisation.

    Problems resulting from trying to use more aggressive optimisations than
    the default should be taken up with the compiler vendor.

    --
    Andrew I MacIntyre "These thoughts are mine alone..."
    E-mail: (pref) | Snail: PO Box 370
    (alt) | Belconnen ACT 2616
    Web: http://www.andymac.org/ | Australia
     
    Andrew MacIntyre, Dec 29, 2003
    #6
  7. Andrew MacIntyre <> writes:

    > On Sun, 28 Dec 2003, Berthold Hoellmann wrote:
    >
    >> CXXFLAGS=-O3 -fstrict-aliasing -funroll-loops -fschedule-insns2 \
    >> -ffast-math -fno-math-errno -funsafe-math-optimizations \
    >> -fno-trapping-math -march=athlon-xp -mfpmath=sse -msse -mmmx -m3dnow
    >>
    >> CFLAGS=-O3 -fstrict-aliasing -funroll-loops -fschedule-insns2 \
    >> -ffast-math -fno-math-errno -funsafe-math-optimizations \
    >> -fno-trapping-math -march=athlon-xp -mfpmath=sse -msse -mmmx -m3dnow

    >
    > Build Python with the configure defaults before you start getting creative
    > with optimisation.
    >
    > Problems resulting from trying to use more aggressive optimisations than
    > the default should be taken up with the compiler vendor.


    Hello,

    make distclean
    env CFLAGS= CXXFLAGS= ./configure --with-thread --with-fpectl \
    --with-signal-module --with-pymalloc --enable-shared --with-cxx=g++
    env LANG=C CFLAGS= CXXFLAGS= make test

    gave:

    test_quopri
    test_random
    test_re
    make: *** [test] Segmentation fault

    aggain.

    Regards

    Berthold
    --
    / http://starship.python.net/crew/bhoel/
    It is unlawful to use this email address for unsolicited ads
    (USC Title 47 Sec.227). I will assess a US$500 charge for
    reviewing and deleting each unsolicited ad.
     
    Berthold Höllmann, Dec 29, 2003
    #7
  8. Andrew MacIntyre <> writes:

    > On Sun, 28 Dec 2003, Berthold Hoellmann wrote:
    >
    >> > When I use
    >> >
    >> > ./configure --with-thread --with-fpectl --with-signal-module \
    >> > --with-pymalloc --enable-shared --with-cxx=g++
    >> >
    >> > make test
    >> >
    >> > on 2.3.3 I get
    >> >
    >> > ...
    >> > test_queue
    >> > test_quopri
    >> > test_random
    >> > test_re
    >> > make: *** [test] Speicherzugriffsfehler (Speicherauszug erstellt)


    make: *** [test] Segmentation fault (core dumped)

    >>
    >> Everything works fine if I remove the "--enable-shared" flag from
    >> configure.

    >
    > You don't mention what platform you are seeing this on. The output of a
    > verbose test run (python Lib/test/regrtest.py -v test_re), preferably with
    > error messages translated to English, may help diagnose the issue.


    env LANG=C LD_LIBRARY_PATH=/home/devel/compile/Python-2.3.3:/usr/local/v/lib:/usr/lib/qt3/lib:/usr/local/pgsql/lib:/usr/teTeX/lib./python Lib/test/regrtest.py -v test_re

    (LD_LIBRARY_PATH is taken from the "make test" output) gives:

    test_re
    test_anyall (test.test_re.ReTests) ... ok
    test_basic_re_sub (test.test_re.ReTests) ... ok
    test_bigcharset (test.test_re.ReTests) ... ok
    test_bug_113254 (test.test_re.ReTests) ... ok
    test_bug_114660 (test.test_re.ReTests) ... ok
    test_bug_117612 (test.test_re.ReTests) ... ok
    test_bug_418626 (test.test_re.ReTests) ... ok
    test_bug_448951 (test.test_re.ReTests) ... ok
    test_bug_449000 (test.test_re.ReTests) ... ok
    test_bug_449964 (test.test_re.ReTests) ... ok
    test_bug_462270 (test.test_re.ReTests) ... ok
    test_bug_527371 (test.test_re.ReTests) ... ok
    test_bug_545855 (test.test_re.ReTests) ... ok
    test_bug_612074 (test.test_re.ReTests) ... ok
    test_bug_725106 (test.test_re.ReTests) ... ok
    test_bug_725149 (test.test_re.ReTests) ... ok
    test_bug_764548 (test.test_re.ReTests) ... ok
    test_category (test.test_re.ReTests) ... ok
    test_constants (test.test_re.ReTests) ... ok
    test_expand (test.test_re.ReTests) ... ok
    test_finditer (test.test_re.ReTests) ... ok
    test_flags (test.test_re.ReTests) ... ok
    test_getattr (test.test_re.ReTests) ... ok
    test_getlower (test.test_re.ReTests) ... ok
    test_groupdict (test.test_re.ReTests) ... ok
    test_ignore_case (test.test_re.ReTests) ... ok
    test_non_consuming (test.test_re.ReTests) ... ok
    test_not_literal (test.test_re.ReTests) ... ok
    test_pickling (test.test_re.ReTests) ... ok
    test_qualified_re_split (test.test_re.ReTests) ... ok
    test_qualified_re_sub (test.test_re.ReTests) ... ok
    test_re_escape (test.test_re.ReTests) ... ok
    test_re_findall (test.test_re.ReTests) ... ok
    test_re_groupref (test.test_re.ReTests) ... ok
    test_re_groupref_exists (test.test_re.ReTests) ... ok
    test_re_match (test.test_re.ReTests) ... ok
    test_re_split (test.test_re.ReTests) ... ok
    test_re_subn (test.test_re.ReTests) ... ok
    test_repeat_minmax (test.test_re.ReTests) ... ok
    test_scanner (test.test_re.ReTests) ... ok
    test_search_coverage (test.test_re.ReTests) ... ok
    test_search_star_plus (test.test_re.ReTests) ... ok
    test_special_escapes (test.test_re.ReTests) ... ok
    test_sre_character_literals (test.test_re.ReTests) ... ok
    test_stack_overflow (test.test_re.ReTests) ... ok
    test_symbolic_refs (test.test_re.ReTests) ... ok

    ----------------------------------------------------------------------
    Ran 46 tests in 0.401s

    OK
    Running re_tests test suite
    1 test OK.
    CAUTION: stdout isn't compared in verbose mode:
    a test that passes in verbose mode may fail without it.

    so it seems that the other modules loaded in the "make test" run
    decrease the avaliable stack so that the test does not succeed but
    calling the test alone finds enough stack space avaliable?

    > I know that there are platforms where the amount of stack space available
    > to a threaded process is not easily controlled, and recent versions of gcc
    > are creating much larger stack frames than earlier versions. The sre
    > module in 2.3.x (and earlier) is recursive and thus sensitive to stack
    > space availability. A core dump is a likely indicator of this. Read the
    > Modules/_sre.c source file for more info. sre in 2.4 will be
    > significantly improved in this regard.
    >
    > FYI, since 2.3 PyMalloc is a default option so you don't need
    > --with-pymalloc, and most recent Linux and BSD systems will default to
    > building with --with-thread and --with-signal-module.


    I really should know, but I usually keep "good" configure settings for
    reuse and normally don't think much about the settings when everything
    works.

    Regards

    Berthold
    --
    / http://starship.python.net/crew/bhoel/
    It is unlawful to use this email address for unsolicited ads
    (USC Title 47 Sec.227). I will assess a US$500 charge for
    reviewing and deleting each unsolicited ad.
     
    Berthold Höllmann, Dec 29, 2003
    #8
  9. On Mon, 29 Dec 2003, Berthold Höllmann wrote:

    > make distclean
    > env CFLAGS= CXXFLAGS= ./configure --with-thread --with-fpectl \
    > --with-signal-module --with-pymalloc --enable-shared --with-cxx=g++
    > env LANG=C CFLAGS= CXXFLAGS= make test
    >
    > gave:
    >
    > test_quopri
    > test_random
    > test_re
    > make: *** [test] Segmentation fault


    1. Just to confirm that the configure generated Makefile is what it
    should be, what are the OPT and BASECFLAGS values?

    2. Do a verbose run of the test_re test with the interpreter you built
    above (ie with the default optimisations):
    ./python Lib/test/regrtest.py -v test_re >test_re.log 2>&1

    3. If it coredumps (which I expect it to based on your advice above), use
    gdb to get the backtrace.

    The test log and the gdb backtrace should help isolate where things are
    going astray.

    BTW, I'm not sure why you're explicitly using --with-thread,
    --with-signal-module and --with-pymalloc as they are all enabled by
    default. I'm also unsure as to whether you really need to specify
    --with-cxx; it might be worth a try configure'ing without it.

    --
    Andrew I MacIntyre "These thoughts are mine alone..."
    E-mail: (pref) | Snail: PO Box 370
    (alt) | Belconnen ACT 2616
    Web: http://www.andymac.org/ | Australia
     
    Andrew MacIntyre, Dec 30, 2003
    #9
  10. On Mon, 29 Dec 2003, Berthold Höllmann wrote:

    > so it seems that the other modules loaded in the "make test" run
    > decrease the avaliable stack so that the test does not succeed but
    > calling the test alone finds enough stack space avaliable?


    The above message crossed with my other reply.

    This looks very much like the stack problem I referred to, which I am
    familiar with on FreeBSD. I have been able to build with
    --enable-shared on FreeBSD (after some configure tweaking), and see a
    similar effect - ie test_re coredumps with --enable-shared and passes
    without (and this with gcc 2.95.4). From the backtrace, I see that a
    USE_RECURSION_LIMIT of 7200 would have worked, compared to the 7500 that
    works for the standard build.

    That it passes the individual test but not the full test suite for you
    suggests that your build is right on the stacksize limit, and lowering
    USE_RECURSION_LIMIT by only a small amount would get you through.

    Note that there are some subtleties here - even though your ulimit says
    unlimited stackspace, in the presence of threads this may not hold.
    It certainly doesn't on FreeBSD 4.x where there the pthreads support has a
    hardcoded 1MB stack for the primary thread (which is what is running the
    regression test and is distinct from the stack created for each
    instantiated thread). The LinuxThreads port on FreeBSD seems not to have
    the same restriction. Exactly what is happening on your Linux system I
    don't know; you would have to do some research to find out.

    Depending on how you wish to proceed, you have a couple of options:
    - build normally, then blow away Modules/_sre.o and recompile
    Modules/_sre.c with -Os (just temporarily doctor the Makefile).
    From what I've seen -Os cuts the stack consumption enough to get
    you through the full test suite, and doesn't seriously impact sre's
    performance.
    - modify the USE_RECURSION_LIMIT macro in Modules/_sre.c to a lower value
    and rebuild. Linux would be using the default definition of 10000 at
    line 98 (for 2.3.3).
    - find a way to change the default stack size for the primary thread.

    If you play with the second or third options, a report to the Python bug
    tracker on SF with the conclusion you reach would be useful. Note that
    such a bug report should include details of OS & compiler, so that
    suitable #ifdef'ery can be written, and should be based on Python's
    default compiler switches. As I noted previously, sre in Python 2.4 will
    not have this issue, but Anthony Baxter has said that he expects to do a
    2.3.4 release (April/May O4 as I recall).

    --
    Andrew I MacIntyre "These thoughts are mine alone..."
    E-mail: (pref) | Snail: PO Box 370
    (alt) | Belconnen ACT 2616
    Web: http://www.andymac.org/ | Australia
     
    Andrew MacIntyre, Dec 30, 2003
    #10
  11. Andrew MacIntyre <> writes:

    > On Mon, 29 Dec 2003, Berthold Höllmann wrote:
    >
    >> so it seems that the other modules loaded in the "make test" run
    >> decrease the avaliable stack so that the test does not succeed but
    >> calling the test alone finds enough stack space avaliable?

    ....
    > Note that there are some subtleties here - even though your ulimit says
    > unlimited stackspace, in the presence of threads this may not hold.
    > It certainly doesn't on FreeBSD 4.x where there the pthreads support has a
    > hardcoded 1MB stack for the primary thread (which is what is running the


    I found a document stating it is 2MB for each thread on Linux using Google

    > regression test and is distinct from the stack created for each
    > instantiated thread). The LinuxThreads port on FreeBSD seems not to have
    > the same restriction. Exactly what is happening on your Linux system I
    > don't know; you would have to do some research to find out.
    >
    > Depending on how you wish to proceed, you have a couple of options:
    > - build normally, then blow away Modules/_sre.o and recompile
    > Modules/_sre.c with -Os (just temporarily doctor the Makefile).
    > From what I've seen -Os cuts the stack consumption enough to get
    > you through the full test suite, and doesn't seriously impact sre's
    > performance.


    This does work for me

    > - modify the USE_RECURSION_LIMIT macro in Modules/_sre.c to a lower value
    > and rebuild. Linux would be using the default definition of 10000 at
    > line 98 (for 2.3.3).


    #define USE_RECURSION_LIMIT 6835

    does succeed,

    #define USE_RECURSION_LIMIT 6840

    gives segfault for me. Will submit bugreport.

    > - find a way to change the default stack size for the primary thread.


    To be honest: I don't have an idea where to look at for this.

    Regards

    Berthold
    --
    / http://starship.python.net/crew/bhoel/
    It is unlawful to use this email address for unsolicited ads
    (USC Title 47 Sec.227). I will assess a US$500 charge for
    reviewing and deleting each unsolicited ad.
     
    Berthold Höllmann, Jan 3, 2004
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Douglass Turner
    Replies:
    2
    Views:
    2,100
    Manfred Bartz
    Sep 4, 2003
  2. Alex Stapleton

    smtplib Segfaults Python under Debian

    Alex Stapleton, Mar 1, 2005, in forum: Python
    Replies:
    1
    Views:
    416
    Cousin Stanley
    Mar 1, 2005
  3. Mark Asbach
    Replies:
    1
    Views:
    491
    Leo Kislov
    Nov 3, 2006
  4. Riccardo Di Meo
    Replies:
    0
    Views:
    358
    Riccardo Di Meo
    Aug 2, 2008
  5. Marten Lehmann
    Replies:
    3
    Views:
    4,291
    Marten Lehmann
    Sep 16, 2010
Loading...

Share This Page