Compiling Guppy-PE extension modules

Discussion in 'Python' started by Sverker Nilsson, Nov 29, 2005.

  1. I have been informed that Guppy-PE (http://guppy-pe.sourceforge.net)
    has failed to compile its extension modules with a Microsoft .NET 2003
    compiler under Windows 2000.

    [To the person who informed me about this in an email per 27 Nov:
    Thanks for your message.
    I couldn't reply to you because your email address bounced my
    mail.]

    One of the problems, seems to be string constants in the C source that
    contain newlines. I am using GCC on Linux so, I missed this with the
    standard warning options. Compiling with -pedantic reveals 'a lot' of
    places where this is a problem.

    I could fix this but it takes some work and it makes the source code
    less readable so I was wondering ...

    Is this a common problem? Or maybe it is just the compiler version
    mentioned that doesn't handle it?

    Does someone know of some switch to enable the Microsoft compiler to
    accept strings with newlines?

    If so, the setup could perhaps be made to pass this switch to the
    compiler when building under Windows.

    Or else,..

    somebody knows about a script to convert the files to the format the
    Microsoft compiler wants? I guess the build compilation could then be
    setup to pipe them through this or otherwise convert the files as
    needed.

    Otherwise, I guess I'll just have to hand-hack around this problem,
    write a conversion script, or whatever. I will see.

    Sorry for any inconvenience so far.

    Regards,

    Sverker Nilsson

    PS. I know it's not ANSI-correct but why do we have to work to make
    our source codes less clear?
     
    Sverker Nilsson, Nov 29, 2005
    #1
    1. Advertising

  2. "Sverker Nilsson" <> schrieb im Newsbeitrag
    news:...
    > I have been informed that Guppy-PE (http://guppy-pe.sourceforge.net)
    > has failed to compile its extension modules with a Microsoft .NET 2003
    > compiler under Windows 2000.
    >
    > [To the person who informed me about this in an email per 27 Nov:
    > Thanks for your message.
    > I couldn't reply to you because your email address bounced my
    > mail.]
    >
    > One of the problems, seems to be string constants in the C source that
    > contain newlines. I am using GCC on Linux so, I missed this with the
    > standard warning options. Compiling with -pedantic reveals 'a lot' of
    > places where this is a problem.
    >
    > I could fix this but it takes some work and it makes the source code
    > less readable so I was wondering ...
    >
    > Is this a common problem? Or maybe it is just the compiler version
    > mentioned that doesn't handle it?
    >
    > Does someone know of some switch to enable the Microsoft compiler to
    > accept strings with newlines?
    >
    > If so, the setup could perhaps be made to pass this switch to the
    > compiler when building under Windows.
    >
    > Or else,..
    >
    > somebody knows about a script to convert the files to the format the
    > Microsoft compiler wants? I guess the build compilation could then be
    > setup to pipe them through this or otherwise convert the files as
    > needed.
    >
    > Otherwise, I guess I'll just have to hand-hack around this problem,
    > write a conversion script, or whatever. I will see.
    >
    > Sorry for any inconvenience so far.
    >
    > Regards,
    >
    > Sverker Nilsson
    >
    > PS. I know it's not ANSI-correct but why do we have to work to make
    > our source codes less clear?
    >


    Thank you for posting to comp.lang.python .

    Beside the problem with the multiline strings in sets.c I was getting also:

    src\sets\sets.c(70) : error C2099: initializer is not a constant
    src\sets\sets.c(71) : error C2099: initializer is not a constant
    src\sets\sets.c(71) : warning C4028: formal parameter 1 different from
    declaration
    src\sets\sets.c(72) : error C2099: initializer is not a constant
    src\sets\sets.c(72) : warning C4047: 'initializing' : 'PyTypeObject *'
    differs in levels of indirection from 'int (__cdecl *)(PyObject *)'
    src\sets\sets.c(72) : warning C4028: formal parameter 1 different from
    declaration
    error: command 'E:\VisualC++NET2003\Vc7\bin\cl.exe'
    failed with exit status 2

    Anyone here who succeeded to compile and use this module with
    Python 2.4.2 on Windows 2000/XP? Please share your experience.

    Claudio
     
    Claudio Grondi, Nov 29, 2005
    #2
    1. Advertising

  3. Sverker Nilsson

    Dave Hansen Guest

    On 29 Nov 2005 11:53:31 -0800 in comp.lang.python, "Sverker Nilsson"
    <> wrote:

    [...]
    >One of the problems, seems to be string constants in the C source that
    >contain newlines. I am using GCC on Linux so, I missed this with the


    I'm not sure what you mean by this. Something like

    char long_string[] = " A long string that
    extends to subsequent lines
    before closing the quote? ";

    >standard warning options. Compiling with -pedantic reveals 'a lot' of
    >places where this is a problem.
    >
    >I could fix this but it takes some work and it makes the source code
    >less readable so I was wondering ...


    Eye of the beholder, I guess.

    You were aware that C supports concatenation of string literals
    separated only by whitespace? For example, if the problem is similar
    to the long_string example above, and it does what I think it does,
    the following code should be equivalent and will compile on any good C
    compiler (even Microsoft's).

    char long_string[] = " A long string that\n"
    " extends to subsequent lines\n"
    " before closing the quote? ";

    Not that much worse, and you could probably write a Python script to
    automate the changes. If a quote opens but doesn't close on a line,
    append a newline and close the quote, then prepend an opening quote on
    the next line. The only tricky part is making sure the opening quote
    isn't inside a comment or character literal.

    >
    >Is this a common problem? Or maybe it is just the compiler version
    >mentioned that doesn't handle it?


    I wasn't even aware GCC supported such an extension. Certainly no
    other C compiler I've used has anything like this.

    >
    >Does someone know of some switch to enable the Microsoft compiler to
    >accept strings with newlines?


    Not that I'm aware of. Sorry.

    Maybe one exists, but if so, I'd resist the temptation to use it. It
    might disappear in a later version of the compiler, or you might need
    to use some other compiler that doesn't support multiline strings, and
    you're back where you started.

    [...]

    >
    >PS. I know it's not ANSI-correct but why do we have to work to make
    >our source codes less clear?


    Hysterical raisins. Regards,
    -=Dave

    --
    Change is inevitable, progress is not.
     
    Dave Hansen, Nov 29, 2005
    #3
  4. "Claudio Grondi" <> wrote:

    > Beside the problem with the multiline strings in sets.c I was getting also:
    >
    > src\sets\sets.c(70) : error C2099: initializer is not a constant
    > src\sets\sets.c(71) : error C2099: initializer is not a constant
    > src\sets\sets.c(71) : warning C4028: formal parameter 1 different from
    > declaration
    > src\sets\sets.c(72) : error C2099: initializer is not a constant
    > src\sets\sets.c(72) : warning C4047: 'initializing' : 'PyTypeObject *'
    > differs in levels of indirection from 'int (__cdecl *)(PyObject *)'
    > src\sets\sets.c(72) : warning C4028: formal parameter 1 different from
    > declaration
    > error: command 'E:\VisualC++NET2003\Vc7\bin\cl.exe'
    > failed with exit status 2


    Some of those messages are misleading, confused me at least. Seems
    like a bug in the error reporting part of the compiler. Anyway, the
    code was not ANSI and what I got around that place was:

    sets.c:69: warning: ANSI forbids initialization between function
    pointer and `void *'
    sets.c:71: warning: ANSI forbids initialization between function
    pointer and `void *'

    There were a bunch of other problems too.

    I have made a new version now, 0.1.1 .
    It compiles cleanly with gcc -pedantic .

    When looking in the code I became worried that there may be other
    problems with Microsoft Windows since there is some magic with
    DL_IMPORT that may be necessary and that I think I added most of the
    time but I may have missed some. I have no easy way to test with
    MSWindows currently and it will probably take a while before I get
    around to do it.

    The new version is on the home page:

    http://guppy-pe.sourceforge.net

    Regards,

    Sverker
     
    Sverker Nilsson, Dec 1, 2005
    #4
  5. > I have made a new version now, 0.1.1 .
    > It compiles cleanly with gcc -pedantic .


    but the problem with sets.c remains:

    C:\VisualC++NET2003\Vc7\bin\cl.exe /c /nologo /Ox /MD /W3 /G7 /GX
    /DNDEBUG -IE:\Python24\include -IE:\Python24\PC /Tcsrc/sets/sets.c
    /Fobuild\temp.win32-2.4\Re
    lease\src/sets/sets.obj
    sets.c
    src\sets\sets.c(68) : error C2099: initializer is not a constant
    src\sets\sets.c(68) : warning C4047: 'initializing' : 'PyTypeObject *'
    differs in levels of indirection from 'NyHeapDef_SizeGetter'
    src\sets\sets.c(69) : error C2099: initializer is not a constant
    src\sets\sets.c(69) : warning C4028: formal parameter 1 different from
    declaration
    src\sets\sets.c(70) : error C2099: initializer is not a constant
    src\sets\sets.c(70) : warning C4047: 'initializing' : 'PyTypeObject *'
    differs in levels of indirection from 'int (__cdecl *)(PyObject *)'
    src\sets\sets.c(70) : warning C4028: formal parameter 1 different from
    declaration
    src\sets\sets.c(70) : warning C4028: formal parameter 1 different from
    declaration
    error: command 'E:\VisualC++NET2003\Vc7\bin\cl.exe' failed with exit status
    2

    The MSDN help gives a simple example of code raising compiler error C2099:
    // C2099.c
    int j;
    int *p;
    j = *p; // C2099, *p is not a constant

    The example code shows to me, that there is a good reason compiler
    generates an error in that case.
    j = *p; leads to an assignment of a random (and therefore maybe leading to a
    non deterministic crash of the executable during runtime) value to a
    variable, what can't be intended in a deterministic program.

    Hope this helps.

    Claudio
     
    Claudio Grondi, Dec 2, 2005
    #5
  6. "Claudio Grondi" <> wrote:

    > but the problem with sets.c remains:
    >
    > C:\VisualC++NET2003\Vc7\bin\cl.exe /c /nologo /Ox /MD /W3 /G7 /GX
    > /DNDEBUG -IE:\Python24\include -IE:\Python24\PC /Tcsrc/sets/sets.c
    > /Fobuild\temp.win32-2.4\Re
    > lease\src/sets/sets.obj
    > sets.c
    > src\sets\sets.c(68) : error C2099: initializer is not a constant
    > src\sets\sets.c(68) : warning C4047: 'initializing' : 'PyTypeObject *'
    > differs in levels of indirection from 'NyHeapDef_SizeGetter'
    > src\sets\sets.c(69) : error C2099: initializer is not a constant
    > src\sets\sets.c(69) : warning C4028: formal parameter 1 different from
    > declaration
    > src\sets\sets.c(70) : error C2099: initializer is not a constant
    > src\sets\sets.c(70) : warning C4047: 'initializing' : 'PyTypeObject *'
    > differs in levels of indirection from 'int (__cdecl *)(PyObject *)'
    > src\sets\sets.c(70) : warning C4028: formal parameter 1 different from
    > declaration
    > src\sets\sets.c(70) : warning C4028: formal parameter 1 different from
    > declaration
    > error: command 'E:\VisualC++NET2003\Vc7\bin\cl.exe' failed with exit status
    > 2
    >
    > The MSDN help gives a simple example of code raising compiler error C2099:
    > // C2099.c
    > int j;
    > int *p;
    > j = *p; // C2099, *p is not a constant
    >
    > The example code shows to me, that there is a good reason compiler
    > generates an error in that case.
    > j = *p; leads to an assignment of a random (and therefore maybe leading to a
    > non deterministic crash of the executable during runtime) value to a
    > variable, what can't be intended in a deterministic program.


    This is confusing, because the problem of the example code is not that
    *p is not constant but rather that p is undefined aka uninitialized.

    Humm..

    Compiling the following program:

    #include <stdio.h>
    int main(void) {
    int j;
    int *p;
    j = *p;
    printf("%d\n",j); /* Makes sure the optimizer doesnt remove the
    previous code */
    return 0;
    }

    with

    gcc -O3 -Wall main.c

    gives me:

    main.c: In function `main':
    main.c:4: warning: `p' might be used uninitialized in this function

    (And I get a Segmentation fault from running the executable.)

    The -Wall flag enables, umm, as it is saying 'all' warnings
    though perhaps not really all, and this flag + others is used
    by the compilation command generated by distutils when
    building with gcc.

    I don't see any warnings when building Guppy.

    So there should be no case (as easy to discover) as that in the
    example.

    So I am confused.

    I was beginning to wonder if we were talking about the same file/code.
    This code is from sets.c lines 66..71

    static NyHeapDef nysets_heapdefs[] = {
    {0, &NyMutBitSet_Type, (NyHeapDef_SizeGetter) mutbitset_indisize},
    {0, &NyCplBitSet_Type, 0, cplbitset_traverse},
    {0, &NyNodeSet_Type, nodeset_indisize, nodeset_traverse,
    nodeset_relate},
    {0}
    };

    I can't see how there can be problems with initializers not being
    constants here unless, perhaps if the compiler has a problem since
    those functions (mutbitset_indisize etc) and the types that are used
    as initializers are in separately compiled files. But gcc -Wall
    -pedantic handles it and doesn't warn. I don't have the ANSI standard
    but I have the 2nd edition of The C Programming Language by Kernighan
    and Ritchie (1988, 'Based on Draft-Proposed ANSI C'), and it is saying

    - quotes -

    A8.7 Initialization
    ....

    All the expressions in the initializer for a static object or array
    must be constant expressions as described in &A7.19
    ....

    A7.19 Constant Expressions

    Syntactically, a constant expression is an expression restricted to a
    subset of operators...
    ....

    More latitude is permitted for the constant expressions of
    initializers... Initializers must evaluate either to a constant or to
    the address of a previously declared external or static object plus or
    minus a constant.

    - end quotes -

    If this means the Microsoft .NET compiler is perhaps not compliant
    with standard/ANSI C then since it was from 2003 maybe they have come
    out with an update now that fixes this problem. I know, it may not
    really help you because it may cost money, I don't know how long one
    can get updates without paying. Alternatively, I think GCC is
    available for Microsoft Windows although I don't know how it handles
    the .NET architecture.

    For me, if the case is according to hypothesis, and I would have to
    rewrite the code to put all the functions and type objects in a single
    file, it would be.. umm.. painful. (Though I could get around
    it with #include trickery but it is still painful.)

    Sverker
     
    Sverker Nilsson, Dec 2, 2005
    #6
  7. "Sverker Nilsson" <> schrieb im Newsbeitrag
    news:...
    > "Claudio Grondi" <> wrote:
    >
    > > but the problem with sets.c remains:
    > >
    > > C:\VisualC++NET2003\Vc7\bin\cl.exe /c /nologo /Ox /MD /W3 /G7 /GX
    > > /DNDEBUG -IE:\Python24\include -IE:\Python24\PC /Tcsrc/sets/sets.c
    > > /Fobuild\temp.win32-2.4\Re
    > > lease\src/sets/sets.obj
    > > sets.c
    > > src\sets\sets.c(68) : error C2099: initializer is not a constant
    > > src\sets\sets.c(68) : warning C4047: 'initializing' : 'PyTypeObject *'
    > > differs in levels of indirection from 'NyHeapDef_SizeGetter'
    > > src\sets\sets.c(69) : error C2099: initializer is not a constant
    > > src\sets\sets.c(69) : warning C4028: formal parameter 1 different from
    > > declaration
    > > src\sets\sets.c(70) : error C2099: initializer is not a constant
    > > src\sets\sets.c(70) : warning C4047: 'initializing' : 'PyTypeObject *'
    > > differs in levels of indirection from 'int (__cdecl *)(PyObject *)'
    > > src\sets\sets.c(70) : warning C4028: formal parameter 1 different from
    > > declaration
    > > src\sets\sets.c(70) : warning C4028: formal parameter 1 different from
    > > declaration
    > > error: command 'E:\VisualC++NET2003\Vc7\bin\cl.exe' failed with exit

    status
    > > 2
    > >
    > > The MSDN help gives a simple example of code raising compiler error

    C2099:
    > > // C2099.c
    > > int j;
    > > int *p;
    > > j = *p; // C2099, *p is not a constant
    > >
    > > The example code shows to me, that there is a good reason compiler
    > > generates an error in that case.
    > > j = *p; leads to an assignment of a random (and therefore maybe leading

    to a
    > > non deterministic crash of the executable during runtime) value to a
    > > variable, what can't be intended in a deterministic program.

    >
    > This is confusing, because the problem of the example code is not that
    > *p is not constant but rather that p is undefined aka uninitialized.
    >
    > Humm..
    >
    > Compiling the following program:
    >
    > #include <stdio.h>
    > int main(void) {
    > int j;
    > int *p;
    > j = *p;
    > printf("%d\n",j); /* Makes sure the optimizer doesnt remove the
    > previous code */
    > return 0;
    > }
    >
    > with
    >
    > gcc -O3 -Wall main.c
    >
    > gives me:
    >
    > main.c: In function `main':
    > main.c:4: warning: `p' might be used uninitialized in this function
    >
    > (And I get a Segmentation fault from running the executable.)
    >
    > The -Wall flag enables, umm, as it is saying 'all' warnings
    > though perhaps not really all, and this flag + others is used
    > by the compilation command generated by distutils when
    > building with gcc.
    >
    > I don't see any warnings when building Guppy.
    >
    > So there should be no case (as easy to discover) as that in the
    > example.
    >
    > So I am confused.
    >
    > I was beginning to wonder if we were talking about the same file/code.
    > This code is from sets.c lines 66..71
    >
    > static NyHeapDef nysets_heapdefs[] = {
    > {0, &NyMutBitSet_Type, (NyHeapDef_SizeGetter) mutbitset_indisize},
    > {0, &NyCplBitSet_Type, 0, cplbitset_traverse},
    > {0, &NyNodeSet_Type, nodeset_indisize, nodeset_traverse,
    > nodeset_relate},
    > {0}
    > };

    The code in my sets.c file is lines 67..72:
    static NyHeapDef nysets_heapdefs[] = {
    {0, &NyMutBitSet_Type, (NyHeapDef_SizeGetter) mutbitset_indisize},
    {0, &NyCplBitSet_Type, 0, cplbitset_traverse},
    {0, &NyNodeSet_Type, nodeset_indisize, nodeset_traverse,
    nodeset_relate},
    {0}
    };

    Maybe you can explain what this declaration is good for and where the
    identifiers are defined, so that I have a chance to fix it myself?
    Currently I have no idea where to start and what is wrong with it.
    It seems, that the only one not declared identifier is NyHeapDef_SizeGetter,
    but this does not help because if I preceede the code with
    extern NyHeapDef_SizeGetter
    I get:

    src\sets\sets.c(68) : warning C4091: 'extern ' : ignored on left of 'int
    (__cdecl *)(PyObject *)' when no variable is declared
    src\sets\sets.c(78) : error C2099: initializer is not a constant
    src\sets\sets.c(78) : warning C4047: 'initializing' : 'PyTypeObject *'
    differs in levels of indirection from 'NyHeapDef_SizeGetter'
    src\sets\sets.c(79) : error C2099: initializer is not a constant
    src\sets\sets.c(79) : warning C4028: formal parameter 1 different from
    declaration
    src\sets\sets.c(80) : error C2099: initializer is not a constant
    src\sets\sets.c(80) : warning C4047: 'initializing' : 'PyTypeObject *'
    differs in levels of indirection from 'int (__cdecl *)(PyObject *)'
    src\sets\sets.c(80) : warning C4028: formal parameter 1 different from
    declaration
    src\sets\sets.c(80) : warning C4028: formal parameter 1 different from
    declaration
    error: command 'E:\VisualC++NET2003\Vc7\bin\cl.exe' failed with exit status
    2

    where the code at line 68 is:

    extern NyHeapDef_SizeGetter; // line 68
    // extern mutbitset_indisize;
    // extern NyCplBitSet_Type;
    // extern cplbitset_traverse;
    // extern NyNodeSet_Type;
    // extern nodeset_indisize;
    // extern nodeset_traverse;
    // extern nodeset_relate;

    static NyHeapDef nysets_heapdefs[] = {
    {0, &NyMutBitSet_Type, (NyHeapDef_SizeGetter) mutbitset_indisize},
    {0, &NyCplBitSet_Type, 0, cplbitset_traverse},
    {0, &NyNodeSet_Type, nodeset_indisize, nodeset_traverse,
    nodeset_relate},
    {0}
    };


    >
    > I can't see how there can be problems with initializers not being
    > constants here unless, perhaps if the compiler has a problem since
    > those functions (mutbitset_indisize etc) and the types that are used
    > as initializers are in separately compiled files. But gcc -Wall
    > -pedantic handles it and doesn't warn. I don't have the ANSI standard
    > but I have the 2nd edition of The C Programming Language by Kernighan
    > and Ritchie (1988, 'Based on Draft-Proposed ANSI C'), and it is saying
    >
    > - quotes -
    >
    > A8.7 Initialization
    > ...
    >
    > All the expressions in the initializer for a static object or array
    > must be constant expressions as described in &A7.19
    > ...
    >
    > A7.19 Constant Expressions
    >
    > Syntactically, a constant expression is an expression restricted to a
    > subset of operators...
    > ...
    >
    > More latitude is permitted for the constant expressions of
    > initializers... Initializers must evaluate either to a constant or to
    > the address of a previously declared external or static object plus or
    > minus a constant.
    >
    > - end quotes -
    >
    > If this means the Microsoft .NET compiler is perhaps not compliant
    > with standard/ANSI C then since it was from 2003 maybe they have come
    > out with an update now that fixes this problem. I know, it may not
    > really help you because it may cost money, I don't know how long one
    > can get updates without paying. Alternatively, I think GCC is
    > available for Microsoft Windows although I don't know how it handles
    > the .NET architecture.
    >
    > For me, if the case is according to hypothesis, and I would have to
    > rewrite the code to put all the functions and type objects in a single
    > file, it would be.. umm.. painful. (Though I could get around
    > it with #include trickery but it is still painful.)
    >
    > Sverker
    >


    I don't know if it applies here, but in this context the extern keyword
    comes to my mind.

    Here is what MSDN says about it:
    The extern keyword declares a variable or function and specifies that it has
    external linkage (its name is visible from files other than the one in which
    it's defined). When modifying a variable, extern specifies that the variable
    has static duration (it is allocated when the program begins and deallocated
    when the program ends). The variable or function may be defined in another
    source file, or later in the same file. Declarations of variables and
    functions at file scope are external by default.

    Maybe it helps?

    Claudio
     
    Claudio Grondi, Dec 2, 2005
    #7

  8. > "Claudio Grondi" <> wrote:

    I don't know if it applies here, but in this context the extern keyword

    > comes to my mind.


    [snip extracts from Microsoft docs]

    Perhaps. But I suspect it isn't that simple since ...
    I'd think even if I don't use the extern keyword the compiler
    should be able to figure it out. (As a service to the programmer.)
    And it shouldn't be very difficult to do.

    I'll probably not try this out for a while,
    but maybe you can check it yourself or ask someone that knows C.

    Regards,


    Sverker
     
    Sverker Nilsson, Dec 3, 2005
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Sverker Nilsson

    Guppy-PE 0.1.5 released

    Sverker Nilsson, Oct 12, 2006, in forum: Python
    Replies:
    0
    Views:
    286
    Sverker Nilsson
    Oct 12, 2006
  2. Sverker Nilsson

    Guppy-PE / Heapy 0.1.8

    Sverker Nilsson, Apr 8, 2008, in forum: Python
    Replies:
    0
    Views:
    429
    Sverker Nilsson
    Apr 8, 2008
  3. Sverker Nilsson

    Guppy-PE/Heapy 0.1.9 released

    Sverker Nilsson, Jun 23, 2009, in forum: Python
    Replies:
    0
    Views:
    268
    Sverker Nilsson
    Jun 23, 2009
  4. Boris Borcic
    Replies:
    10
    Views:
    474
    Ethan Furman
    Sep 11, 2009
  5. Juan Declet-Barreto

    guppy

    Juan Declet-Barreto, Nov 9, 2011, in forum: Python
    Replies:
    0
    Views:
    177
    Juan Declet-Barreto
    Nov 9, 2011
Loading...

Share This Page