Python 2.6 and wrapping C libraries on Windows

Discussion in 'Python' started by L. Lindstrom, Apr 30, 2008.

  1. L. Lindstrom

    L. Lindstrom Guest

    I have read that Python extension modules must link to the same C
    run-time as the Python interpreter. This I can appreciate. But does this
    requirement extend to the C libraries an extension module wraps. The
    case in point is Pygame and SDL. The Pygame extension modules are built
    with distutils, so for Python 2.6 using Visual Studio 2008 should ensure
    the .pyd files link to msvcr90.dll. But SDL is built using Msys/MinGW
    and the configure/make tool chain. This fails when linking to
    msvcr90.dll since the small test programs configure builds lack manifest
    files. They fail to load msvcr90.dll, raising an R6034 error instead. So
    besides heap management and FILE pointers, is there any reason SDL, or
    any C dependency, needs to link to the same C run-time as Python? If I
    ensure SDL frees memory it allocates and does not directly access a file
    opened by Python can I just use another C run-time such as msvcrt?


    --
    Lenard Lindstrom
    "%s@%s.%s" % ('len-l', 'telus', 'net')
     
    L. Lindstrom, Apr 30, 2008
    #1
    1. Advertising

  2. L. Lindstrom schrieb:
    > I have read that Python extension modules must link to the same C
    > run-time as the Python interpreter. This I can appreciate. But does this
    > requirement extend to the C libraries an extension module wraps. The
    > case in point is Pygame and SDL. The Pygame extension modules are built
    > with distutils, so for Python 2.6 using Visual Studio 2008 should ensure
    > the .pyd files link to msvcr90.dll. But SDL is built using Msys/MinGW
    > and the configure/make tool chain. This fails when linking to
    > msvcr90.dll since the small test programs configure builds lack manifest
    > files. They fail to load msvcr90.dll, raising an R6034 error instead. So
    > besides heap management and FILE pointers, is there any reason SDL, or
    > any C dependency, needs to link to the same C run-time as Python? If I
    > ensure SDL frees memory it allocates and does not directly access a file
    > opened by Python can I just use another C run-time such as msvcrt?
    >


    Your analysis of the problem and the implication of mixing CRTs is
    correct. However ...

    It should be trivial to modify the build systemof SDL so that the
    manifest is integrated into the DLLs. Everything else is a hack. It
    *should* work and in reality it *does* work for most cases. But someday
    you'll hit a solid wall and get strange and hard to debug segfaults.

    It's in your own interest to get it right in the first place. And you'd
    serve the Python community greatly by providing a nice tutorial how to
    modify 3rd party builds. *hint* :)

    If you need any help feel free to contact me. The new build system is
    mostly my work with help from Martin, Amaury and other core developers.

    Christian
     
    Christian Heimes, Apr 30, 2008
    #2
    1. Advertising

  3. L. Lindstrom

    sturlamolden Guest

    On Apr 30, 8:06 pm, "L. Lindstrom" <> wrote:

    > I have read that Python extension modules must link to the same C
    > run-time as the Python interpreter. This I can appreciate. But does this
    > requirement extend to the C libraries an extension module wraps.


    This somewhat of a misconception. You cannot reliably mix and blend
    CRT resources across different CRTs. This is not really a Python
    problem. It applies to any program. The reason this is important for
    Python C extensions, is mainly the possibility of accessing a Python
    file object as a pointer to a FILE struct in C. If you get a FILE*
    pointer from one CRT, you should not pass it to another CRT's fread.
    Likewise, if you allocate memory with one CRT's malloc(), you should
    not release the memory with another CRT's free(). As long as your
    libraries don't share CRT resources, it does not matter that the link
    to different CRTs for their internal work.
     
    sturlamolden, Apr 30, 2008
    #3
  4. L. Lindstrom

    L. Lindstrom Guest

    Christian Heimes wrote:
    > L. Lindstrom schrieb:
    >> I have read that Python extension modules must link to the same C
    >> run-time as the Python interpreter. This I can appreciate. But does this
    >> requirement extend to the C libraries an extension module wraps. The
    >> case in point is Pygame and SDL. The Pygame extension modules are built
    >> with distutils, so for Python 2.6 using Visual Studio 2008 should ensure
    >> the .pyd files link to msvcr90.dll. But SDL is built using Msys/MinGW
    >> and the configure/make tool chain. This fails when linking to
    >> msvcr90.dll since the small test programs configure builds lack manifest
    >> files. They fail to load msvcr90.dll, raising an R6034 error instead. So
    >> besides heap management and FILE pointers, is there any reason SDL, or
    >> any C dependency, needs to link to the same C run-time as Python? If I
    >> ensure SDL frees memory it allocates and does not directly access a file
    >> opened by Python can I just use another C run-time such as msvcrt?
    >>

    >
    > Your analysis of the problem and the implication of mixing CRTs is
    > correct. However ...
    >
    > It should be trivial to modify the build systemof SDL so that the
    > manifest is integrated into the DLLs. Everything else is a hack. It
    > *should* work and in reality it *does* work for most cases. But someday
    > you'll hit a solid wall and get strange and hard to debug segfaults.
    >
    > It's in your own interest to get it right in the first place. And you'd
    > serve the Python community greatly by providing a nice tutorial how to
    > modify 3rd party builds. *hint* :)
    >
    > If you need any help feel free to contact me. The new build system is
    > mostly my work with help from Martin, Amaury and other core developers.
    >
    > Christian
    >

    Linking to msvcr90.dll is possible with MinGW. The problem is with the
    configure scripts. So I can run configure against msvcrt.dll, then
    switch to mscvr90.dll for make. If this works I will make SDL and a test
    program available on-line so someone can find the appropriate manifests.


    --
    Lenard Lindstrom
    "%s@%s.%s" % ('len-l', 'telus', 'net')
     
    L. Lindstrom, May 1, 2008
    #4
  5. L. Lindstrom

    L. Lindstrom Guest

    sturlamolden wrote:
    > On Apr 30, 8:06 pm, "L. Lindstrom" <> wrote:
    >
    >> I have read that Python extension modules must link to the same C
    >> run-time as the Python interpreter. This I can appreciate. But does this
    >> requirement extend to the C libraries an extension module wraps.

    >
    > This somewhat of a misconception. You cannot reliably mix and blend
    > CRT resources across different CRTs. This is not really a Python
    > problem. It applies to any program. The reason this is important for
    > Python C extensions, is mainly the possibility of accessing a Python
    > file object as a pointer to a FILE struct in C. If you get a FILE*
    > pointer from one CRT, you should not pass it to another CRT's fread.
    > Likewise, if you allocate memory with one CRT's malloc(), you should
    > not release the memory with another CRT's free(). As long as your
    > libraries don't share CRT resources, it does not matter that the link
    > to different CRTs for their internal work.
    >
    >
    >
    >
    >
    >

    SDL has functions for separating memory allocation and file access.
    Going back to msvcrt.dll is a last resort. I may have an idea on how to
    build it for msvcr90.dll.

    --
    Lenard Lindstrom
    "%s@%s.%s" % ('len-l', 'telus', 'net')
     
    L. Lindstrom, May 1, 2008
    #5
  6. L. Lindstrom

    L. Lindstrom Guest

    L. Lindstrom wrote:
    > Christian Heimes wrote:
    >> L. Lindstrom schrieb:

    [snip]
    >>> esides heap management and FILE pointers, is there any reason SDL, or
    >>> any C dependency, needs to link to the same C run-time as Python? If I
    >>> ensure SDL frees memory it allocates and does not directly access a file
    >>> opened by Python can I just use another C run-time such as msvcrt?
    >>>

    >>
    >> Your analysis of the problem and the implication of mixing CRTs is
    >> correct. However ...
    >>
    >> It should be trivial to modify the build systemof SDL so that the
    >> manifest is integrated into the DLLs. Everything else is a hack. It
    >> *should* work and in reality it *does* work for most cases. But someday
    >> you'll hit a solid wall and get strange and hard to debug segfaults.
    >>

    [snip]
    > Linking to msvcr90.dll is possible with MinGW. The problem is with the
    > configure scripts. So I can run configure against msvcrt.dll, then
    > switch to mscvr90.dll for make. If this works I will make SDL and a test
    > program available on-line so someone can find the appropriate manifests.
    >
    >

    Here is my attempt to link SDL and a test program with msvcr90.dll. It
    can be found at http://www3.telus.net/len_l/pygame . The md5sum is:

    f5b71d9934d35c35a24b668ad8023146 *VC-2008-Run-Time-test.zip

    Both lack a manifest. The test program and SDL work when a surrogate
    run-time is provided, a renamed msvcr71.dll . So if someone can show me
    the necessary manifest to make SDL use the C run-time installed by the
    Python 2.6 installer I would appreciate it. SDL is built with MinGW so I
    doubt I can distribute the run-time with it.

    --
    Lenard Lindstrom
    "%s@%s.%s" % ('len-l', 'telus', 'net')
     
    L. Lindstrom, May 1, 2008
    #6
  7. L. Lindstrom

    illume Guest

    Hi,

    after a little research it appears that win9x is not supported by the
    msvcr90.dll run time.

    Can you confirm this Lenard?

    Has anyone tested the new python binaries that link to msvcr90.dll on
    win9x machines?

    cheers,




    On May 1, 3:05 pm, "L. Lindstrom" <> wrote:
    > L. Lindstrom wrote:
    > > Christian Heimes wrote:
    > >> L. Lindstrom schrieb:

    > [snip]
    > >>> esides heap management and FILE pointers, is there any reason SDL, or
    > >>> any C dependency, needs to link to the same C run-time as Python? If I
    > >>> ensure SDL frees memory it allocates and does not directly access a file
    > >>> opened by Python can I just use another C run-time such as msvcrt?

    >
    > >> Your analysis of the problem and the implication of mixing CRTs is
    > >> correct. However ...

    >
    > >> It should be trivial to modify the build systemof SDL so that the
    > >> manifest is integrated into the DLLs. Everything else is a hack. It
    > >> *should* work and in reality it *does* work for most cases. But someday
    > >> you'll hit a solid wall and get strange and hard to debug segfaults.

    >
    > [snip]
    > > Linking to msvcr90.dll is possible with MinGW. The problem is with the
    > > configure scripts. So I can run configure against msvcrt.dll, then
    > > switch to mscvr90.dll for make. If this works I will make SDL and a test
    > > program available on-line so someone can find the appropriate manifests.

    >
    > Here is my attempt to link SDL and a test program with msvcr90.dll. It
    > can be found athttp://www3.telus.net/len_l/pygame. The md5sum is:
    >
    > f5b71d9934d35c35a24b668ad8023146 *VC-2008-Run-Time-test.zip
    >
    > Both lack a manifest. The test program and SDL work when a surrogate
    > run-time is provided, a renamed msvcr71.dll . So if someone can show me
    > the necessary manifest to make SDL use the C run-time installed by the
    > Python 2.6 installer I would appreciate it. SDL is built with MinGW so I
    > doubt I can distribute the run-time with it.
    >
    > --
    > Lenard Lindstrom
    > "%s@%s.%s" % ('len-l', 'telus', 'net')
     
    illume, May 1, 2008
    #7
  8. L. Lindstrom

    illume Guest

    hi again,

    I should have said, the msvcr90.dll does not work on win9x machines -
    as well as not being supported by ms.


    cu,



    On May 1, 4:02 pm, illume <> wrote:
    > Hi,
    >
    > after a little research it appears that win9x is not supported by the
    > msvcr90.dll run time.
    >
    > Can you confirm this Lenard?
    >
    > Has anyone tested the new python binaries that link to msvcr90.dll on
    > win9x machines?
    >
    > cheers,
    >
    > On May 1, 3:05 pm, "L. Lindstrom" <> wrote:
    >
    > > L. Lindstrom wrote:
    > > > Christian Heimes wrote:
    > > >> L. Lindstrom schrieb:

    > > [snip]
    > > >>> esides heap management and FILE pointers, is there any reason SDL, or
    > > >>> any C dependency, needs to link to the same C run-time as Python? If I
    > > >>> ensure SDL frees memory it allocates and does not directly access a file
    > > >>> opened by Python can I just use another C run-time such as msvcrt?

    >
    > > >> Your analysis of the problem and the implication of mixing CRTs is
    > > >> correct. However ...

    >
    > > >> It should be trivial to modify the build systemof SDL so that the
    > > >> manifest is integrated into the DLLs. Everything else is a hack. It
    > > >> *should* work and in reality it *does* work for most cases. But someday
    > > >> you'll hit a solid wall and get strange and hard to debug segfaults.

    >
    > > [snip]
    > > > Linking to msvcr90.dll is possible with MinGW. The problem is with the
    > > > configure scripts. So I can run configure against msvcrt.dll, then
    > > > switch to mscvr90.dll for make. If this works I will make SDL and a test
    > > > program available on-line so someone can find the appropriate manifests.

    >
    > > Here is my attempt to link SDL and a test program with msvcr90.dll. It
    > > can be found athttp://www3.telus.net/len_l/pygame. The md5sum is:

    >
    > > f5b71d9934d35c35a24b668ad8023146 *VC-2008-Run-Time-test.zip

    >
    > > Both lack a manifest. The test program and SDL work when a surrogate
    > > run-time is provided, a renamed msvcr71.dll . So if someone can show me
    > > the necessary manifest to make SDL use the C run-time installed by the
    > > Python 2.6 installer I would appreciate it. SDL is built with MinGW so I
    > > doubt I can distribute the run-time with it.

    >
    > > --
    > > Lenard Lindstrom
    > > "%s@%s.%s" % ('len-l', 'telus', 'net')
     
    illume, May 1, 2008
    #8
  9. illume schrieb:
    > Hi,
    >
    > after a little research it appears that win9x is not supported by the
    > msvcr90.dll run time.
    >
    > Can you confirm this Lenard?
    >
    > Has anyone tested the new python binaries that link to msvcr90.dll on
    > win9x machines?


    It doesn't matter to use because Python 2.6 and 3.0 require at least
    Windows 2000 SP4. The 9x, ME and NT series aren't supported any more.

    Christian
     
    Christian Heimes, May 1, 2008
    #9
  10. L. Lindstrom

    illume Guest

    dropping win98 support? was Re: Python 2.6 and wrapping C librarieson Windows

    Ah, why is that?

    There's still at least 1.1% of people using win98, if you believe this
    source of stats:
    http://www.w3schools.com/browsers/browsers_os.asp

    I just noticed that win9x winme and win nt are all being dropped from
    python.

    I know they are old and crufty, but there's still heaps of people
    using them.

    Someone pointed me to the pep, where the un-support seems planned:
    http://www.python.org/dev/peps/pep-0011/


    Seems like a lot of people using it, so it's still worthwhile making
    2.6 work with win98.





    On May 1, 10:09 pm, Christian Heimes <> wrote:
    > illume schrieb:
    >
    > > Hi,

    >
    > > after a little research it appears that win9x is not supported by the
    > > msvcr90.dll run time.

    >
    > > Can you confirm thisLenard?

    >
    > > Has anyone tested the new python binaries that link to msvcr90.dll on
    > > win9x machines?

    >
    > It doesn't matter to use because Python 2.6 and 3.0 require at least
    > Windows 2000 SP4. The 9x, ME and NT series aren't supported any more.
    >
    > Christian
     
    illume, May 1, 2008
    #10
  11. Re: dropping win98 support?

    illume schrieb:
    > Ah, why is that?
    >
    > There's still at least 1.1% of people using win98, if you believe this
    > source of stats:
    > http://www.w3schools.com/browsers/browsers_os.asp
    >
    > I just noticed that win9x winme and win nt are all being dropped from
    > python.
    >
    > I know they are old and crufty, but there's still heaps of people
    > using them.


    The Python core developer team has limited resources. We don't want to
    waste our energy with supporting ancient operation systems. Microsoft
    has dropped the support for the 9x and NT series several years ago.
    Dropping the support as well makes future development and new features
    much easier for us.

    Python can finally depend on the wide api functions and sane unicode
    support.

    > Someone pointed me to the pep, where the un-support seems planned:
    > http://www.python.org/dev/peps/pep-0011/
    >
    >
    > Seems like a lot of people using it, so it's still worthwhile making
    > 2.6 work with win98.


    People can still use Python 2.5. Users of deprecated and unsupported
    OSes can't except new versions of software to run on their boxes.

    Christian
     
    Christian Heimes, May 1, 2008
    #11
  12. L. Lindstrom

    Terry Reedy Guest

    Re: dropping win98 support? was Re: Python 2.6 and wrapping Clibraries on Windows

    "illume" <> wrote in message
    news:...
    | Ah, why is that?

    Were any of the reasons given in
    http://www.python.org/dev/peps/pep-0011/
    unclear?
    It appears you are already aware of MS's non-support of Win98

    | Seems like a lot of people using it, so it's still worthwhile making
    | 2.6 work with win98.

    Since the warning was given in 2.5, no one has agreed enough to volunteer
    to do do.
     
    Terry Reedy, May 1, 2008
    #12
  13. L. Lindstrom

    illume Guest

    Re: dropping win98 support? was Re: Python 2.6 and wrapping Clibraries on Windows

    On May 2, 8:37 am, "Terry Reedy" <> wrote:
    > "illume" <> wrote in message
    >
    > news:...
    > | Ah, why is that?
    >
    > Were any of the reasons given inhttp://www.python.org/dev/peps/pep-0011/
    > unclear?
    > It appears you are already aware of MS's non-support of Win98



    Hello,

    It seems the main reason is for ease of maintenance. However the Pep
    title is misleading with regards to win9x+winMe+win2k - which is where
    my confusion, questions and argument came from.
    "Title: Removing support for little used platforms"

    There are still *lots* of people who still use win95, 98, 98se, me,
    and win2k - as shown by the statistics I linked to in a previous
    post. If you want more statistics about the number of people using
    what OS they are fairly easy to find with a search engine. One day
    win9x will be finally dead, but that's not yet(and the w3c stats show
    it's usage actually increasing in march!).

    It is probably way too late in the process to put back code - and as
    you say no python developers have volunteered. So I won't argue any
    more for it to come back.

    We'll just have to recommend a different python implementation than
    2.6 or 3.0 for people who want to support people with these old
    computers.


    cheers,
     
    illume, May 2, 2008
    #13
  14. Re: dropping win98 support? was Re: Python 2.6 and wrapping Clibraries on Windows

    illume schrieb:
    > There are still *lots* of people who still use win95, 98, 98se, me,
    > and win2k - as shown by the statistics I linked to in a previous
    > post. If you want more statistics about the number of people using
    > what OS they are fairly easy to find with a search engine. One day
    > win9x will be finally dead, but that's not yet(and the w3c stats show
    > it's usage actually increasing in march!).


    Windows 2000 is still supported by Python 2.6 and 3.0 although you may
    get into trouble if you haven't installed at least SP4.

    Christian
     
    Christian Heimes, May 2, 2008
    #14
  15. En Thu, 01 May 2008 09:09:21 -0300, Christian Heimes <>
    escribió:

    > illume schrieb:
    >> Has anyone tested the new python binaries that link to msvcr90.dll on
    >> win9x machines?

    >
    > It doesn't matter to use because Python 2.6 and 3.0 require at least
    > Windows 2000 SP4. The 9x, ME and NT series aren't supported any more.


    That should be menctioned in the What's new? document - I could not find
    any reference.

    --
    Gabriel Genellina
     
    Gabriel Genellina, May 2, 2008
    #15
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Karsten Wutzke
    Replies:
    21
    Views:
    945
    Roedy Green
    Jun 29, 2007
  2. cnb
    Replies:
    0
    Views:
    301
  3. xkenneth
    Replies:
    1
    Views:
    773
    Stef Mientki
    Jun 8, 2009
  4. Sriram Srinivasan
    Replies:
    13
    Views:
    596
    Benjamin Kaplan
    Nov 12, 2009
  5. zyng
    Replies:
    1
    Views:
    170
    Roedy Green
    Jan 11, 2014
Loading...

Share This Page