Python 2.6 and wrapping C libraries on Windows

[L. Lindstrom]

I have read that Python extension modules must link to the same C
run-time as the Python interpreter. This I can appreciate. But does this
requirement extend to the C libraries an extension module wraps. The
case in point is Pygame and SDL. The Pygame extension modules are built
with distutils, so for Python 2.6 using Visual Studio 2008 should ensure
the .pyd files link to msvcr90.dll. But SDL is built using Msys/MinGW
and the configure/make tool chain. This fails when linking to
msvcr90.dll since the small test programs configure builds lack manifest
files. They fail to load msvcr90.dll, raising an R6034 error instead. So
besides heap management and FILE pointers, is there any reason SDL, or
any C dependency, needs to link to the same C run-time as Python? If I
ensure SDL frees memory it allocates and does not directly access a file
opened by Python can I just use another C run-time such as msvcrt?

 

[Christian Heimes]

I have read that Python extension modules must link to the same C
run-time as the Python interpreter. This I can appreciate. But does this
requirement extend to the C libraries an extension module wraps. The
case in point is Pygame and SDL. The Pygame extension modules are built
with distutils, so for Python 2.6 using Visual Studio 2008 should ensure
the .pyd files link to msvcr90.dll. But SDL is built using Msys/MinGW
and the configure/make tool chain. This fails when linking to
msvcr90.dll since the small test programs configure builds lack manifest
files. They fail to load msvcr90.dll, raising an R6034 error instead. So
besides heap management and FILE pointers, is there any reason SDL, or
any C dependency, needs to link to the same C run-time as Python? If I
ensure SDL frees memory it allocates and does not directly access a file
opened by Python can I just use another C run-time such as msvcrt?

Your analysis of the problem and the implication of mixing CRTs is
correct. However ...

It should be trivial to modify the build systemof SDL so that the
manifest is integrated into the DLLs. Everything else is a hack. It
*should* work and in reality it *does* work for most cases. But someday
you'll hit a solid wall and get strange and hard to debug segfaults.

It's in your own interest to get it right in the first place. And you'd
serve the Python community greatly by providing a nice tutorial how to
modify 3rd party builds. *hint*

If you need any help feel free to contact me. The new build system is
mostly my work with help from Martin, Amaury and other core developers.

Christian

 

[sturlamolden]

I have read that Python extension modules must link to the same C
run-time as the Python interpreter. This I can appreciate. But does this
requirement extend to the C libraries an extension module wraps.

This somewhat of a misconception. You cannot reliably mix and blend
CRT resources across different CRTs. This is not really a Python
problem. It applies to any program. The reason this is important for
Python C extensions, is mainly the possibility of accessing a Python
file object as a pointer to a FILE struct in C. If you get a FILE*
pointer from one CRT, you should not pass it to another CRT's fread.
Likewise, if you allocate memory with one CRT's malloc(), you should
not release the memory with another CRT's free(). As long as your
libraries don't share CRT resources, it does not matter that the link
to different CRTs for their internal work.

 

[L. Lindstrom]

Your analysis of the problem and the implication of mixing CRTs is
correct. However ...

It should be trivial to modify the build systemof SDL so that the
manifest is integrated into the DLLs. Everything else is a hack. It
*should* work and in reality it *does* work for most cases. But someday
you'll hit a solid wall and get strange and hard to debug segfaults.

It's in your own interest to get it right in the first place. And you'd
serve the Python community greatly by providing a nice tutorial how to
modify 3rd party builds. *hint*

If you need any help feel free to contact me. The new build system is
mostly my work with help from Martin, Amaury and other core developers.

Christian

Linking to msvcr90.dll is possible with MinGW. The problem is with the
configure scripts. So I can run configure against msvcrt.dll, then
switch to mscvr90.dll for make. If this works I will make SDL and a test
program available on-line so someone can find the appropriate manifests.

 

[L. Lindstrom]

This somewhat of a misconception. You cannot reliably mix and blend
CRT resources across different CRTs. This is not really a Python
problem. It applies to any program. The reason this is important for
Python C extensions, is mainly the possibility of accessing a Python
file object as a pointer to a FILE struct in C. If you get a FILE*
pointer from one CRT, you should not pass it to another CRT's fread.
Likewise, if you allocate memory with one CRT's malloc(), you should
not release the memory with another CRT's free(). As long as your
libraries don't share CRT resources, it does not matter that the link
to different CRTs for their internal work.

SDL has functions for separating memory allocation and file access.
Going back to msvcrt.dll is a last resort. I may have an idea on how to
build it for msvcr90.dll.

 

[L. Lindstrom]

L. Lindstrom schrieb: [snip]

esides heap management and FILE pointers, is there any reason SDL, or
any C dependency, needs to link to the same C run-time as Python? If I
ensure SDL frees memory it allocates and does not directly access a file
opened by Python can I just use another C run-time such as msvcrt?

Your analysis of the problem and the implication of mixing CRTs is
correct. However ...

It should be trivial to modify the build systemof SDL so that the
manifest is integrated into the DLLs. Everything else is a hack. It
*should* work and in reality it *does* work for most cases. But someday
you'll hit a solid wall and get strange and hard to debug segfaults.

[snip]
Linking to msvcr90.dll is possible with MinGW. The problem is with the
configure scripts. So I can run configure against msvcrt.dll, then
switch to mscvr90.dll for make. If this works I will make SDL and a test
program available on-line so someone can find the appropriate manifests.

Here is my attempt to link SDL and a test program with msvcr90.dll. It
can be found at http://www3.telus.net/len_l/pygame . The md5sum is:

f5b71d9934d35c35a24b668ad8023146 *VC-2008-Run-Time-test.zip

Both lack a manifest. The test program and SDL work when a surrogate
run-time is provided, a renamed msvcr71.dll . So if someone can show me
the necessary manifest to make SDL use the C run-time installed by the
Python 2.6 installer I would appreciate it. SDL is built with MinGW so I
doubt I can distribute the run-time with it.

 

[illume]

Hi,

after a little research it appears that win9x is not supported by the
msvcr90.dll run time.

Can you confirm this Lenard?

Has anyone tested the new python binaries that link to msvcr90.dll on
win9x machines?

cheers,

L. Lindstrom schrieb: [snip]
esides heap management and FILE pointers, is there any reason SDL, or
any C dependency, needs to link to the same C run-time as Python? If I
ensure SDL frees memory it allocates and does not directly access a file
opened by Python can I just use another C run-time such as msvcrt?
Your analysis of the problem and the implication of mixing CRTs is
correct. However ...
It should be trivial to modify the build systemof SDL so that the
manifest is integrated into the DLLs. Everything else is a hack. It
*should* work and in reality it *does* work for most cases. But someday
you'll hit a solid wall and get strange and hard to debug segfaults.

[snip]
Linking to msvcr90.dll is possible with MinGW. The problem is with the
configure scripts. So I can run configure against msvcrt.dll, then
switch to mscvr90.dll for make. If this works I will make SDL and a test
program available on-line so someone can find the appropriate manifests.

Here is my attempt to link SDL and a test program with msvcr90.dll. It
can be found athttp://www3.telus.net/len_l/pygame. The md5sum is:

f5b71d9934d35c35a24b668ad8023146 *VC-2008-Run-Time-test.zip

Both lack a manifest. The test program and SDL work when a surrogate
run-time is provided, a renamed msvcr71.dll . So if someone can show me
the necessary manifest to make SDL use the C run-time installed by the
Python 2.6 installer I would appreciate it. SDL is built with MinGW so I
doubt I can distribute the run-time with it.

 

[illume]

hi again,

I should have said, the msvcr90.dll does not work on win9x machines -
as well as not being supported by ms.


cu,

Hi,

after a little research it appears that win9x is not supported by the
msvcr90.dll run time.

Can you confirm this Lenard?

Has anyone tested the new python binaries that link to msvcr90.dll on
win9x machines?

cheers,

Christian Heimes wrote:
L. Lindstrom schrieb: [snip]
esides heap management and FILE pointers, is there any reason SDL, or
any C dependency, needs to link to the same C run-time as Python? If I
ensure SDL frees memory it allocates and does not directly access a file
opened by Python can I just use another C run-time such as msvcrt?
Your analysis of the problem and the implication of mixing CRTs is
correct. However ...
It should be trivial to modify the build systemof SDL so that the
manifest is integrated into the DLLs. Everything else is a hack. It
*should* work and in reality it *does* work for most cases. But someday
you'll hit a solid wall and get strange and hard to debug segfaults.

[snip]
Linking to msvcr90.dll is possible with MinGW. The problem is with the
configure scripts. So I can run configure against msvcrt.dll, then
switch to mscvr90.dll for make. If this works I will make SDL and a test
program available on-line so someone can find the appropriate manifests.

Here is my attempt to link SDL and a test program with msvcr90.dll. It
can be found athttp://www3.telus.net/len_l/pygame. The md5sum is:

f5b71d9934d35c35a24b668ad8023146 *VC-2008-Run-Time-test.zip

Both lack a manifest. The test program and SDL work when a surrogate
run-time is provided, a renamed msvcr71.dll . So if someone can show me
the necessary manifest to make SDL use the C run-time installed by the
Python 2.6 installer I would appreciate it. SDL is built with MinGW so I
doubt I can distribute the run-time with it.

 

[Christian Heimes]

Hi,

after a little research it appears that win9x is not supported by the
msvcr90.dll run time.

Can you confirm this Lenard?

Has anyone tested the new python binaries that link to msvcr90.dll on
win9x machines?

It doesn't matter to use because Python 2.6 and 3.0 require at least
Windows 2000 SP4. The 9x, ME and NT series aren't supported any more.

Christian

 

[illume]

Ah, why is that?

There's still at least 1.1% of people using win98, if you believe this
source of stats:
http://www.w3schools.com/browsers/browsers_os.asp

I just noticed that win9x winme and win nt are all being dropped from
python.

I know they are old and crufty, but there's still heaps of people
using them.

Someone pointed me to the pep, where the un-support seems planned:
http://www.python.org/dev/peps/pep-0011/


Seems like a lot of people using it, so it's still worthwhile making
2.6 work with win98.

 

[Christian Heimes]

Ah, why is that?

There's still at least 1.1% of people using win98, if you believe this
source of stats:
http://www.w3schools.com/browsers/browsers_os.asp

I just noticed that win9x winme and win nt are all being dropped from
python.

I know they are old and crufty, but there's still heaps of people
using them.

The Python core developer team has limited resources. We don't want to
waste our energy with supporting ancient operation systems. Microsoft
has dropped the support for the 9x and NT series several years ago.
Dropping the support as well makes future development and new features
much easier for us.

Python can finally depend on the wide api functions and sane unicode
support.

Someone pointed me to the pep, where the un-support seems planned:
http://www.python.org/dev/peps/pep-0011/


Seems like a lot of people using it, so it's still worthwhile making
2.6 work with win98.

People can still use Python 2.5. Users of deprecated and unsupported
OSes can't except new versions of software to run on their boxes.

Christian

 

[Terry Reedy]

| Ah, why is that?

Were any of the reasons given in
http://www.python.org/dev/peps/pep-0011/
unclear?
It appears you are already aware of MS's non-support of Win98

| Seems like a lot of people using it, so it's still worthwhile making
| 2.6 work with win98.

Since the warning was given in 2.5, no one has agreed enough to volunteer
to do do.

 

[illume]

| Ah, why is that?

Were any of the reasons given inhttp://www.python.org/dev/peps/pep-0011/
unclear?
It appears you are already aware of MS's non-support of Win98

Hello,

It seems the main reason is for ease of maintenance. However the Pep
title is misleading with regards to win9x+winMe+win2k - which is where
my confusion, questions and argument came from.
"Title: Removing support for little used platforms"

There are still *lots* of people who still use win95, 98, 98se, me,
and win2k - as shown by the statistics I linked to in a previous
post. If you want more statistics about the number of people using
what OS they are fairly easy to find with a search engine. One day
win9x will be finally dead, but that's not yet(and the w3c stats show
it's usage actually increasing in march!).

It is probably way too late in the process to put back code - and as
you say no python developers have volunteered. So I won't argue any
more for it to come back.

We'll just have to recommend a different python implementation than
2.6 or 3.0 for people who want to support people with these old
computers.


cheers,

 

[Christian Heimes]

There are still *lots* of people who still use win95, 98, 98se, me,
and win2k - as shown by the statistics I linked to in a previous
post. If you want more statistics about the number of people using
what OS they are fairly easy to find with a search engine. One day
win9x will be finally dead, but that's not yet(and the w3c stats show
it's usage actually increasing in march!).

Windows 2000 is still supported by Python 2.6 and 3.0 although you may
get into trouble if you haven't installed at least SP4.

Christian

 

[Gabriel Genellina]

It doesn't matter to use because Python 2.6 and 3.0 require at least
Windows 2000 SP4. The 9x, ME and NT series aren't supported any more.

That should be menctioned in the What's new? document - I could not find
any reference.