MinGW and Python

[sturlamolden]

Look what I just found:

http://jove.prohosting.com/iwave/ipython/pyMinGW.html

A build process for python 2.4.2 (i.e. not the latest) for MinGW.

 

[Alex Martelli]

this is the second time I've seen that 14% figure. OOC, where does it come
from? the data sets you posted show an average 12.6% speedup. 14 is an
odd way to round.

I believe 12.6% is the result if you pick the faster speed as a
denominator (and so it's the right figure for a _slowdown_,
hypothetically moving from fast to slow case), 14% if you pick the
slower speed as a denominator (i.e., for a _speedup_).

Of course if you work with times rather than speeds it's the other way
around.

I don't think it's very useful to talk about average speedups from a
benchmark of equally-weighted feature tests. the data shows wildly varying
differences in performance for each test. a real-world application could
be much slower or much faster on either platform depending on its feature
mix. not the type of thing that's amenable to expression as a single
value.

I believe the single figure is a useful summary. Even the most
sophisticated benchmarks are eventually boiled down to single figures,
as in "so many SPECmarks" etc, because in everyday discourse a scalar is
what you can reasonably discuss. Sure, philosophically speaking it
makes no sense to say that a compiler is better or worse than another at
optimization, without adding a lawyer's brief worth of qualifications
each and every time. In real life, it's a useful and practical
temporary simplification, and engineers (as opposed to mathematicians
and philosophers) have a bias towards practical usefulness.


Alex

 

[Alex Martelli]

Maybe someone with access to Windows and some free time could compare
pybench compiled under msvc and mingw. The best way to test the compiler
is to eliminate all other variables.

I assume you mean "compare pybench on Python 2.4.3 compiled under" etc.
Unfortunately, I suspect the ex-officio gcc defenders will only react by
finding yet another quibble (anything to avoid admitting that gcc may
not be as good at optimizing as a _Microsoft_ product -- perish the
thought!), therefore I'm disinclined to invest my scarce free time in
that particular endeavour.


Alex

 

[Edward Elliott]

I assume you mean "compare pybench on Python 2.4.3 compiled under" etc.
Yep

Unfortunately, I suspect the ex-officio gcc defenders will only react by
finding yet another quibble (anything to avoid admitting that gcc may
not be as good at optimizing as a _Microsoft_ product -- perish the
thought!), therefore I'm disinclined to invest my scarce free time in
that particular endeavour.

No problem. I don't expect anyone to follow through, it was more a
suggestion to forestall rampant speculation.

 

[Edward Elliott]

I believe the single figure is a useful summary. Even the most
sophisticated benchmarks are eventually boiled down to single figures,
as in "so many SPECmarks" etc, because in everyday discourse a scalar is
what you can reasonably discuss. Sure, philosophically speaking it
makes no sense to say that a compiler is better or worse than another at
optimization, without adding a lawyer's brief worth of qualifications

^^^^^^^^^^^^^^
hey is that a shot at me?

each and every time. In real life, it's a useful and practical
temporary simplification, and engineers (as opposed to mathematicians
and philosophers) have a bias towards practical usefulness.

I agree for benchmarks in general. It's more this particular benchmark I
object to as not being representative. It's like the difference between
SPECmark testing various CPU functions and Winstone measuring real-world
application performance. If you're a CPU designer counting coup, SPECmark
matters. For just about everyone else, Winstone tells you more. And when
you're talking about small (less than an order of magnitude) differences in
SPECmark, platform/system/application issues becomes the dominant factor in
performance. A CPU that's twice as fast won't help my i/o-bound server.

From an engineering standpoint, the pybench number isn't that useful to me.
It tells me little about the practical speed of my application on two
different python interpreters. That's all I'm saying. No need to sic the
philosophers on me (a fate I would not wish on my worst enemy).

 

[Neal Becker]

I cannot predict this, though it would be great if somebody who does
have both VS2003 and mingw could give it a try.

What I did just post on another thread over the last couple of days is
about MacOSX, which also uses gcc: 14% faster pybench using Python 2.4.3
under Win2000 under Parallels Workstation beta, compared to 2.4.3
Universal directly on MacOSX -- the standard build of 2.4.3 in either
cause, i.e., the one built with MS compilers on Windows, vs the one
built with Apple's gcc on MacOSX.


Alex

Please when quoting such benchmarks include gcc version. gcc >= 4.1 is
supposed to have a lot of performance improvements. This is the current
release. Since mingw is usually current, I haven't checked, but they may
be using 4.1 now.

 

[Fredrik Lundh]

Please when quoting such benchmarks include gcc version. gcc >= 4.1 is
supposed to have a lot of performance improvements. This is the current
release. Since mingw is usually current, I haven't checked, but they may
be using 4.1 now.

does
"the standard build of 2.4.3 in either case"

do you have trouble understanding ?

(what makes you think that 2.4.3 for windows is compiled with the best possible
compiler for the windows environment?)

</F>

 

[Fredrik Lundh]

Please when quoting such benchmarks include gcc version. gcc >= 4.1 is
supposed to have a lot of performance improvements. This is the current
release. Since mingw is usually current, I haven't checked, but they may
be using 4.1 now.

what part of

"the standard build of 2.4.3 in either case"

do you have trouble understanding ?

(what makes you think that 2.4.3 for windows is compiled with the best possible
compiler for the windows environment?)

</F>

 

[Brian Elmegaard]

release. Since mingw is usually current, I haven't checked, but they may
be using 4.1 now.

It is not, it is 3.4.2.
http://www.mingw.org/download.shtml#hdr2

 

[Guest]

don't know what you mean by "endorsed". does it lack features of the C89
ANSI standard?

There isn't an import library for these DLLs anywhere, and Microsofts
documents it (msvcrt.dll) as not intended for application use:

http://msdn2.microsoft.com/en-us/library/abx4dbyh(VS.80).aspx

'The msvcrt.dll is now a "known DLL," meaning that it is a system
component owned and built by Windows. It is intended for future use only
by system-level components.'

That would be a problem then. So what happens when you compile python with
msvc, and why can't mingw just replicate that?

I link with msvcr71.dll (currently). To distribute Python, I need to
include a copy of msvcr71.dll, which I can, because the MSVC license
allows me to redistribute that DLL.

When I link with mingw, I have a choice of DLLs to link with, including
msvcrt.dll, msvcrt4.dll, msvcr71.dll, and perhaps others - I don't even
need the DLLs on my system to link with them.

However, I cannot redistribute these DLLs when I compile with MingW
(unless I also have a copy of VS.NET - I would have to reread its
license to find out whether it requires that the application is actually
built with MSVC to allow for redistribution).

Regards,
Martin

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

MSVCRT.DLL has been a standard system compent of Windows since at least
Windows 98. Many other system components depend on it. Essentially,
MSVCRT.DLL is an "undocumented" part of the Windows API. It's not
exactly "endorsed", Microsoft would rather you use it's current
compiler and runtime, but it is the standard "official" Windows system
C library.

See

http://msdn2.microsoft.com/en-us/library/abx4dbyh(VS.80).aspx

'The msvcrt.dll is now a "known DLL," meaning that it is a system
component owned and built by Windows. It is intended for future use only
by system-level components.'

The SDK stopped including an import library for it (I believe earlier
versions of the SDK still had an import library).

Regardless, there is no version of the MS C++ library that links against
msvcrt.dll. So if Python is linked against msvcrt.dll, you can't really
build C++ extensions anymore (with MSVC), unless you are certain that
mixing CRTs causes no harm for your application.

Regards,
Martin

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

Not exactly. They're both GCC, but the MinGW compiler that you can
download from MinGW WWW site is a native Win32 appliction, while the
"MinGW" compiler included with Cygwin and invoked by "-mno-cygwin" is a
Cygwin application.

Any Cygwin application is a native Win32 application, which just happens
to link with cygwin1.dll (which also is a native Win32 DLL).

Just to make that clear: Everything that cygwin does is done solely with
the Win32 API. The C library differs in semantics from the MS C library,
but the system API is just the same.

Regards,
Martin

 

[Ross Ridge]

MSVCRT.DLL has been a standard system compent of Windows since at least
Windows 98. Many other system components depend on it. Essentially,
MSVCRT.DLL is an "undocumented" part of the Windows API. It's not
exactly "endorsed", Microsoft would rather you use it's current
compiler and runtime, but it is the standard "official" Windows system
C library.

http://msdn2.microsoft.com/en-us/library/abx4dbyh(VS.80).aspx

'The msvcrt.dll is now a "known DLL," meaning that it is a system
component owned and built by Windows. It is intended for future use only
by system-level components.'

Exactly, it's the standard Windows system C library.

The SDK stopped including an import library for it (I believe earlier
versions of the SDK still had an import library).

An import library for it is easy enough to obtain. You can make your
own, or just grab the one from MinGW. If you insist on a Microsoft
source for it then you can find it in the Windows DDK.

Regardless, there is no version of the MS C++ library that links against
msvcrt.dll.

The version of the MS C++ library in Visual Studio C++ 6 links against
MSVCRT.DLL.

So if Python is linked against msvcrt.dll, you can't really build C++
extensions anymore (with MSVC), unless you are certain that
mixing CRTs causes no harm for your application.

Python 2.4 is linked against MSVCR71.DLL, so you can't really build C
or C++ extensions anymore with Microsoft's current compiler either.

Ross Ridge

 

[Ross Ridge]

Not exactly. They're both GCC, but the MinGW compiler that you can
download from MinGW WWW site is a native Win32 appliction, while the
"MinGW" compiler included with Cygwin and invoked by "-mno-cygwin" is a
Cygwin application.

Any Cygwin application is a native Win32 application, which just happens
to link with cygwin1.dll (which also is a native Win32 DLL)

Nonetheless, Cygwin applications are not generally considered native
Win32 applications because of the dependency on CYGWIN1.DLL and the
related environment. While what you're saying a strictly true, the
term "native Win32" is used to make a distinction between a port of a
program that doesn't use or require the Cygwin environment from one
that does. For example, the official version of Python for Windows is
considered a native Win32 application because it's implemented using
the Windows APIs directly, as opposed to the Cygwin's version of Python
which implemented using Cygwin's POSIX emulation API, and only uses the
Windows API indirectly.

Ross Ridge

 

[Philippe Martin]

This might relevant.

http://www.aceshardware.com/read.jsp?id=153

Philippe

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

Exactly, it's the standard Windows system C library.

That's not how I read it. To me, it says: it can be used by other
parts of Windows itself (i.e. system-level components), but it is not
intended to be used by third-party applications (such as Python),
as these are not system-level components.

The version of the MS C++ library in Visual Studio C++ 6 links against
MSVCRT.DLL.

I see (*). Unfortunately, that version isn't available anymore, and
wasn't available anymore when Python 2.4 was released.

Python 2.4 is linked against MSVCR71.DLL, so you can't really build C
or C++ extensions anymore with Microsoft's current compiler either.

Right: you need to use VS.NET 2003 (which was the current compiler
when Python 2.4 was first released).

Regards,
Martin

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

Nonetheless, Cygwin applications are not generally considered native
Win32 applications because of the dependency on CYGWIN1.DLL and the
related environment. While what you're saying a strictly true, the
term "native Win32" is used to make a distinction between a port of a
program that doesn't use or require the Cygwin environment from one
that does.

I know it is common to take that view, but I believe it is wrong, no
matter how you look at it:

- Is winword.exe not a native Win32 library because it uses "MSO.DLL"?
- A cygwin application does *not* require the Cygwin environment.
You don't need to invoke it from bash.exe, you don't need
to install Cygwin to run it, and you don't need a directory structure
including /usr or /etc to make it work.

For example, the official version of Python for Windows is
considered a native Win32 application because it's implemented using
the Windows APIs directly

That isn't (entirely) true. Python largely doesn't use the Win32 API
directly, but instead, it uses the C library of the C compiler it
was built with. It also uses the Win32 API directly, and more so
in Python 2.5 than in earlier versions, but reliance on the C library
still hasn't been dropped.

For Py3k, perhaps the C library can be dropped entirely: Guido wants
to drop usage of stdio as an implementation of the file object; if
then malloc/free is dropped in favour of a "direct" implementation
(e.g. based on HeapAlloc), then Python could become a "native Win32
application" in the sense you defined above.

Regards,
Martin

 

[sturlamolden]

That's not how I read it. To me, it says: it can be used by other
parts of Windows itself (i.e. system-level components), but it is not
intended to be used by third-party applications (such as Python),
as these are not system-level components.

That is correct. And it is the reson why the MinGW team is working on
removing the dependency on this CRT. I

 

[sturlamolden]

Martin v. Löwis wrote:

- Is winword.exe not a native Win32 library because it uses "MSO.DLL"?
- A cygwin application does *not* require the Cygwin environment.

Cygwin executables are native windows ".exe files" just like MinGW
executables. They are built by the same compiler, a port of GCC to 32
bit Windows originally written by Mumit Khan. MinGW and Cygwin apps run
at the same speed. However, the CYGWIN1.dll is bloated and an has
unwanted overhead for many system calls. But as such, CYGWIN1.dll is
nothing more than a CRT with extended "Linux API" support, including
posix and xlib.

The problem is actually *licensing issues* related to CYGWIN1.DLL. It
cannot always be linked. CYGWIN1.DLL can only be used for Open Source
development.

"In accordance with section 10 of the GPL, Red Hat permits programs
whose sources are distributed under a license that complies with the
Open Source definition to be linked with libcygwin.a/cygwin1.dll
without libcygwin.a/cygwin1.dll itself causing the resulting program to
be covered by the GNU GPL."

Thus, if you have a commercial closed source application, you need to
use MinGW; that is, you cannot link with libcygwin.a/cygwin1.dll.

I am not sure if Cygwin has link libraries for the Win32 API. Is there
e.g. a libuser32.a for linking with user32.dll? But MinGW has that.

 

[sturlamolden]

Martin v. Löwis wrote:

- Is winword.exe not a native Win32 library because it uses "MSO.DLL"?
- A cygwin application does *not* require the Cygwin environment.

Cygwin executables are native windows ".exe files" just like MinGW
executables. They are built by the same compiler, a port of GCC to 32
bit Windows originally written by Mumit Khan. MinGW and Cygwin apps run
at the same speed. However, the CYGWIN1.dll is bloated and an has
unwanted overhead for many system calls. But as such, CYGWIN1.dll is
nothing more than a CRT with extended "Linux API" support, including
posix and xlib.

The problem is actually *licensing issues* related to CYGWIN1.DLL. It
cannot always be linked. CYGWIN1.DLL can only be used for Open Source
development.

"In accordance with section 10 of the GPL, Red Hat permits programs
whose sources are distributed under a license that complies with the
Open Source definition to be linked with libcygwin.a/cygwin1.dll
without libcygwin.a/cygwin1.dll itself causing the resulting program to
be covered by the GNU GPL."

Thus, if you have a commercial closed source application, you need to
use MinGW; that is, you cannot link with libcygwin.a/cygwin1.dll.

I am not sure if Cygwin has link libraries for the Win32 API. Is there
e.g. a libuser32.a for linking with user32.dll? But MinGW has that.