build curiosities of svn head (on WinXP)

[David Murmann]

hi all!

i just built revision 41809 under winxp using a rather uncommon
setup (at least i think so). since i have no visual studio here,
i only used freely available tools: cygwin to get the source, the
microsoft compiler/linker and NAnt (nant.sf.net) as the build tool
to interpret the .vcproj-files (which may be the uncommon part).

as far as i understand this setup is not supported in any way, but
i noticed two problems, that could be interesting for others as well.

first, the ProjectGUID's in "pythoncore.vcproj" and "pcbuild.sln"
mismatch, i fixed that locally by changing "pythoncore.vcproj"
(maybe visual studio is ignoring/shadowing this problem?).

second, the build order in "pcbuild.sln" for elementtree seems to be
wrong, nant tried to build elementtree before pythoncore (which failed).
i fixed this by building elementtree separately.

so my question is: will something like this be changed to support this
setup (or at least to get the project files "cleaner")? should i submit
a patch and see if it gets accepted?

apart from that everything went fine, and i could reproduce the expected
failure (ATM) of the regression test suite:

http://mail.python.org/pipermail/python-dev/2005-December/059033.html

btw, if anyone is interested in the (rather small) build-script for
nant, just ask,

so long, David.

 

[Steve Holden]

hi all!

i just built revision 41809 under winxp using a rather uncommon
setup (at least i think so). since i have no visual studio here,
i only used freely available tools: cygwin to get the source, the
microsoft compiler/linker and NAnt (nant.sf.net) as the build tool
to interpret the .vcproj-files (which may be the uncommon part).
Excellent!

as far as i understand this setup is not supported in any way, but
i noticed two problems, that could be interesting for others as well.

first, the ProjectGUID's in "pythoncore.vcproj" and "pcbuild.sln"
mismatch, i fixed that locally by changing "pythoncore.vcproj"
(maybe visual studio is ignoring/shadowing this problem?).

That could be. I don't believe anyone who's doing any serious work on
the Windows implementation is using anything other than Visual Studio (I
seem to remember that a few copies were donated a while ago).

second, the build order in "pcbuild.sln" for elementtree seems to be
wrong, nant tried to build elementtree before pythoncore (which failed).
i fixed this by building elementtree separately.

Yes, the elementtree module is a new arrival for 3.5, so the build may
not yet be perfectly specified. This is useful information.

so my question is: will something like this be changed to support this
setup (or at least to get the project files "cleaner")? should i submit
a patch and see if it gets accepted?

I suspect that the patch's chances of being accepted are much higher if
its author will commit to providing ongoing support for it. At present
the developers seem to feel that this is the only reliable way to
incorporate a new platform (someone recently announced they had a
working DOS implementation of 2.4, but there was some resistance to
including it as a standard part of the build because of doubts about
ongoing support).

apart from that everything went fine, and i could reproduce the expected
failure (ATM) of the regression test suite:

http://mail.python.org/pipermail/python-dev/2005-December/059033.html

btw, if anyone is interested in the (rather small) build-script for
nant, just ask,

so long, David.

This is terrific news. Just recently there have been indications that
the Windows platform needs a larger base of developers, and this might
be just the thing to promote that. It might be worth a quick post to
python-dev to see what interest your new development gets. Well done!

regards
Steve

 

[Steve Holden]

David Murmann wrote [...]

apart from that everything went fine, and i could reproduce the expected
failure (ATM) of the regression test suite:

http://mail.python.org/pipermail/python-dev/2005-December/059033.html

btw, if anyone is interested in the (rather small) build-script for
nant, just ask,

so long, David.

This is terrific news. Just recently there have been indications that
the Windows platform needs a larger base of developers, and this might
be just the thing to promote that. It might be worth a quick post to
python-dev to see what interest your new development gets. Well done!

David:

Apparently Tim Peters has just tracked down the causes of the build
errors with VC7, so the Subversion repository should now hold a valid
project file for you to test against.

regards
Steve

 

[Tim Peters]

[David Murmann]
....
[Steve Holden]

Yes, the elementtree module is a new arrival for 3.5, so the build may
not yet be perfectly specified. This is useful information.

I just checked in a fix for that. Thanks!

....

Note that all tests pass on Windows as of Sunday (in release and debug
builds, with and without -O).

 

[David Murmann]

[David Murmann]
...
[Steve Holden]

Yes, the elementtree module is a new arrival for 3.5, so the build may
not yet be perfectly specified. This is useful information.

I just checked in a fix for that. Thanks!

this works for me, thanks for the quick patch!

Note that all tests pass on Windows as of Sunday (in release and debug
builds, with and without -O).

here i have problems. some tests fail more or less randomly.
after some testing, i found that it seems to be related to
the parsing of float literals in python code (wild guess).
for example, test_pow failed giving me this traceback:

Traceback (most recent call last):
File "D:\develop\cygwin\usr\local\src\python\python\trunk\lib\test\test_pow.py
", line 109, in test_bug705231
eq(pow(a, 1.23e167), 1.0)
AssertionError: -1.0 != 1.0

this seems very strange to me, as i get this on the interactive
prompt:1.0

another example is test_colorsys. this test hangs, because it
uses "frange(0.0, 1.0, 0.2)" (which does, what it looks like),
but the 0.2 is somehow read as 0.0 resulting in an infinite
loop.

i have no clue whether these failures are related to my build
environment or something in the python codebase (or the weather?).
all tests passed when building with gcc from cygwin btw. i will
try building 2.4.2 with nant later and see what that does...

i am lost at the moment, David.

PS: maybe i should note: i built python without _bsddb and without
_tcl but i don't think this should be causing problems.

 

[Fredrik Lundh]

here i have problems. some tests fail more or less randomly.
after some testing, i found that it seems to be related to
the parsing of float literals in python code (wild guess).
for example, test_pow failed giving me this traceback:

Traceback (most recent call last):
File "D:\develop\cygwin\usr\local\src\python\python\trunk\lib\test\test_pow.py
", line 109, in test_bug705231
eq(pow(a, 1.23e167), 1.0)
AssertionError: -1.0 != 1.0

this seems very strange to me, as i get this on the interactive
prompt:
1.0

sounds like some odd locale problem. consider
-1.0

another example is test_colorsys. this test hangs, because it
uses "frange(0.0, 1.0, 0.2)" (which does, what it looks like),
but the 0.2 is somehow read as 0.0 resulting in an infinite
loop.

and

frange(0, 1, 0)

try setting the locale (via the locale module) from the interactive prompt,
and see if Python still handles floating point values correctly.

</F>

 

[David Murmann]

try setting the locale (via the locale module) from the interactive prompt,
and see if Python still handles floating point values correctly.

well, it does not:
3.0

so this is where the problem has to be, but i am still not sure what to do
about this. is this a problem with my configuration, with my build or
with python?

i tried changing my windows configuration to use '.' as the decimal point,
after which the above worked as expected, but the test suite still fails
on some tests (audioop, binop, bisect, difflib, doctest, xdrlib fail and
colorsys still hangs). surely my workaround is no general solution and
it doesn't even work, so there has to be a problem somewhere else.

also note that some of these tests (all except difflib and doctest) pass,
if i change them in any way (like adding insignificant whitespace) and run
them alone.

thanks for the help, David.

 

[David Murmann]

i will try building 2.4.2 with nant later and see what that does...

FYI i did this now and it worked fine, all tests passed.

 

[Fredrik Lundh]

so this is where the problem has to be, but i am still not sure what to do
about this. is this a problem with my configuration, with my build or
with python?

it's a python bug, and it has been introduced relatively recently. iirc,
there has been some recent tweaks to the floating-point parsing code...

please file a report over at the bug tracker (and mark it as urgent).

</F>

 

[Fredrik Lundh]

it's a python bug, and it has been introduced relatively recently. iirc,
there has been some recent tweaks to the floating-point parsing code...

please file a report over at the bug tracker (and mark it as urgent).

see

http://www.python.org/sf/1391872

for bug report and patch.

</F>

 

[Paul Moore]

hi all!

i just built revision 41809 under winxp using a rather uncommon
setup (at least i think so). since i have no visual studio here,
i only used freely available tools: cygwin to get the source, the
microsoft compiler/linker and NAnt (nant.sf.net) as the build tool
to interpret the .vcproj-files (which may be the uncommon part).

This is phenomenally cool!

btw, if anyone is interested in the (rather small) build-script for
nant, just ask,

I haven't seen anyone ask, so can I? I'd love to see the build script.

Actually, beyond the script, what tool setup is needed? I have various
bits of stuff (MS Visual C++ toolkit 2003, platform SDK, Visual Studio
8, VS .NET 2003). I found a few oddities, such as problems finding a
usable msvcrt.lib (should it not have been using msvcr71.lib?) and
issues generating pythonnt_rc.h.

I sort-of got it working, and I was using a fairly old version of the
sources (it was just a quick hack, to see if I could work out what I
needed as a build script) but it would be nice to see the actual
toolset listed out.

A good place to put this information would be on the Python Wiki. I'll
put something there if I can pull together the relevant information.

Cheers,
Paul.

 

[David Murmann]

I haven't seen anyone ask, so can I? I'd love to see the build script.

sorry it took me so long, i have been busy the last couple of days,
but here it is:

"PCBuild/nant-sln.build":
---------------------------------------------------------------
<?xml version="1.0"?>
<project>
<target name="build" description="Build all targets.">
<solution configuration="release">
<projects>
<include name="make_versioninfo.vcproj" />
</projects>
</solution>
<exec program="make_versioninfo" output="pythonnt_rc.h" />

<solution configuration="release" solutionfile="pcbuild.sln">
<excludeprojects>
<include name="_bsddb.vcproj" />
</excludeprojects>
</solution>
</target>
</project>
---------------------------------------------------------------

note that bsddb is explicitly excluded because i have not figured
out how to build it without visual studio. the rest of the extensions
are more or less straightforward if you follow the instructions in
"PCBuild/readme.txt".

Actually, beyond the script, what tool setup is needed? I have various
bits of stuff (MS Visual C++ toolkit 2003, platform SDK, Visual Studio
8, VS .NET 2003). I found a few oddities, such as problems finding a
usable msvcrt.lib (should it not have been using msvcr71.lib?) and
issues generating pythonnt_rc.h.

as you can see in my .build-file i had problems with pythonnt_rc.h too.
AFAICT it should be generated by "make_versioninfo.vcproj" but somehow
it doesn't (i did not investigate further cause the above workaround
works quite well). i had no problems with any libs while building the
core python, but when building c-extensions (especially ssl was tricky
if i remember correctly).

I sort-of got it working, and I was using a fairly old version of the
sources (it was just a quick hack, to see if I could work out what I
needed as a build script) but it would be nice to see the actual
toolset listed out.

the tools i installed were:
- cygwin (with appropriate tools like svn)

- visual c++ toolkit

- nant

- the platform sdk (you dont need it complete but i dont know exactly what
parts you need...)

- platform sdk for sp2 (i dont know whether you need this at all)

- MS .NET framework redistributable

that should be it, but i might be forgetting something.
you then need to set some environment variables like PATH, LIB and INCLUDE
to point to the directories in the different packages.

 

[Paul Moore]

sorry it took me so long, i have been busy the last couple of days,
but here it is:

[...]

No problem, thanks for providing it.

I'll give it a go, from scratch, and write up the resulting process on
the Python Wiki. I assume that's OK with you?

Now watch how long it takes me, to see what a *really* slow response
looks like!

Paul.