Compiling/Installing Python 2.7 on OSX 10.6

[Jeremy]

I'm having trouble installing Python 2.7 on OSX 10.6 I was able to
successfully compile it from source, but ran into problems when I did
make install. The error I got (I received many similar errors) was:

/usr/bin/install -c -m 644 ../LICENSE /home/jlconlin/Library/
Frameworks/Python.framework/Versions/2.7/lib/python2.7/LICENSE.txt
PYTHONPATH=/home/jlconlin/Library/Frameworks/Python.framework/Versions/
2.7/lib/python2.7 DYLD_FRAMEWORK_PATH=/home/jlconlin/src/Python-2.7/
build: \
./python -Wi -tt /home/jlconlin/Library/Frameworks/Python.framework/
Versions/2.7/lib/python2.7/compileall.py \
-d /home/jlconlin/Library/Frameworks/Python.framework/Versions/2.7/
lib/python2.7 -f \
-x 'bad_coding|badsyntax|site-packages|lib2to3/tests/data' \
/home/jlconlin/Library/Frameworks/Python.framework/Versions/2.7/lib/
python2.7
Listing /home/jlconlin/Library/Frameworks/Python.framework/Versions/
2.7/lib/python2.7 ...
Compiling /home/jlconlin/Library/Frameworks/Python.framework/Versions/
2.7/lib/python2.7/._BaseHTTPServer.py ...
Sorry: TypeError: ('compile() expected string without null bytes',)
Compiling /home/jlconlin/Library/Frameworks/Python.framework/Versions/
2.7/lib/python2.7/._Bastion.py ...
Sorry: TypeError: ('compile() expected string without null bytes',)
Compiling /home/jlconlin/Library/Frameworks/Python.framework/Versions/
2.7/lib/python2.7/._CGIHTTPServer.py ...
Sorry: TypeError: ('compile() expected string without null bytes',)

As you can see I am compiling/installing in my home directory instead
of for the whole system. The script I used to compile Python 2.7 is:

#! /bin/sh

export CFLAGS="-arch x86_64"
export LDFLAGS="-arch x86_64"

.../configure --prefix=$HOME/usr/local \
--enable-framework=$HOME/Library/Frameworks \
--disable-toolbox-glue \
MACOSX_DEPLOYMENT_TARGET=10.6

make
make install

Can anyone help me fix the install error?

Thanks,
Jeremy

PS. Python compiled correctly, but a few modules were not found/made
but I don't think they are important.

Python build finished, but the necessary bits to build these modules
were not found:
_bsddb dl gdbm
imageop linuxaudiodev ossaudiodev
spwd sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for
the module's name.

 

[Ned Deily]

I'm having trouble installing Python 2.7 on OSX 10.6 I was able to
successfully compile it from source, but ran into problems when I did
make install. The error I got (I received many similar errors) was:

/usr/bin/install -c -m 644 ../LICENSE /home/jlconlin/Library/
Frameworks/Python.framework/Versions/2.7/lib/python2.7/LICENSE.txt
PYTHONPATH=/home/jlconlin/Library/Frameworks/Python.framework/Versions/
2.7/lib/python2.7 DYLD_FRAMEWORK_PATH=/home/jlconlin/src/Python-2.7/
build: \
./python -Wi -tt /home/jlconlin/Library/Frameworks/Python.framework/
Versions/2.7/lib/python2.7/compileall.py \
-d /home/jlconlin/Library/Frameworks/Python.framework/Versions/2.7/
lib/python2.7 -f \
-x 'bad_coding|badsyntax|site-packages|lib2to3/tests/data' \
/home/jlconlin/Library/Frameworks/Python.framework/Versions/2.7/lib/
python2.7
Listing /home/jlconlin/Library/Frameworks/Python.framework/Versions/
2.7/lib/python2.7 ...
Compiling /home/jlconlin/Library/Frameworks/Python.framework/Versions/
2.7/lib/python2.7/._BaseHTTPServer.py ...
Sorry: TypeError: ('compile() expected string without null bytes',)
Compiling /home/jlconlin/Library/Frameworks/Python.framework/Versions/
2.7/lib/python2.7/._Bastion.py ...
Sorry: TypeError: ('compile() expected string without null bytes',)
Compiling /home/jlconlin/Library/Frameworks/Python.framework/Versions/
2.7/lib/python2.7/._CGIHTTPServer.py ...
Sorry: TypeError: ('compile() expected string without null bytes',)

How did you obtain and unpack the source? It looks like you used
something that created the old-style "._" hidden forks when extracting
the source. The path names look a little suspicious, too:
/home/jlconlin. What file system type are these files on? You
shouldn't run into problems if you use an HFS+ file system (for
instance) and extract the tarball from the command line using
/usr/bin/tar.

PS. Python compiled correctly, but a few modules were not found/made
but I don't think they are important.

Python build finished, but the necessary bits to build these modules
were not found:
_bsddb dl gdbm
imageop linuxaudiodev ossaudiodev
spwd sunaudiodev

Yes, all of those are to be expected on an OS X 64-bit build.

 

[Jeremy]

How did you obtain and unpack the source?  It looks like you used
something that created the old-style "._" hidden forks when extracting
the source.  

I downloaded the source from python.org and extracted with 'tar -xzvf
Python-2.7.tgz' My home space is on some network somewhere. I think
the network filesystem creates the ._ at the beginning of the files.
It's really quite annoying.

The path names look a little suspicious, too:
/home/jlconlin.  What file system type are these files on?  You
shouldn't run into problems if you use an HFS+ file system (for
instance) and extract the tarball from the command line using
/usr/bin/tar.

I am intentionally installing in my home directory (i.e., /home/
jlconlin) because I don't have access to /usr/local. Supposedly this
is possible, and in fact common.

Yes, all of those are to be expected on an OS X 64-bit build.

Is it safe to ignore these modules then?

Thanks,
Jeremy

 

[Ned Deily]

I downloaded the source from python.org and extracted with 'tar -xzvf
Python-2.7.tgz' My home space is on some network somewhere. I think
the network filesystem creates the ._ at the beginning of the files.
It's really quite annoying.

It is and really shouldn't be happening. If I understand correctly,
whoever administers your system is doing its users a disservice by
putting OS X home directories on such a file system.

I am intentionally installing in my home directory (i.e., /home/
jlconlin) because I don't have access to /usr/local. Supposedly this
is possible, and in fact common.

It is common and not normally a problem. I was just noting that the
path name was not the OS X default of /Users/jlconlin.

That said, there are a couple of options. Either find another file
system to install to or, after extracting, you may be able to delete the
spurious '._' files by a judicious use of find (-name '\.\_*' perhaps),
or you could probably just ignore all the "compiling" errors. Those
aren't "compile" errors in the sense of C compiler errors; rather they
are from one of the final install steps that produces optimized .pyc and
..pyo versions of all of the standard library .py files. The ._ files
aren't python files but they do end in .py so compileall mistakenly
tries to bytecompile them, too.

Is it safe to ignore these modules then?

Yes.

 

[Philip Semanchuk]

It is and really shouldn't be happening. If I understand correctly,
whoever administers your system is doing its users a disservice by
putting OS X home directories on such a file system.


It is common and not normally a problem. I was just noting that the
path name was not the OS X default of /Users/jlconlin.

That said, there are a couple of options. Either find another file
system to install to or, after extracting, you may be able to delete the
spurious '._' files by a judicious use of find (-name '\.\_*' perhaps),
or you could probably just ignore all the "compiling" errors. Those
aren't "compile" errors in the sense of C compiler errors; rather they
are from one of the final install steps that produces optimized .pyc and
.pyo versions of all of the standard library .py files. The ._ files
aren't python files but they do end in .py so compileall mistakenly
tries to bytecompile them, too.

You might want to try this before running tar to see if it inhibits the .._ files:
export COPYFILE_DISABLE=True


I know that tells tar to ignore those files (resource forks, no?) when building a tarball. I don't know if it helps with extraction though.

Good luck
Philip

 

[Ned Deily]

You might want to try this before running tar to see if it inhibits the ._
files:
export COPYFILE_DISABLE=True


I know that tells tar to ignore those files (resource forks, no?) when
building a tarball. I don't know if it helps with extraction though.

Interesting. It's been so long since I've had to deal with ._ 's (which
is where metadata for extended attributes including resource forks are
stored), I had forgotten about that poorly documented option for 10.5
and 10.6.

A little experiment: from OS X 10.6, I NFS-mount a remote Linux (ext3)
file system and have created files on it with extended attributes.
Using ls on either the OS X or the Linux side, the ._ files appear as
regular files. On the Linux side, I use gnu tar to archive the files
and move that archive back to OS X. If I then use the stock Apple 10.6
tar to extract that archive to an HFS+ directory, the extended
attributes are by default restored properly (they can be viewed with ls
-l@) and no '._' files - great! If I first export
COPYFILE_DISABLE=True, then the tar extraction appears to ignore the ._
files: the extended attributes are not set and the ._ files still do not
appear.

So the COPYFILE_DISABLE trick may very well work for this issue. It
still raises the question of why the ._ files are being created in the
first place. They shouldn't be on the python.org tarball so it would
seem most likely they are due to some operation on the OS X machine that
causes extended attributes to be created. Nothing wrong with that, just
kind of interesting.

 

[Jeremy]

Interesting.  It's been so long since I've had to deal with ._ 's (which
is where metadata for extended attributes including resource forks are
stored), I had forgotten about that poorly documented option for 10.5
and 10.6.

A little experiment: from OS X 10.6, I NFS-mount a remote Linux (ext3)
file system and have created files on it with extended attributes.  
Using ls on either the OS X or the Linux side, the ._ files appear as
regular files.  On the Linux side,  I use gnu tar to archive the files
and move that archive back to OS X.  If I then use the stock Apple 10.6
tar to extract that archive to an HFS+ directory, the extended
attributes are by default restored properly (they can be viewed with ls
-l@) and no '._' files - great!  If I first export
COPYFILE_DISABLE=True, then the tar extraction appears to ignore the ._
files: the extended attributes are not set and the ._ files still do not
appear.

So the COPYFILE_DISABLE trick may very well work for this issue.  It
still raises the question of why the ._ files are being created in the
first place.  They shouldn't be on the python.org tarball so it would
seem most likely they are due to some operation on the OS X machine that
causes extended attributes to be created.  Nothing wrong with that, just
kind of interesting.

What I have done is perform the installation on a local hard drive
(not network storage). This prevents any ._* files from being
created. Now I just have to copy the installation to ~/Library/
Frameworks or just link to the local copy. I started the compilation
when I left, tomorrow I'll finish up and see how it went. I don't
anticipate any more problems.

Thanks,
Jeremy