Some of this has been said, some of it hasn't. I'm also not 100% on
everything here, but please discuss. This is poorly understood and
needs documenting in order to really progress on the issue. Too many
people I've heard saying "bah windows is too much of a pain" - but go
onto rubyforge, and look at the OCI download numbers. Whatever you're
feelings, that's far from irrelevant. The fact that only a handful of
rubyists are actually capable of building ruby + all of ext/* is quite
sickening, really (at least to me).
No point in keeping this discussion in the Zed thread...
Absolutely. I have been thinking about raising this topic for a long
time, but I'm also working on fixing some parts of it, so I was
holding off until I have time to complete that work. I guess though,
as it's here, now is as good a time as any. I've approached a few
people for assistance on the ideas in the past, but we're all busy
trying to survive ourselves to really get things going (yet). Even
with a lack of time, if some people want to group together, I'd be
more than happy to donate what I can to getting a more stable
environment up on the Windows platform - and that includes code I've
written against the issues.
I could be wrong, but I sense a deeper question in Thurfir's message.
What is it about Linux (or *BSD or Mac OS X) that avoids all
the compilation problems that arise in the Windows environment?
Can a Windows expert confirm my understanding that:
I am by no means an expert, but I have been studying this recently,
and have come to some understanding of the issues. I'm not much of a C/
C++ programmer, but I have done a lot of building, and sent /
committed patch files to build processes of quite a number of
projects. I also actively use Windows, *nix (in many flavours), and
more recently OS X.
Unlink Linux (and *BSD, Mac OS X...), Windows does not come packaged
with a development environment and in fact there are multiple
mutually-incompatible development environments (Cygwin, MinGW, VC6,
VC8). For the most part you can not mix-and-match object code (dlls)
created by these different environments with the biggest problem
being that there is no common memory allocation library.
Well, actually OS X doesn't come "pre-packaged" either, but the build
chain there is based on Open Source stuff from the *nix world, and for
ruby, most of the pre-requisites are already available as libs
(readline, curses, openssl, zlib, etc). Once you install Xcode, you
pretty much get a production grade compiler setup for Ruby that will
Just Work.
Under Windows, you can compile 1.8.6 trivially on any of the compilers
mentioned. Ruby herself, in 1.8.6 is a very very clean compilation and
Just Works. The ext/* stuff however, is a different story. OpenSSL
compilation on Windows is actually a real pain - you need to get a
copy of perl if you want to use the MS compilers (if you're using the
pre-packaged build chain that is), and under MinGW you have to rebuild
the directory structure by hand, as the symlinks in the tarball will
not expand properly. I don't know why the OpenSSL devs ship their
build packages like this, I find it terribly annoying, but it is what
it is.
As for readline, well it's broken on win32, and several of the builds
out there are archaic. GnuWin32 is useful - but again, stupid choices
make life difficult, such as the pre-compiled binaries being hardcoded
(!!!) to particular locations on disk. It's just insane - and this
isn't the fault of the operating system - this is third party
craziness. Other libs have similar issues, I won't got through the
whole ext/* stack now. zlib is easier. Getting just rubygems + ruby up
and running is quite easy, if you use one of the many pre-compiled
openssl builds out there and build + link against that.
As for the MSVCRT issues, I have read many many mixed reports. The
summary is roughly like this, AFAIK:
- You *can* mix msvcrt versions safely *if* you don't pass structs /
pointers across the boundaries for which the api has changed. In
reality, for some applications, this is unworkable - however there are
several production ready gems out there that are working just fine on
the OCI (MSVCRT) which are compiled with VS 2005 (MSVCR7/8 (I can't
remember the exact version -> VS pairings, sorry).
- Due to the above, and several apps having issues with it - as well
as the bloat it causes, no one likes mixing up MSVCR versions. That's
fair enough.
- I have been told by several MS developers (whether or not this is
true, someone with more knowledge of the linking strategies and
A[PB]Is will need to confirm - that it's possible to make these things
safe using particularly compiled .lib files. I've also heard stories
of shims being put in place, but they may come with a performance
overhead, and are complex low level things in this domain (I suspect
also quite specific to particular problems).
- MinGW and MSVCRT (VS 6) are compatible.
- Performance on MinGW is up about 30% over VS6, for 1.8.6. The gap
is closing in 1.9, and isn't so bad for later MS compilers.
- I haven't tried Intels compiler, and someone probably should,
although it's buyware.
- MinGW has cross-compiler capabilities, and that might be worth
noting here. I know there are a couple of gems out there that are
built like this. If possible, it might make sense to make something
available for building pre-compiled gems like this. The problem is, C
toolchains and dependancies are such a PITA very often.
- In other build environments, I have built software that doesn't
link against the standard runtimes locally at all. I need to become
more familiar with the build tools and ABI to say more than that though.
- There are bugs in the 1.8.6 ext/* makefiles which means that
subshells spawned seem to use different relative / absolute paths in
different ways. In order to compile everything off of a single 'make'
under mingw, I found I had to add -I [absolute_path] AND -I
[relative_path]. I think the same errors have thrown quite a few
people who later just gave up. IIRC it's openssl vs. readline that
don't compile properly - and those are a harder two anyway, so people
get angry and give up very rapidly.
- The version of bison in MinGW doesn't build parse.c properly
anymore :-(
There is absolutely *nothing* stopping the ruby community getting into
MinGW and helping everyone out by providing better build chains for
them based on ruby apps and capabilities. Seriously, rubys core
interpreter (1.8.6) builds really easily on top of the standard lib
under mingw and other OSes, and isn't packed up as share/buy ware like
the most commonly use perl / python builds on win32 - that's important
to some people (whatever you might think of that as a merit yourself).
More than this though, a dose of pragmatic powerful scripting
capability could go a *long* way in helping MinGW move forward, so if
anyone is interested, take a serious look at a project like this. Even
just bootstrapping the default required mingwPORT.sh files would go a
long way into making the MinGW build process simpler.
What actually needs "fixing" is we need someone to release a build
chain for the build chains (for ruby herself). I'm working on
something, but I have not had time to finish it, as I'm in a busy
startup at present, so it's only good for internal use right now. I
want to try and get into working with the rubygems folks once I'm more
familiar with the chains and issues. Also interesting to join forces
with might be the multiruby and related projects (there's a build-farm
type project somewhere too but I can't remember the details, Ryan?)
I assume this is why some Windows software comes distributed with
its own collection of 'standard' dlls compiled in the same
development environment as the underlying application.
Well, XP comes with two really, a native api, and the MSVCRT. As you
install software, at some point you'll get an app with newer MSVCR
versions being installed from the re-distributable. There are
disgusting rumors about GPL violations and other complex licensing
issues with linking against their newer libs - but this is just so
totally anti-pragmatic. I found the Wireshark toolchain documentation
very good - and something we should aspire too as a base minimum.
The end result is that the least-common-denominator for all these
environments is source code, which still requires the 'end-user' to
install a development environment of some sort and to manage the
compile/link process for Ruby and for any and all 3rd party
libraries/gems/extensions that they need.
This is actually true of all the operating systems. It's very common
to have it there already on *nix, and many developers install Xcode
very early on in their configuration of a new OSX box, as this is
required for fink / macports / general open source software
compilation. This would be very similar on win32 if only mingw's build
chain wasn't so nasty at this point in time.
I also gather that each of those environments has very different
build utilities (the Windows equivalent of make, config, autoconf,
and so on). For an extension writer, the problem is that they can't
really know what the build process is going to look like on an
arbitrary Windows system making it very difficult to
Well, sure, but most of that actually works for a C app, at least,
there are good example builds around that people can "borrow". Going
into C++ is a different matter (there are more compiler and api
differences there - i'm also less familiar though), but lets start
with the most common first.