Alternatives to make

C

coal

I recall someone posting here about an alternative to make that
they're
developing. I've tried searching for the threads, but haven't been
able to
dig it up yet. What I remember is that they might be using C++ to
develop their tool and that the threads have been within the past two
or
three years. I vaguely recall there being few replies to one of the
threads.

I need to replace make. If you have any suggestions on that or can
help
me find those threads I mentioned, please let me know. Thanks in
advance.

Brian Wood
Ebenezer Enterprises
www.webEbenezer.net
 
J

joshuamaurice

I recall someone posting here about an alternative to make that
they're
developing.  I've tried searching for the threads, but haven't been
able to
dig it up yet.  What I remember is that they might be using C++ to
develop their tool and that the threads have been within the past two
or
three years.  I vaguely recall there being few replies to one of the
threads.

I need to replace make.  If you have any suggestions on that or can
help
me find those threads I mentioned, please let me know.  Thanks in
advance.

Although a little off topic, might I suggest reconsidering for any
sizable code base? What's wrong with make?

My company recently switched to Maven, and I regret it every day. I
miss Make's ability to do a single build on multiple threads and being
able to do the minimal build necessary from current changes, aka
actual incremental builds. Make is designed with 1 thing in mind:
building your code as fast as possible, and when properly used, it
does so quite well. This time saved in builds directly translates to
saved developer time, which is generally something of which you want
to maximize utility. (Moreover, Make is actually a stable product,
whereas Maven is bug ridden.)
 
J

James Kanze

On Apr 20, 10:12 pm, (e-mail address removed) wrote:
Although a little off topic, might I suggest reconsidering for
any sizable code base? What's wrong with make?

Which make? Almost every make I've seen is different. About
the only thing they have in common is the fact that whether a
line starts with a tab character or spaces is significant, which
I wouldn't consider something to be proud of.
My company recently switched to Maven, and I regret it every
day. I miss Make's ability to do a single build on multiple
threads and being able to do the minimal build necessary from
current changes, aka actual incremental builds.

Not all versions of make can use multiple threads (or processes,
which makes more sense). On the other hand, some can use
multiple machines, spreading the load through the network.

As for the "minimal necessary build", I've never heard of a make
that could do that---change a comment in a header, and either
make will recompile everything which uses the header, or it
won't recompile things when you change something which requires
recompilation. Make's algorithm for managing this recompilation
has a very large granularity, with a very, very primitive notion
of "significant change", and historically depended on signficant
user input to determine dependencies (although most modern makes
have means of at least partially automating this, in
collaboration with the compiler).
 
D

Dilip

I recall someone posting here about an alternative to make that
they're
developing.  

I believe Google uses SCons (http://www.scons.org/) but its built in
Python, so that may not be exactly what you are looking for. It does
satisfy the "replacement for make" part though.
 
N

Noah Roberts

I recall someone posting here about an alternative to make that
they're
developing. I've tried searching for the threads, but haven't been
able to
dig it up yet. What I remember is that they might be using C++ to
develop their tool and that the threads have been within the past two
or
three years. I vaguely recall there being few replies to one of the
threads.

I need to replace make. If you have any suggestions on that or can
help
me find those threads I mentioned, please let me know. Thanks in
advance.

CMake is nice but not as a replacement for make, but a makefile
generator. Sort of like automake but much, much better.
 
J

Jorgen Grahn

I recall someone posting here about an alternative to make that
they're
developing.
[...]
I need to replace make.  If you have any suggestions on that or can
help
me find those threads I mentioned, please let me know.  Thanks in
advance.

I think if you look in the Wikipedia, you'll find a number of
"make killers", including the one you remember. But ...
Although a little off topic, might I suggest reconsidering for any
sizable code base? What's wrong with make?

My company recently switched to Maven, and I regret it every day. I
miss Make's ability to do a single build on multiple threads

Multiple concurrent processes, you mean. Not present in all make
implementations, but perhaps you mean GNU Make and assume everyone
should standardize on that one. (And perhaps you are right.)
and being
able to do the minimal build necessary from current changes, aka
actual incremental builds. Make is designed with 1 thing in mind:
building your code as fast as possible, and when properly used, it
does so quite well.

The key here is "properly used". Most makefiles I've seen are not done
properly. They rarely describe the actual dependencies, and there is
often one make invocation per source code directory ("recursive make").

If that is the poster's problem, I recommend trying to use make
efficiently before discarding it.

/Jorgen
 
J

joshuamaurice

Which make?  Almost every make I've seen is different.  About
the only thing they have in common is the fact that whether a
line starts with a tab character or spaces is significant, which
I wouldn't consider something to be proud of.


Not all versions of make can use multiple threads (or processes,
which makes more sense).  On the other hand, some can use
multiple machines, spreading the load through the network.

Yes, standardize on a newer version of GNU Make and be done with it.
As for the "minimal necessary build", I've never heard of a make
that could do that---change a comment in a header, and either
make will recompile everything which uses the header, or it
won't recompile things when you change something which requires
recompilation.

I'm sorry that Make is not magic. Yes, if you change a reader, you
should recompile all things which use that header. If this becomes a
significant burden, consider the pimpl idiom and other such idioms
expressly designed to lower such dependencies and thus lower compile
times for incremental builds.
Make's algorithm for managing this recompilation
has a very large granularity,

And that is the key. Recursive Make Considered Harmful:
http://aegis.sourceforge.net/auug97.pdf
Make works best at the highest granularity. This allows it to better
schedule things concurrently, among other things. The paper describes
it better than I can.
with a very, very primitive notion
of "significant change",

Yes. I'll give you this "flaw" in Make. It only understands "out of
date" and "up to date", which may require some additional user work to
differentiate between other cases where you don't need to rebuild some
file from scratch and only need to update it slightly. GNU Make
understands this, and provides things like .PRECIOUS to help
facilitate that.
and historically depended on signficant
user input to determine dependencies (although most modern makes
have means of at least partially automating this, in
collaboration with the compiler).

Yes, that's the key to make a good Make build system for C++ code,
automatic tracking of header dependencies. g++ as is allows you to do
this in one line of makefile. Otherwise you might need to write a
small script / program to act like the c preprocessor to get all
includes, but that's not terribly hard, and probably available online.
(See the Recursive Make Considered Harmful paper.)
 
C

coal

Although a little off topic, might I suggest reconsidering for any
sizable code base? What's wrong with make?

I want to be able to return 1 from main() to indicate success
and 0 to indicate failure. Make incorrectly, in my opinion,
interprets 1 to be a failure and zero to be success. I found
in researching the problem that if you precede a command with
a '-', make will ignore non-zero values from the command.
That helps, but make also needs to know a zero value
indicates an error with the command. I haven't found a way
to get that to work, so I'm looking for a more flexible
build tool.

I'm working on a program that automates interaction with
the C++ Middleware service. In other words, rather than
having to go to the web page to use the service, it's
integrated into the build process. I have Linux and
Windows versions of the program working. I've only tested
the Windows version on the command line and not in any
sort of build environment. The Linux version is here --
http://webEbenezer.net/build_integration.html

The Windows version should be added to that page in a day
or two. Please keep in mind that this is a first draft
and is kind of rough. For example, the reading of the
config file uses strtok and expects that the fields in
the file be in a specific order. G-d willing, we'll
be able to revisit some of that later. But for now
it seems more important to include compression and
encryption support first. I'm not very familiar with
either of those topics. From what I can tell from
looking at openssl.org, they first compress and then
encrypt. Is that the order that is generally followed?
Also, if someone would like to help me add the ssl code
to direct.cc, I'll acknowledge you on the website and
if you have a site, I'll add a link to your site.

A number of years ago I used gzip to compress the
output and then sent an email with an attachment.
I abandoned that and have just been using the web
interface for my own development purposes. Now that
I have the command line interface working, I want to
integrate compression again. Does anyone have
recommendations on C++ compression libraries? The
library needs to be available on at least Linux and Windows.


Brian Wood
Ebenezer Enterprises
www.webEbenezer.net
 
C

coal

I recall someone posting here about an alternative to make that
they're
developing.
[...]
I need to replace make.  If you have any suggestions on that or can
help
me find those threads I mentioned, please let me know.  Thanks in
advance.

I think if you look in the Wikipedia, you'll find a number of
"make killers", including the one you remember. But ...

First, thanks to Sean for the link to Wikipedia. I looked at
it for a few minutes and was kind of disappointed because I
didn't find the one I was looking for. I only paged through
the first few sections though. So thanks for mentioning it
again, because I went back and looked more closely and under
"Build automation software table (draft)" I found it. It
is called makepp. Here's a link to a 2008 thread in this
newsgroup by the author of makepp:
http://tinyurl.com/cqk225

I'm not sure if it will do what I want, but it sounds like
it's better than gnu make.


Brian Wood
Ebenezer Enterprises
www.webEbenezer.net
 
P

peter koch

I want to be able to return 1 from main() to indicate success
and 0 to indicate failure.  
That is plain stupid. Typically, the returnvalue is an errorcode with
0 meaning no error.
Make incorrectly, in my opinion,
interprets 1 to be a failure and zero to be success.
I have heard of many stupid opinions, but this one certainly is the
worse ones.

/Peter
 
C

coal

First, thanks to Sean for the link to Wikipedia.  I looked at
it for a few minutes and was kind of disappointed because I
didn't find the one I was looking for.  I only paged through
the first few sections though.  So thanks for mentioning it
again, because I went back and looked more closely and under
"Build automation software table (draft)"  I found it.  It
is called makepp.  Here's a link to a 2008 thread in this
newsgroup by the author of makepp:http://tinyurl.com/cqk225

I'm not sure if it will do what I want, but it sounds like
it's better than gnu make.

Hmm. I tried downloading makepp the stable version and
now I'm having some problems. When I tried to go to the
Download directory it said something about a stale NFS
file and I couldn't cd to that directory. I decided to
reboot and now after I enter my password (this is on Linux)
it says, "Could not update ICEauthority file." I'm not
sure if the problem is specific to my circumstance or had
something to do with the download. Anyway, just wanted
to warn people to be careful about it.

Brian Wood
Ebenezer Enterprises
www.webEbenezer.net
 
J

joshuamaurice

That is plain stupid. Typically, the returnvalue is an errorcode with
0 meaning no error.


I have heard of many stupid opinions, but this one certainly is the
worse ones.

Harsh, but generally correct. In the programming world of C++, 0 is
false and 1 is true. However, in the process world, 0 means success
and nonzero means failure, and shells assume this, for example the
shell's && and ||. `foo && echo 1` will echo 1 iff foo returns 0. It's
the convention, so you'll have to deal with it.
 
C

coal

From what I can tell from
looking at openssl.org, they first compress and then
encrypt. Is that the order that is generally followed?

Here's a some info on the topic of using both compression
and encryption in an application.
http://blogs.msdn.com/drnick/archive/2008/09/03/composing-compression-and-encryption.aspx

A number of years ago I used gzip to compress the
output and then sent an email with an attachment.
I abandoned that and have just been using the web
interface for my own development purposes.  Now that
I have the command line interface working, I want to
integrate compression again.  Does anyone have
recommendations on C++ compression libraries?  The
library needs to be available on at least Linux and Windows.

Unless something better pops up I think I'm going to give
bzip -- www.bzip.org -- a whirl. I'd prefer a C++ library,
but bzip seems pretty good for a C library.

Next question. Would it make sense to concatenate all
of the input files (potentially numerous header files and
a middle file) and then compress that file or compress
each file separately? My guess is the former would
be better.


Brian Wood
Ebenezer Enterprises
www.webEbenezer.net
 
F

Fred Zwarts

Yes, that's the key to make a good Make build system for C++ code,
automatic tracking of header dependencies. g++ as is allows you to do
this in one line of makefile. Otherwise you might need to write a
small script / program to act like the c preprocessor to get all
includes, but that's not terribly hard, and probably available online.
(See the Recursive Make Considered Harmful paper.)

I agree with this. The problem with using a separate script/program is to
get all predefined macros and include directories correct. Some compilers
also use headers that cannot be found as normal files, to complicate things.

What I am missing however is an automatic way to track the object
dependencies for linking my programs.
I have an object library (archive in Linux terms) with objects for utility classes and
functions, which is used in many main programs.
If I make the executable dependent on the archive, then all programs are linked again,
if only one object is replaced in the archive. What I want is that if one of the
objects in the archive is replaced, only those programs are linked that actually use
the object.
What I do now is to let the linker create a linker map and I use a script to parse
the linker map and create the dependencies. This saves a lot of linking time in
many cases. A problem is that the linker map sometimes changes its format with
new versions of the linker, as it is meant to be human readable.
I really miss a linker flag to create such an dependency file, comparable with that of
the compiler to create a dependency file for headers.
 
J

James Kanze

Yes, standardize on a newer version of GNU Make and be done
with it.

That's what I do, for various reasons. But when you say "make",
I would generally interpret it to mean generic make; the common
subset which works in all (or almost all) makes. Regretfully,
that subset is pretty small, and very painful to work with.

(GNU make is Turing complete, and has a function which evaluates
the expansion of text, which means that it can be used to
implement a full fledged build system.)
I'm sorry that Make is not magic. Yes, if you change a reader,
you should recompile all things which use that header. If this
becomes a significant burden, consider the pimpl idiom and
other such idioms expressly designed to lower such
dependencies and thus lower compile times for incremental
builds.

The compilation firewall idiom isn't really relevant when it
comes to changing a comment in a header.

It's quite imaginable to conceive of a system which really did
minimum recompiles---I believe Taligent was working on one when
they closed down, and I think Visual Age does this partially as
well. But it does suppose that the tool in question "know" C++;
that it be integrated with the compiler, more or less, which
isn't really the case for very many tools I know. Most of the
tools are, in fact, worse than make in this respect. (Boost
uses Jam, for example, which pretty much guarantees a less than
minimal compilation---some files that need to be compiled won't
be.)
And that is the key. Recursive Make Considered
Harmful:http://aegis.sourceforge.net/auug97.pdf
Make works best at the highest granularity. This allows it to
better schedule things concurrently, among other things. The
paper describes it better than I can.

The paper is a good example of someone writing about something
he doesn't know or understand. But that's not the point. The
smallest granularity make can deal with is the file---a file has
been modified, or it hasn't. It can't determine whether the
modification will require recompilation or not, because it
doesn't know anything about the files contents. So if you
modify an inline function in a header, all sources which include
that header (directly or indirectly) will be recompiled, and not
just the sources which use the inline function which was
modified.
Yes. I'll give you this "flaw" in Make. It only understands
"out of date" and "up to date", which may require some
additional user work to differentiate between other cases
where you don't need to rebuild some file from scratch and
only need to update it slightly. GNU Make understands this,
and provides things like .PRECIOUS to help facilitate that.

Even GNU make only works at the file level. In practice,
however, not all modifications of a header require the
recompilation of all sources which include it. (And I don't see
how .PRECIOUS is relevant to this.)
Yes, that's the key to make a good Make build system for C++
code, automatic tracking of header dependencies. g++ as is
allows you to do this in one line of makefile. Otherwise you
might need to write a small script / program to act like the c
preprocessor to get all includes, but that's not terribly
hard, and probably available online. (See the Recursive Make
Considered Harmful paper.)

It takes a little more than one line. In fact, it takes some
collaboration from the compiler, and some shell behind it. But
it does work, and it works correctly (unlike most other tools
I've seen), because it depends on the compiler to find the
dependencies, rather than trying to reimplement the compiler's
logic internally.
 
J

James Kanze

I agree with this. The problem with using a separate
script/program is to get all predefined macros and include
directories correct. Some compilers also use headers that
cannot be found as normal files, to complicate things.

You agree with what? It takes more than one line in the
makefile to build the dependencies. (For that matter, the GNU
make documentation describes several different ways to do it,
depending on your goals.)

The important part about how GNU make and some other makes
handle this is that they don't handle it themselves. They leave
it up to the compiler. Some compilers, like g++, are designed
with this in mind, and you don't need very much at all make it
work. Whereas others... with VC++, I invoke the compiler with
/E, then pass the output through a a number of other programs to
filter it, e.g.:

cl /E ${cppFlags} /Tp ${fn} |
sed -n 's:^ *# *line *[0-9]* *"\(.*\)".*:\1:p' |
sed 's:\\\\:/:g' |
egrep -v '/Microsoft *.*/VC/' |
egrep -v '/Program Files/' |
sort -u |
sed "s|^|${objectName} : |"
What I am missing however is an automatic way to track the
object dependencies for linking my programs.

That shouldn't be a problem. You have to specify what you're
linking anyway, so the information is already available in the
makefile. It's up to you to use it. (With GNU make, you can
use $(eval $(call ...)) to only specify it once, as an argument
to the function, which then arranges for it to appear both in
the dependencies and in the link command.
I have an object library (archive in Linux terms) with objects
for utility classes and functions, which is used in many main
programs. If I make the executable dependent on the archive,
then all programs are linked again, if only one object is
replaced in the archive. What I want is that if one of the
objects in the archive is replaced, only those programs are
linked that actually use the object.

That's the granularity problem. The granularity of make is a
file; a target can only depend on a file, not something smaller,
and any change in that file will provoke the rebuilding of the
target.
What I do now is to let the linker create a linker map and I
use a script to parse the linker map and create the
dependencies. This saves a lot of linking time in many cases.
A problem is that the linker map sometimes changes its format
with new versions of the linker, as it is meant to be human
readable. I really miss a linker flag to create such an
dependency file, comparable with that of the compiler to
create a dependency file for headers.

The problem, probably, is that most linkers don't support this.
But I see you've found your work-around, the same way I did for
headers and VC++. (Now, of course, someone will point out to me
a simple flag I missed, which will cause VC++ to output the
dependencies like I want:). After all, if the IDE can do it,
the compiler must support it somehow.)
 
J

James Kanze

On Apr 21, 10:18 pm, (e-mail address removed) wrote:
[...]
I want to be able to return 1 from main() to indicate success
and 0 to indicate failure.

The C and C++ standards say that 0 must indicate success. It's
not a question of make, or anything else for that matter---it's
what the standard requires.
Make incorrectly, in my opinion, interprets 1 to be a failure
and zero to be success.

Make normally adopts the conventions of the platform it is on.
Under Unix, 0 is success, and anything else is failure, and
there's no way you can change this, other than rewriting every
utility program on the system, all of the shells, the compilers,
etc., etc.
I found in researching the problem that if you precede a
command with a '-', make will ignore non-zero values from the
command. That helps, but make also needs to know a zero value
indicates an error with the command. I haven't found a way to
get that to work, so I'm looking for a more flexible build
tool.

Make treats non 0 return values as an error because all of the
programs it will invoke will return a non-0 value for an error,
and 0 if there are no errors.
 
J

James Kanze

What on Earth for?

Implementing a simplified make is a good exercise for someone
just learning programming---it's one of the simplest practical
examples I know of something requiring dynamic allocation and
pointers, along with some parsing (to read the dependency file).

Implementing a full featured make, along the lines of GNU make,
is a different question, and if I needed something GNU make
didn't provide (distributed builds, for example), I'd look for
something already implemented before I'd attack it myself.

(Make's grammar is one of the ugliest things around, of course.
But then, C++'s isn't any real beauty, either. But both get the
job done.)
 
M

Michael DOUBEZ

James said:
That shouldn't be a problem. You have to specify what you're
linking anyway, so the information is already available in the
makefile. It's up to you to use it. (With GNU make, you can
use $(eval $(call ...)) to only specify it once, as an argument
to the function, which then arranges for it to appear both in
the dependencies and in the link command.

Can you elaborate on that ? That looks interesting.

[snip]
The problem, probably, is that most linkers don't support this.
But I see you've found your work-around, the same way I did for
headers and VC++. (Now, of course, someone will point out to me
a simple flag I missed, which will cause VC++ to output the
dependencies like I want:). After all, if the IDE can do it,
the compiler must support it somehow.)

Something like /showincludes greping on "Note: including file:" on error
standard output :)
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,774
Messages
2,569,599
Members
45,169
Latest member
ArturoOlne
Top