Best Way to Compile Large Project

Discussion in 'C Programming' started by David T. Ashley, Dec 28, 2006.

  1. I have a large project on a Linux box that may eventually have several
    hundred 'C' source files, which need to be compiled and linked into several
    executables.

    Not all of the executables will use all of the files (i.e. there would be no
    linker references to some symbols).

    The project will naturally be broken down into several or more
    subdirectories.

    Questions:

    a)What is the best way in general to build? Compile all the .c files into
    corresponding .o files, and then link the .o files explicitly?

    b)Any thoughts on doing this with "make"?

    c)Any thoughts on the limitations of the linker?

    Thanks.
    David T. Ashley, Dec 28, 2006
    #1
    1. Advertising

  2. "David T. Ashley" <> writes:

    > I have a large project on a Linux box that may eventually have several
    > hundred 'C' source files, which need to be compiled and linked into several
    > executables.
    >
    > Not all of the executables will use all of the files (i.e. there would be no
    > linker references to some symbols).
    >
    > The project will naturally be broken down into several or more
    > subdirectories.
    >
    > Questions:
    >
    > a)What is the best way in general to build? Compile all the .c files into
    > corresponding .o files, and then link the .o files explicitly?


    Yes. An alternative, if the project is big, with subparts evolving at
    different paces, would be to make libraries of subprojects (modules).

    Also, it could be faster to build the libraries and link several
    programs with a choice of these libraries, rather than linking these
    programs from all the object files everytime. (Better benchmark it,
    it'd depend on the project).


    > b)Any thoughts on doing this with "make"?


    Yes. But read this first:

    http://members.canb.auug.org.au/~millerp/rmch/recu-make-cons-harm.html

    ie. use only one Makefile.


    > c)Any thoughts on the limitations of the linker?


    None. I've got an empty brain. Sorry.


    --
    __Pascal Bourguignon__ http://www.informatimago.com/

    "Specifications are for the weak and timid!"
    Pascal Bourguignon, Dec 28, 2006
    #2
    1. Advertising

  3. David T. Ashley

    Guest

    David T. Ashley wrote:
    > I have a large project on a Linux box that may eventually have several
    > hundred 'C' source files, which need to be compiled and linked into several
    > executables.
    >
    > Not all of the executables will use all of the files (i.e. there would be no
    > linker references to some symbols).
    >
    > The project will naturally be broken down into several or more
    > subdirectories.
    >
    > Questions:
    >
    > a)What is the best way in general to build? Compile all the .c files into
    > corresponding .o files, and then link the .o files explicitly?
    >
    > b)Any thoughts on doing this with "make"?
    >
    > c)Any thoughts on the limitations of the linker?
    >
    > Thanks.


    <OT>
    Have you tried Eclipse and CDT?
    http://www.eclipse.org/
    http://www.eclipse.org/cdt/
    </OT>
    , Dec 28, 2006
    #3
  4. David T. Ashley

    Barry Guest

    "David T. Ashley" <> wrote in message
    news:nBGkh.37629$...
    > I have a large project on a Linux box that may eventually have several
    > hundred 'C' source files, which need to be compiled and linked into

    several
    > executables.
    >
    > Not all of the executables will use all of the files (i.e. there would be

    no
    > linker references to some symbols).
    >
    > The project will naturally be broken down into several or more
    > subdirectories.
    >
    > Questions:
    >
    > a)What is the best way in general to build? Compile all the .c files into
    > corresponding .o files, and then link the .o files explicitly?
    >
    > b)Any thoughts on doing this with "make"?
    >
    > c)Any thoughts on the limitations of the linker?
    >
    > Thanks.
    >
    >
    >


    Build tools are off topic for clc, but you could start by looking
    at make. The last time some of the folks on this board tried to
    respond to a questions about "make" several showed they had
    noclue Best done by a search and use of your man pages.

    <<OT>>
    If you have a fully installed development environment you may
    take a look at autoconf and automake. But, they come with
    caveats with you should look at as well, which may lead you
    to better tools.
    <</OT>>
    Barry, Dec 28, 2006
    #4
  5. > b)Any thoughts on doing this with "make"?
    >

    We're in a situation much like your own. More then 100 files, at least 10
    applications build from both our "framework" code and application specific
    code.

    If you consider make then try out makepp instead:
    http://makepp.sourceforge.net/

    I've used it for like 3 years for now and don't regret it at all!
    It's suppose to be a make replacement and it solves several issues that make
    has.
    E.g. it has an automatic dependency scanner and I've never suffered from
    files which were not correctly recompiled.
    Also it won't need recursing (in most cases) like traditional make.

    Besides you can write extra scripts for it in perl.

    The downside is though that it's a bit slower since it's written in perl.
    However this is a cheap price to pay. Also the latest build from CVS is
    nessecary but not always stable. I've contributed with alot of feedback to
    the project so far but usually it's only for the better.

    One more thing is that it's very portable. We use it on Windows, Linux,
    Macosx, Solaris, AIX and HP-UX and also support several cross compilers
    without additional scripts or applications.

    At least if you consider make then give this one a try.

    -- Henrik
    Henrik Goldman, Dec 28, 2006
    #5
  6. On Wed, 27 Dec 2006 22:11:31 -0500
    "David T. Ashley" <> wrote:

    > I have a large project on a Linux box that may eventually have
    > several hundred 'C' source files, which need to be compiled and
    > linked into several executables.


    That's not a large project :)

    > Not all of the executables will use all of the files (i.e. there
    > would be no linker references to some symbols).


    Then you should consider using shared objects (.so libraries).

    > The project will naturally be broken down into several or more
    > subdirectories.


    One hopes so. Also consider using a version control system. Subversion
    is pretty neat and a doddle to set up.

    > Questions:
    >
    > a)What is the best way in general to build? Compile all the .c files
    > into corresponding .o files, and then link the .o files explicitly?


    See above. One would normally group the related functions into shared
    libraries, and then link the various programs against these libraries.
    Just linking everything statically is so twentieth century, dear :)

    >
    > b)Any thoughts on doing this with "make"?


    Make is the time-honoured way to do this. You need to spend some time
    mastering its syntax, but once you know how to use it, make is a very
    powerful tool. There are alternatives to make, but they have their own
    limitations and idiosyncrasies, in addition to being niche products.

    > c)Any thoughts on the limitations of the linker?


    Unless you have some specific requirements you did not mention, rest
    assured that for a project like yours (building a few executables from
    a moderate number of source files) the Unix linkers have no
    limitations.

    --
    Stefaan A Eeckels
    --
    "We have gone from a world of concentrated knowledge and wisdom to one
    of distributed ignorance. And we know and understand less while being
    increasingly capable." Prof. Peter Cochrane, formerly of BT Labs
    (With thanks to Brian Hamilton Kelly)
    Stefaan A Eeckels, Dec 28, 2006
    #6
  7. David T. Ashley

    toby Guest

    Barry wrote:
    > "David T. Ashley" <> wrote in message
    > news:nBGkh.37629$...
    > > I have a large project on a Linux box that may eventually have several
    > > hundred 'C' source files, which need to be compiled and linked into

    > several
    > > executables.
    > >
    > > Not all of the executables will use all of the files (i.e. there would be

    > no
    > > linker references to some symbols).
    > >
    > > The project will naturally be broken down into several or more
    > > subdirectories.
    > >
    > > Questions:
    > >
    > > a)What is the best way in general to build? Compile all the .c files into
    > > corresponding .o files, and then link the .o files explicitly?
    > >
    > > b)Any thoughts on doing this with "make"?


    Yes.

    > >
    > > c)Any thoughts on the limitations of the linker?
    > >
    > > Thanks.
    > >
    > >
    > >

    >
    > Build tools are off topic for clc, but you could start by looking
    > at make. The last time some of the folks on this board tried to
    > respond to a questions about "make" several showed they had
    > noclue Best done by a search and use of your man pages.


    What also tends to be revealed by make *questions* is that the
    questioner has not read 'info make'[1], which should a first step to
    any serious use of it.

    [1] Online here: http://www.gnu.org/software/make/manual/make.html

    >
    > <<OT>>
    > If you have a fully installed development environment you may
    > take a look at autoconf and automake. But, they come with
    > caveats with you should look at as well, which may lead you
    > to better tools.
    > <</OT>>
    toby, Dec 28, 2006
    #7
  8. David T. Ashley

    Guest

    David T. Ashley wrote:

    > b)Any thoughts on doing this with "make"?


    IMHO once any project grows to anything over one or two files,
    using make is the only way to go.

    Also if you take a little bit of time to understand make files:

    http://www.zeusedit.com/forum/viewtopic.php?t=300

    you'll find make is pretty easy to use.

    Jussi Jumppanen
    Author: Zeus for Windows
    http:\\www.zeusedit.com
    , Dec 28, 2006
    #8
  9. David T. Ashley

    Logan Shaw Guest

    toby wrote:
    > What also tends to be revealed by make *questions* is that the
    > questioner has not read 'info make'[1], which should a first step to
    > any serious use of it.


    Provided you have a specific reason to use GNU make. If not, then
    reading some documentation that isn't GNU-specific might be a better
    idea.

    - Logan
    Logan Shaw, Dec 29, 2006
    #9
  10. David T. Ashley

    Logan Shaw Guest

    wrote:
    > Jussi Jumppanen
    > Author: Zeus for Windows
    > http:\\www.zeusedit.com

    ^^

    I'm sure you meant to give a legal URI there...

    - Logan
    Logan Shaw, Dec 29, 2006
    #10
  11. wrote:
    >
    > <OT>
    > Have you tried Eclipse and CDT?
    > http://www.eclipse.org/
    > http://www.eclipse.org/cdt/
    > </OT>
    >


    While I love, absolutely adore, using Eclipse for Java, my brief
    encounter with CDT made me *run* back to the shelter of vim + make.

    To be fair, I have put rather substantial effort into tuning my vim
    environment for C coding in particular, making the
    edit-compile-run-debug cycle much more pleasant than it might appear to
    a newbie during his first years with the platform.
    james of tucson, Dec 29, 2006
    #11
  12. Logan Shaw wrote:
    > toby wrote:
    >> What also tends to be revealed by make *questions* is that the
    >> questioner has not read 'info make'[1], which should a first step to
    >> any serious use of it.

    >
    > Provided you have a specific reason to use GNU make. If not, then
    > reading some documentation that isn't GNU-specific might be a better
    > idea.
    >
    > - Logan


    The OP did specify Linux, not that it necessarily indicates GNU make,
    but it is a very safe bet.
    james of tucson, Dec 29, 2006
    #12
  13. David T. Ashley

    Richard Bos Guest

    Logan Shaw <> wrote:

    > wrote:
    > > Jussi Jumppanen
    > > Author: Spam for Windows
    > > http:\\www.spamedit.com

    > ^^
    >
    > I'm sure you meant to give a legal URI there...


    I'm sure it just wanted to spam. (And apparently his product "is Y2K",
    according to his FAQ. No, not Y2K-compatible; "Y2K". My confidence in
    its excellence is not stellar.)

    Richard
    Richard Bos, Dec 29, 2006
    #13
  14. David T. Ashley

    toby Guest

    Logan Shaw wrote:
    > toby wrote:
    > > What also tends to be revealed by make *questions* is that the
    > > questioner has not read 'info make'[1], which should a first step to
    > > any serious use of it.

    >
    > Provided you have a specific reason to use GNU make. If not, then
    > reading some documentation that isn't GNU-specific might be a better
    > idea.


    I'd go further and suggest they *should* use GNU make (and read that
    manual). But as James says, it's the default 'make' on Linux and OS X
    anyway (and bundled as 'gmake', Solaris 10).

    >
    > - Logan
    toby, Dec 29, 2006
    #14
  15. "toby" <> wrote in message
    news:...
    >
    > What also tends to be revealed by make *questions* is that the
    > questioner has not read 'info make'[1], which should a first step to
    > any serious use of it.


    I'm very familiar with make. Just that I wanted to know:

    a)Are there any limitations of make that will make it unsuitable for a very
    large project?

    b)Are there any better alternatives?

    Thanks.
    David T. Ashley, Dec 29, 2006
    #15
  16. David T. Ashley

    Default User Guest

    Richard Bos wrote:

    > Logan Shaw <> wrote:
    >
    > > wrote:
    > > > Jussi Jumppanen
    > > > Author: Spam for Windows
    > > > http:\\www.spamedit.com

    > > ^^
    > >
    > > I'm sure you meant to give a legal URI there...

    >
    > I'm sure it just wanted to spam. (And apparently his product "is Y2K",
    > according to his FAQ. No, not Y2K-compatible; "Y2K". My confidence in
    > its excellence is not stellar.)


    Mr. Jumppanen a well-known figure on many other programming and
    compiler newsgroups. I think you've probably done him a disservice my
    labeling him a spammer.




    Brian
    Default User, Dec 29, 2006
    #16
  17. David T. Ashley

    toby Guest

    james of tucson wrote:
    > wrote:
    > >
    > > <OT>
    > > Have you tried Eclipse and CDT?
    > > http://www.eclipse.org/
    > > http://www.eclipse.org/cdt/
    > > </OT>
    > >

    >
    > While I love, absolutely adore, using Eclipse for Java, my brief
    > encounter with CDT made me *run* back to the shelter of vim + make.


    I avoid CDT's "Managed" mode, not because I have had a bad experience,
    but because I prefer make myself - and CDT's "Standard" mode works
    perfectly with ordinary makefiles. You even get an pretty outlining
    Makefile editor.

    >
    > To be fair, I have put rather substantial effort into tuning my vim
    > environment for C coding in particular, making the
    > edit-compile-run-debug cycle much more pleasant than it might appear to
    > a newbie during his first years with the platform.
    toby, Dec 29, 2006
    #17
  18. David T. Ashley

    Diwa Guest

    David T. Ashley wrote:
    > I have a large project on a Linux box that may eventually have several
    > hundred 'C' source files, which need to be compiled and linked into several
    > executables.
    >


    > b)Any thoughts on doing this with "make"?


    Check out www.scons.org
    Diwa, Dec 29, 2006
    #18
  19. David T. Ashley

    toby Guest

    David T. Ashley wrote:
    > "toby" <> wrote in message
    > news:...
    > >
    > > What also tends to be revealed by make *questions* is that the
    > > questioner has not read 'info make'[1], which should a first step to
    > > any serious use of it.

    >
    > I'm very familiar with make. Just that I wanted to know:
    >
    > a)Are there any limitations of make that will make it unsuitable for a very
    > large project?


    100 source files is not a large project.

    >
    > b)Are there any better alternatives?
    >
    > Thanks.
    toby, Dec 29, 2006
    #19
  20. (Richard Bos) writes:
    > Logan Shaw <> wrote:
    >> wrote:
    >> > Jussi Jumppanen
    >> > Author: Spam for Windows
    >> > http:\\www.spamedit.com

    >> ^^
    >>
    >> I'm sure you meant to give a legal URI there...

    >
    > I'm sure it just wanted to spam. (And apparently his product "is Y2K",
    > according to his FAQ. No, not Y2K-compatible; "Y2K". My confidence in
    > its excellence is not stellar.)


    The contents of signatures are not bound by topicality rules, and I
    have no problem with advertisements there. But a signature should be
    delimited by the standard "-- " line.

    (If you're interested in the actual URL, go back to Jussi's article;
    it was altered in followups. There is no "spamedit.com"; when I
    Googled for "spamedit", it asked me if I meant "someidiot".)

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
    We must do something. This is something. Therefore, we must do this.
    Keith Thompson, Dec 29, 2006
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Chris
    Replies:
    4
    Views:
    587
    Chris
    Jul 15, 2004
  2. Replies:
    13
    Views:
    10,864
  3. Chaz Ginger

    Best way to handle large lists?

    Chaz Ginger, Oct 3, 2006, in forum: Python
    Replies:
    19
    Views:
    532
    Fredrik Lundh
    Oct 4, 2006
  4. Nagaraj
    Replies:
    1
    Views:
    835
    Lionel B
    Mar 1, 2007
  5. John Dalberg
    Replies:
    1
    Views:
    500
    =?Utf-8?B?UGV0ZXIgQnJvbWJlcmcgW0MjIE1WUF0=?=
    Mar 28, 2007
Loading...

Share This Page