How to organize files in large applications?

[Jayden Shui]

Hello All,

I'd like to ask your suggestion on organizing files in a large
application? Currently I put all files in the same folder and don't
use name space. I feel harder and harder when adding or managing files
and classes.

Thanks and best regards,

Jayden

 

[ralph]

Hello All,

I'd like to ask your suggestion on organizing files in a large
application? Currently I put all files in the same folder and don't
use name space. I feel harder and harder when adding or managing files
and classes.

Stop what you are doing and ...

#1 - Starting using NameSpaces.

#2 - Adopt a Revision Control System (Source Code Control System).

Without knowing more about your particular problem domain, platform,
or budget, it is impossible to make a specific recommendation, but
Subversion is always a good bet.
http://en.wikipedia.org/wiki/Comparison_of_Subversion_clients

-ralph

 

[Nick Keighley]

I'd like to ask your suggestion on organizing files in a large
application? Currently I put all files in the same folder and don't
use name space. I feel harder and harder when adding or managing files
and classes.

http://www.amazon.com/Large-Scale-Software-Design-John-Lakos/dp/0201633620

it's an old book but the only one I've seen address the subject.

For a start invent some packages. Each package gets its own sub-
directory. try to ensure each package depends on one
"technology" (XML, database, sockets, fonts etc).

I quite liked this book
http://www.amazon.com/Software-Development-Principles-Patterns-Practices/dp/0135974445

[beware broken link]

 

[Jorgen Grahn]

Stop what you are doing and ...

#1 - Starting using NameSpaces.

#2 - Adopt a Revision Control System (Source Code Control System).

I agree 100%, but note that he didn't say he doesn't use one.

/Jorgen

 

[Jorgen Grahn]

Hello All,

I'd like to ask your suggestion on organizing files in a large
application? Currently I put all files in the same folder and don't
use name space.

Name spaces and file organization are orthogonal concepts.

I feel harder and harder when adding or managing files
and classes.

IME, it doesn't matter much. 50 source files in one directory (folder)
or 50 spread into N directories -- it's still 50 source files.

I tend to prefer flat file structures. Unit tests go in a
subdirectory 'test'; other things normally don't.

Whatever you do:
- don't split source and header files in separate 'src' and 'include'
directories. It's not a natural thing to split on.
- don't use recursive Makefiles, where each subsystem is in a
directory and has its own build logic

/Jorgen

 

[Goran]

Hello All,

I'd like to ask your suggestion on organizing files in a large
application? Currently I put all files in the same folder and don't
use name space. I feel harder and harder when adding or managing files
and classes.

Use namespaces to distinguish between broad areas in the project. Say
that you have data layer, communication layer and UI layer. You might
want to have same namespaces, quite likely with same-named classes in
all three. E.g. data::blurp is a class that manipulates blurps in your
program, communication::blurp is a thing that you use to send
data::blurp over a line, and ui::blurp is a thing used to manipulate
data::blurp in UI.

As for folders, same thing. You don't need to match namespaces, but
you can. Folders are interesting if you're using OS shell to
manipulate files. Much less so if you're using an IDE. With an IDE,
you might be able to have folders in some "project organization" tree
without disk folders. An IDE will typically also show your code
artifacts in a tree-like fashion, using namespaces you made, so you
won't need folders.

Goran.

 

[Werner]

Whatever you do:
- don't split source and header files in separate 'src' and 'include'
  directories.  It's not a natural thing to split on.

I disagree (not for the sake of disagreeing ;-).

Especially libraries are often organized by splitting
headers and sources, except when they are templates.
Splitting headers and sources also promotes splitting
interface and implementation (quite natural?). I'm
having Qt in mind whilst writing this, as well as
my own libraries...

Also, from a client perspective, when using libraries,
it is appreciated when some form of folder structure (not
to deep) exists that factors commonality.

Kind Regards,

Werner

 

[Nick Keighley]

Name spaces and file organization are orthogonal concepts.


IME, it doesn't matter much. 50 source files in one directory
(folder) or 50 spread into N directories -- it's still 50 source
files.

well I disagree! I think both ideas are bad. How about 7 directories
with about 7 files in each? The aim is to partition your large complex
application and to small understandable bite-sized pieces.

Now at 50 files it's less important (50 files in one directory sounds
alarm bells though). But it starts to matter when you hit thousands of
files.

I tend to prefer flat file structures. Unit tests go in a
subdirectory 'test'; other things normally don't.

I gues your project is very small.

Whatever you do:
- don't split source and header files in separate 'src' and 'include'
  directories.

I don't do this but the argument is the the user of a library needs
the .h files but he doesn't need the .cpp.

It's not a natural thing to split on.
- don't use recursive Makefiles, where each subsystem is in a
  directory and has its own build logic

why not? You seem to favour a very monolithic design. Highly coupled,
likely low cohesion. This simply won't scale.

 

[Ian Collins]

why not? You seem to favour a very monolithic design. Highly coupled,
likely low cohesion. This simply won't scale.

Recursive Makefiles don't play well with parallel/distributed building.

 

[Joshua Maurice]

why not? You seem to favour a very monolithic design. Highly coupled,
likely low cohesion. This simply won't scale.

Recursive Make Considered Harmful
http://aegis.sourceforge.net/auug97.pdf

PS: Note that I think all current portable and free build systems suck
(for all systems that I know, and I've looked really hard), but some
suck less than others. As one of my hobbies, I'm writing my own. I
think I'm on prototype 4 now. Still, I think there are some very
important lessons contained in that paper for how to write a proper
build system.

 

[Nick Keighley]

Recursive Make Considered Harmfulhttp://aegis.sourceforge.net/auug97.pdf

PS: Note that I think all current portable and free build systems suck
(for all systems that I know, and I've looked really hard), but some
suck less than others. As one of my hobbies, I'm writing my own. I
think I'm on prototype 4 now. Still, I think there are some very
important lessons contained in that paper for how to write a proper
build system.

ok, thanks. I thought the options were modular recursive make or
monolithic build system. it seems not.

 

[ralph]

Stop what you are doing and ...

#1 - Starting using NameSpaces.

#2 - Adopt a Revision Control System (Source Code Control System).

Without knowing more about your particular problem domain, platform,
or budget, it is impossible to make a specific recommendation, but
Subversion is always a good bet.
http://en.wikipedia.org/wiki/Comparison_of_Subversion_clients

It might be useful to expand on #2 above.

The SCCS becomes the repository for your source code, not just to
provide revision services but as an organizer. Think of it as a
database in which you arrange and store your source in logical or
practical units ("folders", "directories", etc. exact terminology will
depend on the SCCS you decide to use) divorced from any physical
storage. Each of the stored elements will have a "working directory".

Thus you can organize your modules into logical units, yet dump them
into a single physical folder while you are working with them. The
SCCS is becomes in effect a view of your project as a whole - where
they are physically stored while developing (often a single folder) is
of secondary interest.

While organizing your project keep an eye out for logical "packages"
that can be treated as separate "deliverables" and once constructed
are seldom touched. These can be moved into projects of their own.
Your current working project only needs to deal with a few .libs and
headers. Almost any OO project can reduce its current modules of
interest dramatically by recognizing and removing such packages.

[Separate package projects also makes it easier to work with the
issues brought up by the others, but that - as they say - is another
story. <g>]

It would be helpful if we knew the IDE (if any) you are using and your
budget.

-ralph

 

[Jorgen Grahn]

I disagree (not for the sake of disagreeing ;-).

Especially libraries are often organized by splitting
headers and sources, except when they are templates.
Splitting headers and sources also promotes splitting
interface and implementation (quite natural?). I'm
having Qt in mind whilst writing this, as well as
my own libraries...

Also, from a client perspective, when using libraries,
it is appreciated when some form of folder structure (not
to deep) exists that factors commonality.

This is fine -- for an actual, installed library like Qt or the C++
standard library.

But IMHO it's not useful when you're looking at the full source code
of the library. You also frequently see this split being done to
non-library code (which I assume is what the OP is writing).

/Jorgen

 

[Jorgen Grahn]

The aim is to partition your large complex
application and to small understandable bite-sized pieces.
Now at 50 files it's less important (50 files in one directory sounds
alarm bells though). But it starts to matter when you hit thousands of
files.


I gues your project is very small.

I made some assumptions. The OP seems to be alone with his code, or at
least not have dozens or hundreds of coworkers. You don't hit
thousands of files that way.

....

why not?

Why not use recursive makefiles? Because they don't work -- see the
paper someone else referenced for a detailed explanation. It would be
nice if they did, but ...

You seem to favour a very monolithic design.

Actually, to some extent I do. I see more problems caused by people
trying to apply large-scale techniques where they aren't needed, than
by people not doing it where it /is/ needed. And when you've made the
small, simple things baroquely complex, it's hard to do anything about
the /real/ issues.

/Jorgen