Locating large header files?

N

none

I have a large application which is around 8 MB when its build. I am pretty sure there are some
unnecessary includes. Is there some tool to locate large or unused header files in soure code?
 
V

Victor Bazarov

I have a large application which is around 8 MB when its build. I am
pretty sure there are some unnecessary includes. Is there some tool to
locate large or unused header files in soure code?

Uh... I read somewhere that there was a tool for weeding out the
includes that weren't used, and GIYF for that. However, do consider
that usually headers contain *declarations* which usually are only used
by the compiler and do not get converted into code (that's the
*definition's* job).

Ask your linker to produce a *map* file and examine it. You're going to
find what libraries you have linked to, and what modules functions you
have defined.

Also, consider that 8MB is *not* a large application. A large
application would be on the scale of *hundreds* of megabytes with a
multitude of modules it loads at run time.

V
 
N

none

Victor said:
Uh... I read somewhere that there was a tool for weeding out the
includes that weren't used, and GIYF for that. However, do consider
that usually headers contain *declarations* which usually are only used
by the compiler and do not get converted into code (that's the
*definition's* job).

Ask your linker to produce a *map* file and examine it. You're going to
find what libraries you have linked to, and what modules functions you
have defined.

Also, consider that 8MB is *not* a large application. A large
application would be on the scale of *hundreds* of megabytes with a
multitude of modules it loads at run time.

V

My application is primary based on template classes and use headers from a a template library so all
the hard work is done a compile time.

It takes more than a 1 minute to compile the application (resulting in the 8 MB executable) which I
think is a long time to wait if I just do small changes. I have tried to move out some of the
classes (which removes the core functionality of the application) and then the compile time is
lowered significantly.

Is there any way to optimize the compile time with all the functionality or is this just something I
have to accept?
 
N

none

none said:
My application is primary based on template classes and use headers from
a a template library so all the hard work is done a compile time.

It takes more than a 1 minute to compile the application (resulting in
the 8 MB executable) which I think is a long time to wait if I just do
small changes. I have tried to move out some of the classes (which
removes the core functionality of the application) and then the compile
time is lowered significantly.

Is there any way to optimize the compile time with all the functionality
or is this just something I have to accept?


BTW: This is my compiler options:

SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -w -lX11 -Dcimg_use_xshm -lXext -Dcimg_use_xrandr -lXrandr
-O3 -fpermissive -march=nocona -m64 -ffast-math -funroll-loops -finline-functions")
 
Ö

Öö Tiib

My application is primary based on template classes and use headers from a a template library so all
the hard work is done a compile time.

Nope. It is either lot of code, lot of static data or lot of RUN-TIME
template usage.
With static data it is easy to reach, just embed some speech and movie-
like animations as static data.
To generate so lot of template usage you probably have to use code
generator or something.
To have honestly so lot of code it takes YEARS for team.

It takes more than a 1 minute to compile the application (resulting in the 8 MB executable) which I
think is a long time to wait if I just do small changes. I have tried to move out some of the
classes (which removes the core functionality of the application) and then the compile time is
lowered significantly.

1 minute? 10 minutes is like 8MB of executable code. As for 8MB in
single app I have never got so far, the multi-hundred-thousands source
lines of code mean often lots of under 1MB modules, tools, plug-ins,
scripts and data files. Sure, in sum these are more than 8MB.
Otherwise it tends to get unmaintainable.
Is there any way to optimize the compile time with all the functionality or is this just something
I have to accept?

Included files do not usually affect the size of result. Most linkers
throw unused crap away. It really does affect compilation speed.

There are some tools that help but problems like unused header files
indicate design issues. Tools that seriously aid with design (help to
reduce need for inclusions, not only remove possibly unused) are often
overpriced.
 
V

Victor Bazarov

[..]
BTW: This is my compiler options:

SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -w -lX11 -Dcimg_use_xshm -lXext
-Dcimg_use_xrandr -lXrandr -O3 -fpermissive -march=nocona -m64
-ffast-math -funroll-loops -finline-functions")

This would be relevant if we were in a compiler newsgroup. We are not.

V
 
V

Victor Bazarov

My application is primary based on template classes and use headers from
a a template library so all the hard work is done a compile time.

So why are you so concerned with how long it takes to compile it? The
hard work needs to be done some time, no?
It takes more than a 1 minute to compile the application (resulting in
the 8 MB executable) which I think is a long time to wait if I just do
small changes.

There are changes and there are changes. There are also ways to
eliminate the compile-time dependencies which usually speeds up
compilation (but no guarantees). PImpl is one. Using
forward-declarations of most classes in the headers instead of pulling
the "needed" headers is the other.
> I have tried to move out some of the classes (which
removes the core functionality of the application) and then the compile
time is lowered significantly.

Not sure what you mean by "moving out" of classes, but if it's critical
to your application, what purpose would moving out serve? You can try
to identify which ones are contributing the most to the compilation time
and the app size, and work on those, but don't expect miracles. If your
app needs its functionality, you gotta keep the classes, no? And you
need to compile those at some point...
Is there any way to optimize the compile time with all the functionality
or is this just something I have to accept?

I already mentioned a couple of ways. The main motif here is to
separate the implementation from the interface. If you can move the
definitions of your member functions from headers into separate C++
source files, do. Also, compartmentalize your code, if you're working
on some part, others shouldn't have to be recompiled. For example, in
our shop we have a policy of "one class per module". It's not always
observed, but it does help to reduce the number of files you touch when
you need to change the implementation (not the definition) of a class.

And, yes, as your application grows in what it can do, the time to
rebuild it will grow as well. Usually proportionally. And if you are
touching some very low level functionality, be prepared to have to
rebuild the entire app...

Good luck!

V
 
A

Alf P. Steinbach

* Victor Bazarov, on 19.05.2010 16:09:
[..]
BTW: This is my compiler options:

SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -w -lX11 -Dcimg_use_xshm -lXext
-Dcimg_use_xrandr -lXrandr -O3 -fpermissive -march=nocona -m64
-ffast-math -funroll-loops -finline-functions")

This would be relevant if we were in a compiler newsgroup. We are not.

Well, there are not that many C++ compilers around. This seems to be g++. I'd
suggest adding '-s' (IIRC) in there to strip away symbols.

Templates generate a lot of symbols, and very long ones.

Possibly a little '-s' (or stripping symbols manually afterwards) could reduce
the 8 MiB to, well, something much smaller... ;-)


Cheers & hth.,

- Alf (practical mode)
 
J

Jonathan Lee

My application is primary based on template classes and use headers from a a template library so all
the hard work is done a compile time.

I suppose what you're saying is that the templates are being
recompiled over and over, when what you're changing isn't
relevant to them.

To avoid this problem, use explicit instantiation (GIYF) for
your template classes. Assuming you're using some kind of
build system, rig things so the instantiation gets compiled
and put into an object file, and hides the dependency on the
header. Then, like your "regular" classes, that object file
won't have to be recompiled unless you change something in
the template.

--Jonathan
 
N

none

Alf said:
* Victor Bazarov, on 19.05.2010 16:09:
[..]
BTW: This is my compiler options:

SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -w -lX11 -Dcimg_use_xshm -lXext
-Dcimg_use_xrandr -lXrandr -O3 -fpermissive -march=nocona -m64
-ffast-math -funroll-loops -finline-functions")

This would be relevant if we were in a compiler newsgroup. We are not.

Well, there are not that many C++ compilers around. This seems to be
g++. I'd suggest adding '-s' (IIRC) in there to strip away symbols.

Templates generate a lot of symbols, and very long ones.

Possibly a little '-s' (or stripping symbols manually afterwards) could
reduce the 8 MiB to, well, something much smaller... ;-)


Cheers & hth.,

- Alf (practical mode)

Thanks that removed 1 MB :) But unfortunately the compile time is the same. Why would anyone want
to include this in the executable if it only takes up space?
 
V

Victor Bazarov

Alf said:
* Victor Bazarov, on 19.05.2010 16:09:
On 5/19/2010 9:26 AM, none wrote:
[..]
BTW: This is my compiler options:

SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -w -lX11 -Dcimg_use_xshm -lXext
-Dcimg_use_xrandr -lXrandr -O3 -fpermissive -march=nocona -m64
-ffast-math -funroll-loops -finline-functions")

This would be relevant if we were in a compiler newsgroup. We are not.

Well, there are not that many C++ compilers around. This seems to be
g++. I'd suggest adding '-s' (IIRC) in there to strip away symbols.

Templates generate a lot of symbols, and very long ones.

Possibly a little '-s' (or stripping symbols manually afterwards)
could reduce the 8 MiB to, well, something much smaller... ;-)


Cheers & hth.,

- Alf (practical mode)

Thanks that removed 1 MB :) But unfortunately the compile time is the
same. Why would anyone want to include this in the executable if it only
takes up space?

Neither the compiler nor the linker are capable of predicting that you
are *never* going to use a debugger with your executable (which often
requires symbols for more efficient and effective debugging). So,
symbols don't just "take up space". As to why putting the symbols in is
a default action, that's up to the implementors of the tool, and
apparently they've decided that unless you tell the tool *not* to create
symbols (by means of -s in your case), they are going to create them.

But this is all irrelevant in a *language* newsgroup. And off-topic.

V
 
B

Brian

So why are you so concerned with how long it takes to compile it?  The
hard work needs to be done some time, no?


There are changes and there are changes.  There are also ways to
eliminate the compile-time dependencies which usually speeds up
compilation (but no guarantees).  PImpl is one.  Using
forward-declarations of most classes in the headers instead of pulling
the "needed" headers is the other.

 > I have tried to move out some of the classes (which


Not sure what you mean by "moving out" of classes, but if it's critical
to your application, what purpose would moving out serve?  You can try
to identify which ones are contributing the most to the compilation time
and the app size, and work on those, but don't expect miracles.  If your
app needs its functionality, you gotta keep the classes, no?  And you
need to compile those at some point...


I already mentioned a couple of ways.  The main motif here is to
separate the implementation from the interface.  If you can move the
definitions of your member functions from headers into separate C++
source files, do.  Also, compartmentalize your code, if you're working
on some part, others shouldn't have to be recompiled.  For example, in
our shop we have a policy of "one class per module".  It's not always
observed, but it does help to reduce the number of files you touch when
you need to change the implementation (not the definition) of a class.

I've found that gcc 4.4.2 is able to produce a 3-4% smaller
executable
when I merge some files. Doing that has an affect on build times,
but
I'm willing to pay that price for a decent executable.


Brian Wood
http://webEbenezer.net
(651) 251-9384
 
V

Victor Bazarov

I've found that gcc 4.4.2 is able to produce a 3-4% smaller
executable
when I merge some files. Doing that has an affect on build times,
but
I'm willing to pay that price for a decent executable.

Interesting. In our team we've observed an opposite effect, albeit with
a different compiler. Merging source modules (by means of inclusion)
automatically before compilation made the builds go much faster (three-
or four-fold), mostly due to shortened linking times, on a few large
pieces, yet the resulting executables are basically the same (although
we mostly build dynamic libraries).

Depending on the tools used, a bunch of smaller files or a set of merged
modules can be just what the doctor ordered either for the frequent
recompilation during development, or to produce the final executable for
presenting to the customer... I know that our group was mostly
concerned with saving time while developing since it shortens the build
by about 15-20% overall (not all parts of our build are refactored for
merging sources), so counting each developer, multiple builds a day, you
get significant savings over the course of a release cycle.

V
 
J

James Kanze

Use a faster build machine!

Or a faster disk, or a faster network. (We're using
a distributed build system, and build time---still way too
slow---is constrained by the network speed.)
 
B

Brian

Or a faster disk, or a faster network.  (We're using
a distributed build system, and build time---still way too
slow---is constrained by the network speed.)

Do I correctly recall you mentioning XML being used in
your work? Maybe the network speed is good enough, but
some tools are hogging resources. Speaking of hogs, I
want to piggy-back something here. I'm working on
transforming the command line interface of the C++ Middleware
Writer from a run-once executable to a server that maintains
a connection with the C++MW. It also won't have to reread
config files if they haven't been changed. The piggy-back
part is that this can be viewed as an evolution of
distributed builds. Avoiding connecting and transmitting/
verifying passwords more than is needed, will help to
reduce the cost of the transaction on all systems involved.

I'm looking for advice on heartbeats. I have "Effective
TCP/IP Programming" by Jon Snader and some books by
Richard Stevens. My thought at the moment is to have the
C++MW initiate a heartbeat/check message to every client
on a periodic basis. Assuming that isn't way off, I'm
wondering about values for that period. Perhaps there are
some more recent books or links that are helpful on the
subject. Thanks in advance.


Brian Wood
http://webEbenezer.net
(651) 251-9384
 
B

Brian

Do I correctly recall you mentioning XML being used in
your work?  Maybe the network speed is good enough, but
some tools are hogging resources.  Speaking of hogs, I
want to piggy-back something here.  I'm working on
transforming the command line interface of the C++ Middleware
Writer from a run-once executable to a server that maintains
a connection with the C++MW.  It also won't have to reread
config files if they haven't been changed.  The piggy-back
part is that this can be viewed as an evolution of
distributed builds.  Avoiding connecting and transmitting/
verifying passwords more than is needed, will help to
reduce the cost of the transaction on all systems involved.

I've now created both Linux and Windows C++ Middleware
Writer Ambassadors. These are servers that run locally
and mediate between users and the C++MW. The code for
these servers is available on my website. They pipeline
requests to the C++MW. I was thinking about posting the
code here on a gradual basis (over a few days) and asking
for some code review help.

Brian Wood
http://webEbenezer.net
651) 251-9384
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,904
Latest member
HealthyVisionsCBDPrice

Latest Threads

Top