I'm a stupid blond :( Please help me!!!

[Gary Labowitz]

Peter van Merkerk wrote:

Also note that with the existence of virtual functions it got much harder
for the linker to figure out if a function is referenced or not.

I'm not sure of this. It wouldn't make sense for a function name that occurs
in a program to be left out of the external references table. Unless linkers
have changed a lot since I worked on them each function call that is
unresolved results in an address for that function of zero and a flag (in a
table of some sort) indicating that it is an unresolved external reference.
The linker's job is to search the libraries it has available and link in the
object file that contains the code for that referenced function. If the
object code of that referenced function also contains code for some other
number of functions, all that code is linked in whether it is referenced or
not. This is what causes bloat.

The opposite side of that coin is to make more and more object files with
less and less functions in them. This leads to library bloat. (It also makes
the linkage process take longer.) It's always a trade-off, and you also want
to keep functionally related code together for maintenance ease.

I can't recall any system specifically linking in only the code of a
referenced function from a module that has several functions in it. You use
one -- you get them all.

The resulting exe has all the referenced function addresses filled in;
usually relative to the start of the exe's code. That table that told where
the unreferenced function addresses were in the object files now tells where
the relative addresses are in the exe. It's the loader's job to adjust them
to actual addresses when the exe is loaded. (I believe they are adjusted to
virtual addresses nowadays so that the code can be placed anywhere in real
memory and the machine's virtual addressing mechanism adjusts all addresses
to real memory address.)

Anyway, the way it always worked before: if you reference a function, you
get the entire object file that contains that function linked in. No way
out.

 

[Joerg Riesmeier]

Thom,

Try asking over in the comp.protocols.dicom group. They deal with
DICOM all day.

sounds reasonable, and in fact this is where we already solved the
"compilation problem".

However, comp.protocols.dicom is where all the C++ related postings
are also visible since the OP asked in both groups and none of the
replyers seems to care whether his posting is only related to one
group or both groups.

Regards,
Joerg Riesmeier

 

[Niklas Borson]

Not at all surprising. After all, that *was* my point! Unused code is
linked in.

All the code described above is used by your program. Your example used
cout, which is a global variable of type ostream, which is a typedef for
basic_ostream< char, char_traits<char> >. For you to use this object it
must first be constructed, which means all of its bases and members must
also be constructed.

I was rebutting a claim that:


I read "removed" as meaning "will not be linked".

That is my reading too. And I think the statement you quoted
remains true: code that isn't *referenced* (directly or indirectly)
by your program will not be linked.

However, you seem to have missed my point that *referenced* is
not the same thing as *used*.

In my previous post, I gave an example which demonstrated how a
function might be *referenced* by a program yet never actually
used. Here's another example:

#include <cstdio>
int main()
{
printf("Hello World!\n");
}

You might think that the call to printf above is trivially simple
and should not add much to the size of the program. After all, how
hard can it be to write a string to standard output?

However, printf contains a lot of functionality. It must contain
or call code to find and parse format specifiers, and to format
various data types (strings, integers, floating point numbers,
pointers, etc.) in various ways (different alignments, widths,
precisions, etc.)

The above program just writes a literal string with no format
specifiers so it *uses* only a small subset of the functionality
in printf. Nevertheless, because the program *references* printf,
it gets all of printf, and all of the functions printf calls, all
the functions they call, and so on.

To do otherwise, the compiler/linker would have to analyze the
parameters you pass to printf, figure out which code paths within
printf might and might not be taken, and generate a custom printf
function just for you with all of the dead code (unused code paths)
eliminated. Dead code elimination is in fact a common compiler
optimization, but only in simple cases which do not depend on
whole-program analysis (e.g., knowing what parameters values might
and might not be passed in).

 

[Peter van Merkerk]

I'm not sure of this. It wouldn't make sense for a function name that occurs
in a program to be left out of the external references table. Unless linkers
have changed a lot since I worked on them each function call that is
unresolved results in an address for that function of zero and a flag (in a
table of some sort) indicating that it is an unresolved external reference.
The linker's job is to search the libraries it has available and link in the
object file that contains the code for that referenced function. If the
object code of that referenced function also contains code for some other
number of functions, all that code is linked in whether it is referenced or
not. This is what causes bloat.

That doesn't happen with all linkers

The opposite side of that coin is to make more and more object files with
less and less functions in them. This leads to library bloat. (It also makes
the linkage process take longer.) It's always a trade-off, and you also want
to keep functionally related code together for maintenance ease.

I can't recall any system specifically linking in only the code of a
referenced function from a module that has several functions in it. You use
one -- you get them all.

Well for one the linker that comes with Microsoft Visual Studio 6 (not
exactly state-of-the-art) links only the code of referenced functions. It
most certainly does not link the code of all functions in an .obj file,
unless every function in that .obj is referenced. What you are saying may be
true for some linkers but certainly not for all.

The resulting exe has all the referenced function addresses filled in;
usually relative to the start of the exe's code. That table that told where
the unreferenced function addresses were in the object files now tells where
the relative addresses are in the exe. It's the loader's job to adjust them
to actual addresses when the exe is loaded. (I believe they are adjusted to
virtual addresses nowadays so that the code can be placed anywhere in real
memory and the machine's virtual addressing mechanism adjusts all addresses
to real memory address.)

I won't comment on this part because it is drifting quite far away from the
topic of this newsgroup. But for the record; this explanation is not
necessarilly entirely accurate (what exactly happens depends on the OS).

Anyway, the way it always worked before: if you reference a function, you
get the entire object file that contains that function linked in. No way
out.

There is a way out; use a better linker. If you are interested I can mail
you a MSVC6 project that demonstrates that an object file is only partially
linked in when only a few functions in that object file are referenced.

 

[Gary Labowitz]

(in

That doesn't happen with all linkers

<<snip>>

Other comments of yours mentions OS differences, and yes, I can believe
that. I draw on my experience with IBM mainframe work for most of this OS
design, and what little I know of the various Intel processors and their
instruction sets. I didn't mean to introduce a deep discussion of various
memory mapping methods.

There is a way out; use a better linker. If you are interested I can mail
you a MSVC6 project that demonstrates that an object file is only partially
linked in when only a few functions in that object file are referenced.

No need, I have all that stuff. But if you want to tell me a library module
that contains several functions and a short code snippet that uses only one
of them that causes only that function to be linked in, I'd like to know it
and try it. I am very interested in exploring this and that information
would speed up my exploration. Do you know of any gnu linkers that work this
optimized way? Thanks. (Or if you think a project would be the best way, I'm
open to that also. I haven't researched yet for a W2K function that would
extract function code from a .obj file or a .dll, but I'm interested to find
it if it's there.)

 

[Peter van Merkerk]

There is a way out; use a better linker. If you are interested I can
mail

No need, I have all that stuff. But if you want to tell me a library module
that contains several functions and a short code snippet that uses only one
of them that causes only that function to be linked in, I'd like to know it
and try it.

Library modules are nothing more than just a collection of object files. So
a simple project consisting of two translation units can demonstrate whether
or
not unused code get linked in:

---foobar.c---
#include <stdio.h>

void foo()
{
puts("*** foo() ***");
}

void bar()
{
puts("*** bar() ***");
}

---main.c---
void foo();
void bar();

int main()
{
foo();
// bar();
return 0;
}

The map file output of a linker should give you some clue what is linked in
an what not (all linkers I have worked with in the recent years are capable
of producing a map file). Though for small executables I often just open the
executable file in a hex viewer; if the linker selectively links in code,
the string "*** bar() ***" should not be in the executable .

I am very interested in exploring this and that information
would speed up my exploration.
Do you know of any gnu linkers that work this
optimized way?

With the code above MSVC produced an executable of 2 KB without string "***
bar() ***". To my surprise the executable made with GCC was larger (11 KB)
and did include the "*** bar() ***". Based on the map file output of the LD
linker it indeed appears to be all or nothing when it comes to inlinking
object
files, I didn't find a command line option to improve this behaviour.
Hmm...who would have thought that a five year old Microsoft tool would do
better than the latest version of the gnu linker...

Since I haven't had any need yet to trim down the size of executables
produced by the GNU toolset yet, I cannot tell what would be a better
alternative for the standard linker. If code bloat problem is common enough,
chances are that there are alternatives for the LD linker.

Thanks. (Or if you think a project would be the best way, I'm
open to that also. I haven't researched yet for a W2K function that would
extract function code from a .obj file or a .dll, but I'm interested to find
it if it's there.)

Most compilers (including GCC) come with dumping tools. Though frankly I
never had a need for extract function code itself, I wonder why you want to
do that?

BTW: there is a difference beteen static linking and dynamic linking. With
DLL's the linker must assume that all exported functions in that DLL will be
called. So any function that is referenced directly or indirectly by those
exported functions will be linked into the DLL, no matter which linker you
use. Because of this there is a big chance that a dynamically linked
executable + DLL > statically linked executable.

 

[Gary Labowitz]

Library modules are nothing more than just a collection of object files. So
a simple project consisting of two translation units can demonstrate whether
or
not unused code get linked in:

---foobar.c---
#include <stdio.h>

void foo()
{
puts("*** foo() ***");
}

void bar()
{
puts("*** bar() ***");
}

---main.c---
void foo();
void bar();

int main()
{
foo();
// bar();
return 0;
}

The map file output of a linker should give you some clue what is linked in
an what not (all linkers I have worked with in the recent years are capable
of producing a map file). Though for small executables I often just open the
executable file in a hex viewer; if the linker selectively links in code,
the string "*** bar() ***" should not be in the executable .


With the code above MSVC produced an executable of 2 KB without string "***
bar() ***". To my surprise the executable made with GCC was larger (11 KB)
and did include the "*** bar() ***". Based on the map file output of the LD
linker it indeed appears to be all or nothing when it comes to inlinking
object
files, I didn't find a command line option to improve this behaviour.
Hmm...who would have thought that a five year old Microsoft tool would do
better than the latest version of the gnu linker...

Since I haven't had any need yet to trim down the size of executables
produced by the GNU toolset yet, I cannot tell what would be a better
alternative for the standard linker. If code bloat problem is common enough,
chances are that there are alternatives for the LD linker.


Most compilers (including GCC) come with dumping tools. Though frankly I
never had a need for extract function code itself, I wonder why you want to
do that?

BTW: there is a difference beteen static linking and dynamic linking. With
DLL's the linker must assume that all exported functions in that DLL will be
called. So any function that is referenced directly or indirectly by those
exported functions will be linked into the DLL, no matter which linker you
use. Because of this there is a big chance that a dynamically linked
executable + DLL > statically linked executable.

Okay, I've run some tests with basically the files you suggested. I changed
to iostream and cout so as to use standard libraries with g++.

Results would seem to say that if an object file is needed for one function
that appears in it, the file is included by the linker in the executable.
That brings in all the other functions' code.

My results: (in bytes)

foo only foo and bar
Dec-c++
main.o 1430 1458
module.o 1636 1636
..exe 450,748 450,748

VC++
main.obj 11,132 11,183
module.obj 33,950 33,950
..exe 245,823 245,823

I deleted all object and exe files before each build. I was afraid of
incrimental linking. The reason I wondered about extracting a single
function from an object file is because that is what would be needed to load
only functions that are referenced in the exe. I notice in the map file from
VC++ that both symbols appear in the dictionary, regardless of whether or
not they are called. The exe's are so large because I left in all debugging
(meaning symbolic) information and did not optimize.
Dropping both foo and bar from main resulted in an exe of 172,095 in C++ and
an exe of 23,036 in Dev-c++. All the extra would be from the iostream
modules being loaded to support cout.

 

[osmium]

Library modules are nothing more than just a collection of object files. So
a simple project consisting of two translation units can demonstrate whether
or
not unused code get linked in:

<snip>

How about doing these two with VC++:

int main()
{return 0;}
------
#include <cmath>
int main()
{
fabs(1.23);
return 0;
}

And report what the size of the two executable files are? Static linking,
of course. I have numbers for my compiler, I would like to see how much
better VC++ is. I hope we all agree that fabs() should be a small handful
of assembler code. So there should be a difference of at most 100 bytes,
right?

 

[Gary Labowitz]

<snip>

How about doing these two with VC++:

int main()
{return 0;}
------
#include <cmath>
int main()
{
fabs(1.23);
return 0;
}

And report what the size of the two executable files are? Static linking,
of course. I have numbers for my compiler, I would like to see how much
better VC++ is. I hope we all agree that fabs() should be a small handful
of assembler code. So there should be a difference of at most 100 bytes,
right?

In bytes I got 172,087 for the first and 196,663 for the second.

 

[Peter van Merkerk]

My results: (in bytes)

foo only foo and bar
Dec-c++
main.o 1430 1458
module.o 1636 1636
.exe 450,748 450,748

VC++
main.obj 11,132 11,183
module.obj 33,950 33,950
.exe 245,823 245,823

I deleted all object and exe files before each build. I was afraid of
incrimental linking. The reason I wondered about extracting a single
function from an object file is because that is what would be needed to load
only functions that are referenced in the exe. I notice in the map file from
VC++ that both symbols appear in the dictionary, regardless of whether or
not they are called. The exe's are so large because I left in all debugging
(meaning symbolic) information and did not optimize.
Dropping both foo and bar from main resulted in an exe of 172,095 in C++ and
an exe of 23,036 in Dev-c++. All the extra would be from the iostream
modules being loaded to support cout.

Executable sizes can be a bit deceiving as the sections grow by 512 bytes.
For example both the foo and foo+bar executables are 2048 bytes (build with
full optimization) on my system, a hex viewer can provide conclusive
evidence whether or not the bar function gets linked in. The microsoft
linker has an option that enables selective linking: /OPT:REF (see also
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore98/HT
ML/_core_.2f.opt.asp). It is very wel possible that this option is not
enabled in debug configurations.

 

[Peter van Merkerk]

<snip>

How about doing these two with VC++:

int main()
{return 0;}
------
#include <cmath>
int main()
{
fabs(1.23);
return 0;
}

And report what the size of the two executable files are? Static linking,
of course. I have numbers for my compiler, I would like to see how much
better VC++ is. I hope we all agree that fabs() should be a small handful
of assembler code.

Not really, as it will link in also code to deal with floating point
exceptions and code for controlling the floating point control word. For
example the fabs() implementation I have here references the __ctrlfp,
__sptype, __except1 and __handle_qnan1 functions which themselves may
reference other functions. The fabs() function may seem simple at first, but
there are quite a few boundary cases (like NaN's) it has to deal with.

The problem with using standard library functions is that you do not know
how exactly they are implemented (unless you dig into their implementation
first) so you have no idea what you are measuring. Consequently no
conclusions can be drawn from such an experiment. That is why I provided the
foo() and bar() functions, their implementation is known, and the strings in
those functions can be easily found in the executable, provided those
functions are linked in the first place.

So there should be a difference of at most 100 bytes,
right?

Wrong, see above.

The code I provided should be adequate to prove whether your linker links
selectively or not.

And even if you don't believe me (which is perfectly ok and a sound
attitude!); Microsoft claims that their linker can selectively link in
functions:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore98/HT
ML/_core_.2f.opt.asp

I think I have made my point, and frankly have no desire to continue this
discussion.

 

[osmium]

:

In bytes I got 172,087 for the first and 196,663 for the second.

Well, isn't that interesting! If I were to write that function it would be

double fabs(double x)
{
if(x < 0.0)
return -x;
}

but it consumes 24,576 bytes! Now I note that 24,576 is a multiple of 4096
so perhaps the actual code is smaller than the numbers indicate. But still
it seems a pretty big chunk for this linker that links only what is
necessary. Almost as if they had included more than the function above.
Maybe it is one of these constructor "thigngys" Or this "reference" stuff.
Perhaps Mr. van Merkerk can explain this odd phenomenon?

The corresponding numbers for my Borland compiler are 8,578 and 14,883. So
it doesn't really look like this advanced VC++ linker is really all that
much of an improvement. Now, I expect a barrage of "It's not important
anyway, RAM is cheap".

Thanks very much to you and to Gary Labowitz for injecting some realism into
this frustrating and annoying thread. I'm outa here!

 

[rwannall]

Referencing DCMTK libraries from VB.NET

To Joerg Riesmeier:

I downloaded the source and used cmake to generate the project and solution files, and then compiled to produce the .exe and .lib files.

What is the next step to take if I now want to create a C# project and add a reference that lets me use the DCMTK libraries? I cannot add a reference to the .lib files, since they are not COM files and obviously not .NET assemblies.

(Apologies to others for off-topic post, not strictly C++. I looked through the OFFIS forums but didn't find the answer to this question, and was waiting for my account activation there to ask the same question in that forum. Found this thread in the meanwhile. As soon as I get my OFFIS account activated, I'll stick to those forums for DCMTK-specific questions.)

Rick Wannall
University of Houston

Jenya,

> My problem is as following. I'm supposed to write a DICOMDIR parser
> - visual one - on MFC. I'm trying to use the DCMTK kit (dcdicdir
> especially).

I propose the following:

0. read how your compiler works
1. read the INSTALL file from DCMTK's main directory
2. compile the toolkit with MSVC (at least module ofstd and dcmdata),
in case you want to use any of the external libraries either
compile them from source or download them in pre-compiled format
from our web site http://dicom.offis.de/dcmtk , you could also
disable particular libraries in file "config/include/cfwin32.h".
3. write a small program that uses the DCMTK, include "dctk.h" and
make sure that the libraries "ofstd.lib" and "dcmdata.lib" can be
found by the linker.

That's all. However, please be warned that your program needs to use
the same settings for "code generation" (e.g. "multi-threaded") as
you've used for the toolkit, otherwise the linking will fail.

Regards,
Joerg Riesmeier
OFFIS