Size of a process

[mysorepushpa]

Hi all,

Suppose I have a C program in which I would have done a couple of
mallocs which amounts upto 1MB. Assuming the size of the program is
around 100KB for other things like code, data and other stuff. Will
this malloc size be counted in the size of the program when the exe is
created for the program.

Regards,
Push

 

[Nate Eldredge]

Hi all,

Suppose I have a C program in which I would have done a couple of
mallocs which amounts upto 1MB. Assuming the size of the program is
around 100KB for other things like code, data and other stuff. Will
this malloc size be counted in the size of the program when the exe is
created for the program.

Why not try it and see?

 

[btrower]

Hi all,

Suppose I have a C program in which I would have done a couple of
mallocs which amounts upto 1MB. Assuming the size of the program is
around 100KB for other things like code, data and other stuff. Will
this malloc size be counted in the size of the program when the exe is
created for the program.

Regards,
Push

No. The malloc() function allocates memory from the heap at run time.
It could not be allocated at compile-time because there is no way to
be certain if the allocation will even take place. Code and static
data are allocated at compile time (though modern compilers don't even
allocated empty static data unless it is used).

On my system, a compiled file does not have the large static array
allocated in the binary image. At run time, the static memory (on my
system) is still not allocated until you write to it. Similarly, the
allocated memory is not even used until you write to it. YMMV and
don't take my word for it. Here is some code that should compile and
run with most C compilers.

----- Snip ------

#include <stdio.h>
#include <memory.h>
#include <malloc.h>

#define TEST_MEMORY_SIZE 100000000

static char m1[TEST_MEMORY_SIZE];

#define pause(s) printf( "Press 'enter' key to %s.\n", s ); (void)
getchar();

int main()
{
void *m2 = NULL;

pause( "write to static memory" );
memset( m1, 65, TEST_MEMORY_SIZE );
pause( "allocate memory" );
m2 = malloc(TEST_MEMORY_SIZE);
if( m2 == NULL) {
printf( "Failed to allocated memory: %lu bytes\n",
TEST_MEMORY_SIZE );
}
else {
pause( "write to allocated memory" );
memset( m2, 66, TEST_MEMORY_SIZE );
pause( "deallocate memory" );
free( m2 );
}
pause( "exit program" );

return( 0 );
}

----- Snip ------

Regards,

Bob Trower

 

[s0suk3]

Note that some declarations, like the following at file scope:

int     bigarray[1024000] = {0};
might add nothing to the size of the executable (but will increase
the memory used at run time), but
int     bigarray[1024000] = {1};
might add about 4 megabytes (depending on the size of an int) to
the size of the executable and increase the memory used at run time.

I can't say for the size of the executable because I'm not familiar
with how that data is stored, but, why would it increase the memory
used at runtime? That declaration allocates 1024000 ints; what value
they are initialized to is irrelevant.

Also, note that an array initializer such as "{1}" will not initialize
all elements to 1. It will initialize the first element to 1, and the
rest to 0. This further makes me doubt that the size of the executable
will increase so much.

Sebastian

 

[Nate Eldredge]

Note that some declarations, like the following at file scope:

int     bigarray[1024000] = {0};
might add nothing to the size of the executable (but will increase
the memory used at run time), but
int     bigarray[1024000] = {1};
might add about 4 megabytes (depending on the size of an int) to
the size of the executable and increase the memory used at run time.

I can't say for the size of the executable because I'm not familiar
with how that data is stored, but, why would it increase the memory
used at runtime? That declaration allocates 1024000 ints; what value
they are initialized to is irrelevant.

I presume he means they both increase the memory used compared to not
having the array at all.

Also, note that an array initializer such as "{1}" will not initialize
all elements to 1. It will initialize the first element to 1, and the
rest to 0. This further makes me doubt that the size of the executable
will increase so much.

nate@vulcan:/tmp$ cat foo.c
int a[1000000] = { INIT };
int main(void) { return 0; }
nate@vulcan:/tmp$ cc -DINIT=0 -o foo0 foo.c
nate@vulcan:/tmp$ cc -DINIT=1 -o foo1 foo.c
nate@vulcan:/tmp$ ls -l foo?
-rwxr-xr-x 1 nate wheel 9146 Nov 22 13:40 foo0
-rwxr-xr-x 1 nate wheel 4009138 Nov 22 13:40 foo1

The usual Unix implementation has two sections for statically allocated
objects: "data" and "bss". The bss section is for objects that are
initialized entirely to zero; space for this section is zeroed out by
the loader, so there is no need to store the zeros in the executable.
But any object that is not *entirely* zeros cannot use this feature and
must go in the data section, where all of the data is loaded from the
executable. There is no way to split an object between the sections.

In principle the system could allocate the array in bss and explicitly
load in only the part that was initialized to something else, but I'm
not aware of such a feature actually being implemented.

 

[Bartc]

Note that some declarations, like the following at file scope:

int bigarray[1024000] = {0};
might add nothing to the size of the executable (but will increase
the memory used at run time), but
int bigarray[1024000] = {1};
might add about 4 megabytes (depending on the size of an int) to
the size of the executable and increase the memory used at run time.

I can't say for the size of the executable because I'm not familiar
with how that data is stored, but, why would it increase the memory
used at runtime? That declaration allocates 1024000 ints; what value
they are initialized to is irrelevant.

Also, note that an array initializer such as "{1}" will not initialize
all elements to 1. It will initialize the first element to 1, and the
rest to 0. This further makes me doubt that the size of the executable
will increase so much.

I've just tried it on two different compilers, and yes the executable is
about 4MB bigger (in those cases).

The fact that only one element is initialised makes no difference. Otherwise
you would have to invent some complex technique of initialising large,
sparse data structures.

 

[s0suk3]

nate@vulcan:/tmp$ cat foo.c
int a[1000000] = { INIT };
int main(void) { return 0; }
nate@vulcan:/tmp$ cc -DINIT=0 -o foo0 foo.c
nate@vulcan:/tmp$ cc -DINIT=1 -o foo1 foo.c
nate@vulcan:/tmp$ ls -l foo?
-rwxr-xr-x  1 nate  wheel     9146 Nov 22 13:40 foo0
-rwxr-xr-x  1 nate  wheel  4009138 Nov 22 13:40 foo1

The usual Unix implementation has two sections for statically allocated
objects: "data" and "bss".  The bss section is for objects that are
initialized entirely to zero; space for this section is zeroed out by
the loader, so there is no need to store the zeros in the executable.
But any object that is not *entirely* zeros cannot use this feature and
must go in the data section, where all of the data is loaded from the
executable.  There is no way to split an object between the sections.

In principle the system could allocate the array in bss and explicitly
load in only the part that was initialized to something else, but I'm
not aware of such a feature actually being implemented.

Thanks for the information. Turns out it's the same on Windows:

type foo.c

int a[1000000] = {INIT};
int main(void) { return 0; }

cl /DINIT=0 /Fefoo0.exe foo.c

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01
for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.

foo.c
Microsoft (R) Incremental Linker Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.

/out:foo0.exe
foo.obj

cl /DINIT=1 /Fefoo1.exe foo.c

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01
for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.

foo.c
Microsoft (R) Incremental Linker Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.

/out:foo1.exe
foo.obj

dir

....
22/11/2008 05:14 p.m. 36.864 foo0.exe
22/11/2008 05:15 p.m. 4.036.608 foo1.exe
....

Sebastian

 

[CBFalconer]

.... snip ...

Thanks for the information. Turns out it's the same on Windows:

type foo.c

int a[1000000] = {INIT};
int main(void) { return 0; }

cl /DINIT=0 /Fefoo0.exe foo.c

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01
for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.

foo.c
Microsoft (R) Incremental Linker Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.

/out:foo0.exe
foo.obj

cl /DINIT=1 /Fefoo1.exe foo.c

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01
for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.

foo.c
Microsoft (R) Incremental Linker Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.

/out:foo1.exe
foo.obj

dir

...
22/11/2008 05:14 p.m. 36.864 foo0.exe
22/11/2008 05:15 p.m. 4.036.608 foo1.exe

Which simply points out how little MS cares about generating
efficient program files. This is all system dependent and OT on
c.l.c.

 

[Nate Eldredge]

... snip ...

Thanks for the information. Turns out it's the same on Windows:

type foo.c

int a[1000000] = {INIT};
int main(void) { return 0; }

cl /DINIT=0 /Fefoo0.exe foo.c

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01
for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.

foo.c
Microsoft (R) Incremental Linker Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.

/out:foo0.exe
foo.obj

cl /DINIT=1 /Fefoo1.exe foo.c

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01
for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.

foo.c
Microsoft (R) Incremental Linker Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.

/out:foo1.exe
foo.obj

dir

...
22/11/2008 05:14 p.m. 36.864 foo0.exe
22/11/2008 05:15 p.m. 4.036.608 foo1.exe

Which simply points out how little MS cares about generating
efficient program files. This is all system dependent and OT on
c.l.c.

True, but I think the dig at MS is unjustified. Do you know of a
system that doesn't do the same?

 

[CBFalconer]

... snip ...

Thanks for the information. Turns out it's the same on Windows:

type foo.c
int a[1000000] = {INIT};
int main(void) { return 0; }

cl /DINIT=0 /Fefoo0.exe foo.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01
for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.

foo.c
Microsoft (R) Incremental Linker Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.

/out:foo0.exe
foo.obj

cl /DINIT=1 /Fefoo1.exe foo.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01
for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.

foo.c
Microsoft (R) Incremental Linker Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.

/out:foo1.exe
foo.obj

dir
...
22/11/2008 05:14 p.m. 36.864 foo0.exe
22/11/2008 05:15 p.m. 4.036.608 foo1.exe

Which simply points out how little MS cares about generating
efficient program files. This is all system dependent and OT
on c.l.c.

True, but I think the dig at MS is unjustified. Do you know of a
system that doesn't do the same?

A valid criticism. No, I don't know of such a system, but I
haven't looked to see what the systems I use do.

 

[Flash Gordon]

CBFalconer wrote, On 23/11/08 03:11:

(e-mail address removed) wrote:
... snip ...
Thanks for the information. Turns out it's the same on Windows:

type foo.c
int a[1000000] = {INIT};
int main(void) { return 0; }

cl /DINIT=0 /Fefoo0.exe foo.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01
for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.

foo.c
Microsoft (R) Incremental Linker Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.

/out:foo0.exe
foo.obj

cl /DINIT=1 /Fefoo1.exe foo.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01
for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.

foo.c
Microsoft (R) Incremental Linker Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.

/out:foo1.exe
foo.obj

dir
...
22/11/2008 05:14 p.m. 36.864 foo0.exe
22/11/2008 05:15 p.m. 4.036.608 foo1.exe
Which simply points out how little MS cares about generating
efficient program files. This is all system dependent and OT
on c.l.c.

True, but I think the dig at MS is unjustified. Do you know of a
system that doesn't do the same?

A valid criticism. No, I don't know of such a system, but I
haven't looked to see what the systems I use do.

I'm pretty sure that the Texas Instruments C compiler I used to use for
the TMS320C25 didn't do this. Admittedly this is an embedded system.

 

[Flash Gordon]

Gordon Burditt wrote, On 23/11/08 07:32:

I'll question whether this change would be *optimization*.

Disk space costs money especially when you get up to high end systems.

What
do compilers normally optimize for? CPU execution time and/or code
space. Not disk space for the executable.

Why not? Sometimes the size of the image is important.

Sometimes you can tell
it which to prefer, because code size optimization is often done
at the expense of execution time and vice versa (example: loop
unrolling). I don't know of any compiler that can be told to
optimize for executable size (other than options to turn off debugging
symbols or remove normal symbols).

Adding some startup code that executes before main() and initializes
the non-zero elements INCREASES execution time and code space. The

It is certainly done on compilers targeting embedded systems.

details of page faults and disk I/O for the two methods are very
system-dependent (reading from the executable may not be more costly
than reading from swap space). The startup code would definitely
be worse if the array were not accessed at all, and it might be
worse if only the beginning of the array were accessed.

There is no "definitely" about it. As you say above the details for the
two methods are very system dependant.

Permission to quote this message without providing proper attribution is
explicitly denied.

 

[jacob navia]

CBFalconer wrote, On 23/11/08 03:11:

(e-mail address removed) wrote:
... snip ...
Thanks for the information. Turns out it's the same on Windows:

type foo.c
int a[1000000] = {INIT};
int main(void) { return 0; }

cl /DINIT=0 /Fefoo0.exe foo.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01
for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.

foo.c
Microsoft (R) Incremental Linker Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.

/out:foo0.exe
foo.obj

cl /DINIT=1 /Fefoo1.exe foo.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01
for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.

foo.c
Microsoft (R) Incremental Linker Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.

/out:foo1.exe
foo.obj

dir
...
22/11/2008 05:14 p.m. 36.864 foo0.exe
22/11/2008 05:15 p.m. 4.036.608 foo1.exe
Which simply points out how little MS cares about generating
efficient program files. This is all system dependent and OT
on c.l.c.
True, but I think the dig at MS is unjustified. Do you know of a
system that doesn't do the same?

A valid criticism. No, I don't know of such a system, but I
haven't looked to see what the systems I use do.

I'm pretty sure that the Texas Instruments C compiler I used to use for
the TMS320C25 didn't do this. Admittedly this is an embedded system.

Neither does gcc
Neither does lcc-win

That is not an optimization, because initializing the array
at startup would slow down the program and would destroy the
working set!

 

[Flash Gordon]

jacob navia wrote, On 23/11/08 11:39:

CBFalconer wrote, On 23/11/08 03:11:

Nate Eldredge wrote:
(e-mail address removed) wrote:
... snip ...
Thanks for the information. Turns out it's the same on Windows:

type foo.c
int a[1000000] = {INIT};
int main(void) { return 0; }

cl /DINIT=0 /Fefoo0.exe foo.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01
for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.

foo.c
Microsoft (R) Incremental Linker Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.

/out:foo0.exe
foo.obj

cl /DINIT=1 /Fefoo1.exe foo.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01
for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.

foo.c
Microsoft (R) Incremental Linker Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.

/out:foo1.exe
foo.obj

dir
...
22/11/2008 05:14 p.m. 36.864 foo0.exe
22/11/2008 05:15 p.m. 4.036.608 foo1.exe
Which simply points out how little MS cares about generating
efficient program files. This is all system dependent and OT
on c.l.c.
True, but I think the dig at MS is unjustified. Do you know of a
system that doesn't do the same?
A valid criticism. No, I don't know of such a system, but I
haven't looked to see what the systems I use do.

I'm pretty sure that the Texas Instruments C compiler I used to use for
the TMS320C25 didn't do this. Admittedly this is an embedded system.

Typo alert. I meant *DID* do this. After all, the program is burnt in to
a PROM and the memory to be initialised in in RAM, so there has to be
some way for it to get from one to the other.

Neither does gcc
Neither does lcc-win

That is not an optimization, because initializing the array
at startup would slow down the program and would destroy the
working set!

Not always a problem.

 

[Mark L Pappin]

[nothing germane to my comments, but I've got jacob scored way down so
I usually don't see his posts unless somebody quotes them]

jacob navia wrote, On 23/11/08 11:39:

[compiled image includes large block of zeros to initialize array]

Rong.

Execution time and on-disk image size are not the only things for
which optimization is possible.

demonstrably_false_claims[jacob]++;

A chunk of code executed once only?

Do you even understand what a working set is?


mlp

 

[Nate Eldredge]

(e-mail address removed) wrote: [...]
Thanks for the information. Turns out it's the same on Windows: [...]
22/11/2008 05:14 p.m. 36.864 foo0.exe
22/11/2008 05:15 p.m. 4.036.608 foo1.exe

Which simply points out how little MS cares about generating
efficient program files. This is all system dependent and OT
on c.l.c.

True, but I think the dig at MS is unjustified. Do you know of a
system that doesn't do the same?

A valid criticism. No, I don't know of such a system, but I
haven't looked to see what the systems I use do.

The system I still code on regularly (old version of Mac OS Classic) with
the Metrowerks CodeWarrior compiler does NOT do this. First off, I have to
reference the array or else it'll eliminate it from the program entirely
(it uses a dead-stripping linker).

int a[1000000] = {INIT};
int main(void) { return a[1]; }

Then, it's 1296 bytes when INIT=0, 1299 when INIT=1. If I change the array to

float a[10][100000] = {{1},{2},{3},{4},{5},{6},{7},{8},{9},{10}};
int main(void) { return a[9][1]; }

then the program is 1384 bytes. I do see the 01 02 03 ... in the
executable, separated by other bytes, presumably telling that there are
100000*sizeof(int)-1 zero bytes between each. I couldn't find any
decompression code in the compiler's runtime sources, so I'm thinking the
OS did the decompression at program startup.

Interesting. I'm curious how it does it.