How to create a process that uses 5Gb of memory?

[P]

I want to write a platform independent program that would use up at
least 5Gb of memory for testing purposes. How would I do that?

Thanks!

P

 

[Flash Gordon]

P wrote, On 03/08/07 21:56:

I want to write a platform independent program that would use up at
least 5Gb of memory for testing purposes. How would I do that?

It won't be very platform independent since most platforms do not have
5GB of memory and many that might will not allow one process to grab
that much. For those platforms that do then try mallocing a 5GB chunk,
if that fails try mallocing 1GB 5 times. Make sure you also write to the
memory otherwise you might not have what has been reported.

Mind you, I find it hard to see why you need to use up that much memory
for testing purposes.

 

[Walter Roberson]

I want to write a platform independent program that would use up at
least 5Gb of memory for testing purposes. How would I do that?

Not possible. Some platforms have an address space smaller than
5 Gb. It isn't uncommon on platforms with a 32 bit address space
for a maximum of 2 Gb to be usable by a program (with the
remaining 2 Gb of the address space reserved for the OS.) The
same applies to platforms that are internally more than 32 bits
but give each program a 32 bit virtual address space; on some
such systems, privileged programs could access more memory, but
the mechanisms for that would be completely system dependant.

You can probe the number of bits available in size_t and determine
if it has more than 32 bits; if it does, then you could attempt
to malloc() the 5 Gb. Note though that some systems will permit
you to malloc() the memory while not actually having that much
memory available, so to actually get ahold of the memory, you
may have to scribble something on every memory "page" (an OS
concept, not a C concept); unfortunately you may have to
make assumptions about the minimum size of a memory "page" in order
to do the scribbling in a reasonable time.

 

[Army1987]

I want to write a platform independent program that would use up at
least 5Gb of memory for testing purposes. How would I do that?

Am I actually reading "platform independent" and "5Gb" in the same
sentence?

(fx:removes glasses and looks closer)
Yes, I am.
The laptop I'm using right now has 1GB RAM and 1GB swap partition.
How can you use 5GB of memory on it?

 

[jacob navia]

Am I actually reading "platform independent" and "5Gb" in the same
sentence?

(fx:removes glasses and looks closer)
Yes, I am.
The laptop I'm using right now has 1GB RAM and 1GB swap partition.
How can you use 5GB of memory on it?

Like this:
#include <stdlib.h>
int main(void)
{
char *s = malloc(5*1024*1024*1024);
}

This is platform independent.

Whether it works is platform dependent.

 

[pete]

Like this:
#include <stdlib.h>
int main(void)
{
char *s = malloc(5*1024*1024*1024);
}

This is platform independent.

Whether it works is platform dependent.

Whether or not (5*1024*1024*1024) is undefined, is platform dependent.

 

[Army1987]

Like this:
#include <stdlib.h>
int main(void)
{
char *s = malloc(5*1024*1024*1024);
}

This is platform independent.

Whether it works is platform dependent.

C&V, please?
All of the operands of * have type int. Where the heck does the
standard guarantees that number to fit in a int?
Indeed, on my platform,
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
int i, j, k, l;
char *s = malloc(5 * 1024 * 1024 * 1024);
char *c = s;
if (s == NULL) { perror("malloc"); exit(EXIT_FAILURE); }
for (i = 0; i < 5; i++)
for (j = 0; j < 1024; j++)
for (k = 0; k < 1024; k++)
for (l = 0; l < 1024; l++)
*c++ = '_';
free(s);
return 0;
}
eventually segfaults, despite I *have* checked "whether it works".
(Understanding why it does is left as an exercise.)

 

[jacob navia]

Whether or not (5*1024*1024*1024) is undefined, is platform dependent.

Yes. Actually I should have tested for

if (32*sizeof(size_t) > 33 && s = malloc(...);

 

[jacob navia]

C&V, please?
All of the operands of * have type int. Where the heck does the
standard guarantees that number to fit in a int?
Indeed, on my platform,
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
int i, j, k, l;
char *s = malloc(5 * 1024 * 1024 * 1024);
char *c = s;
if (s == NULL) { perror("malloc"); exit(EXIT_FAILURE); }
for (i = 0; i < 5; i++)
for (j = 0; j < 1024; j++)
for (k = 0; k < 1024; k++)
for (l = 0; l < 1024; l++)
*c++ = '_';
free(s);
return 0;
}
eventually segfaults, despite I *have* checked "whether it works".
(Understanding why it does is left as an exercise.)

True. I should have added some test along the lines of
if (CHAR_BIT*sizeof(size_t) > 33 && s = malloc(...);

 

[jacob navia]

Yes. Actually I should have tested for

if (32*sizeof(size_t) > 33 && s = malloc(...);

AND SUBSTITUTE THAT "32" for CHAR_BIT!
(normally 8)

 

[Army1987]

True. I should have added some test along the lines of
if (CHAR_BIT*sizeof(size_t) > 33 && s = malloc(...);

If both the operands of * have type int, the result will have type
int, and *it* will be converted to size_t. If the result of the
multiplication doesn't fit in an int, the behavior is undefined.
You mean malloc((size_t)5 * 1024 * 1024 * 1024)?
Also, nothing forbids size_t from having padding bits, if it has
it is well possible that 5GB doesn't fit in it even if CHAR_BIT *
sizeof(size_t) > 33.

 

[Richard]

AND SUBSTITUTE THAT "32" for CHAR_BIT!
(normally 8)

Is a char a byte now? I lose track.

 

[jacob navia]

Is a char a byte now? I lose track.

No, CHAR_BIT is the number of bits in a char.

 

[Army1987]

AND SUBSTITUTE THAT "32" for CHAR_BIT!
(normally 8)

Or better
if ((size_t)(-1) / 1024 / 1024 / 1024 > 5
&& s = malloc((size_t)5 * 1024 * 1024 * 1024)
It will work regardless of padding bits in size_t.

 

[santosh]

Is a char a byte now? I lose track.

In C a char is always a byte.

 

[Peter J. Holzer]

Is a char a byte now? I lose track.

A "byte" in C-speak has been defined as the space occupied by a "char"
at least since C89.
In other contexts, a "byte" may be defined differently (e.g., as
"exactly 8 bits").

hp

 

[Richard]

A "byte" in C-speak has been defined as the space occupied by a "char"
at least since C89.
In other contexts, a "byte" may be defined differently (e.g., as
"exactly 8 bits").

So A byte is CHAR_BITS? Which is in 99.999% of the time 8 bits?

 

[santosh]

So A byte is CHAR_BITS?

In C, yes.

Which is in 99.999% of the time 8 bits?

Maybe not quite that much, if you consider the large embedded world, where
the bulk of C programming is currently happenning.

 

[Richard]

In C, yes.


Maybe not quite that much, if you consider the large embedded world, where
the bulk of C programming is currently happenning.

Venturing off topic here, how many of those system is a "byte" not 8
bits?

 

[Flash Gordon]

Richard wrote, On 06/08/07 11:55:

Venturing off topic here, how many of those system is a "byte" not 8
bits?

I don't know numbers, but certainly the C compiler for the Texas
Instruments TMS320C1x/2x/3x/4x/5x and I believe a lot of the other DSP
chips from other manufacturers as well.