How to create a process that uses 5Gb of memory?

Discussion in 'C Programming' started by P, Aug 3, 2007.

  1. P

    P Guest

    I want to write a platform independent program that would use up at
    least 5Gb of memory for testing purposes. How would I do that?

    Thanks!

    P
    P, Aug 3, 2007
    #1
    1. Advertising

  2. P

    Flash Gordon Guest

    P wrote, On 03/08/07 21:56:
    > I want to write a platform independent program that would use up at
    > least 5Gb of memory for testing purposes. How would I do that?


    It won't be very platform independent since most platforms do not have
    5GB of memory and many that might will not allow one process to grab
    that much. For those platforms that do then try mallocing a 5GB chunk,
    if that fails try mallocing 1GB 5 times. Make sure you also write to the
    memory otherwise you might not have what has been reported.

    Mind you, I find it hard to see why you need to use up that much memory
    for testing purposes.
    --
    Flash Gordon
    Flash Gordon, Aug 3, 2007
    #2
    1. Advertising

  3. In article <>,
    P <> wrote:
    >I want to write a platform independent program that would use up at
    >least 5Gb of memory for testing purposes. How would I do that?


    Not possible. Some platforms have an address space smaller than
    5 Gb. It isn't uncommon on platforms with a 32 bit address space
    for a maximum of 2 Gb to be usable by a program (with the
    remaining 2 Gb of the address space reserved for the OS.) The
    same applies to platforms that are internally more than 32 bits
    but give each program a 32 bit virtual address space; on some
    such systems, privileged programs could access more memory, but
    the mechanisms for that would be completely system dependant.

    You can probe the number of bits available in size_t and determine
    if it has more than 32 bits; if it does, then you could attempt
    to malloc() the 5 Gb. Note though that some systems will permit
    you to malloc() the memory while not actually having that much
    memory available, so to actually get ahold of the memory, you
    may have to scribble something on every memory "page" (an OS
    concept, not a C concept); unfortunately you may have to
    make assumptions about the minimum size of a memory "page" in order
    to do the scribbling in a reasonable time.
    --
    If you lie to the compiler, it will get its revenge. -- Henry Spencer
    Walter Roberson, Aug 3, 2007
    #3
  4. P

    Army1987 Guest

    On Fri, 03 Aug 2007 13:56:44 -0700, P wrote:

    > I want to write a platform independent program that would use up at
    > least 5Gb of memory for testing purposes. How would I do that?


    Am I actually reading "platform independent" and "5Gb" in the same
    sentence?

    (fx:removes glasses and looks closer)
    Yes, I am.
    The laptop I'm using right now has 1GB RAM and 1GB swap partition.
    How can you use 5GB of memory on it?
    --
    Army1987 (Replace "NOSPAM" with "email")
    "Never attribute to malice that which can be adequately explained
    by stupidity." -- R. J. Hanlon (?)
    Army1987, Aug 5, 2007
    #4
  5. P

    jacob navia Guest

    Army1987 wrote:
    > On Fri, 03 Aug 2007 13:56:44 -0700, P wrote:
    >
    >> I want to write a platform independent program that would use up at
    >> least 5Gb of memory for testing purposes. How would I do that?

    >
    > Am I actually reading "platform independent" and "5Gb" in the same
    > sentence?
    >
    > (fx:removes glasses and looks closer)
    > Yes, I am.
    > The laptop I'm using right now has 1GB RAM and 1GB swap partition.
    > How can you use 5GB of memory on it?


    Like this:
    #include <stdlib.h>
    int main(void)
    {
    char *s = malloc(5*1024*1024*1024);
    }

    This is platform independent.

    Whether it works is platform dependent.
    jacob navia, Aug 5, 2007
    #5
  6. P

    pete Guest

    jacob navia wrote:
    >
    > Army1987 wrote:
    > > On Fri, 03 Aug 2007 13:56:44 -0700, P wrote:
    > >
    > >> I want to write a platform independent program that would use up at
    > >> least 5Gb of memory for testing purposes. How would I do that?

    > >
    > > Am I actually reading "platform independent" and "5Gb" in the same
    > > sentence?
    > >
    > > (fx:removes glasses and looks closer)
    > > Yes, I am.
    > > The laptop I'm using right now has 1GB RAM and 1GB swap partition.
    > > How can you use 5GB of memory on it?

    >
    > Like this:
    > #include <stdlib.h>
    > int main(void)
    > {
    > char *s = malloc(5*1024*1024*1024);
    > }
    >
    > This is platform independent.
    >
    > Whether it works is platform dependent.


    Whether or not (5*1024*1024*1024) is undefined, is platform dependent.

    --
    pete
    pete, Aug 5, 2007
    #6
  7. P

    Army1987 Guest

    On Sun, 05 Aug 2007 13:03:49 +0200, jacob navia wrote:

    > Army1987 wrote:
    >> On Fri, 03 Aug 2007 13:56:44 -0700, P wrote:
    >>
    >>> I want to write a platform independent program that would use up at
    >>> least 5Gb of memory for testing purposes. How would I do that?

    >>
    >> Am I actually reading "platform independent" and "5Gb" in the same
    >> sentence?
    >>
    >> (fx:removes glasses and looks closer)
    >> Yes, I am.
    >> The laptop I'm using right now has 1GB RAM and 1GB swap partition.
    >> How can you use 5GB of memory on it?

    >
    > Like this:
    > #include <stdlib.h>
    > int main(void)
    > {
    > char *s = malloc(5*1024*1024*1024);
    > }
    >
    > This is platform independent.
    >
    > Whether it works is platform dependent.

    C&V, please?
    All of the operands of * have type int. Where the heck does the
    standard guarantees that number to fit in a int?
    Indeed, on my platform,
    #include <stdio.h>
    #include <stdlib.h>
    int main(void)
    {
    int i, j, k, l;
    char *s = malloc(5 * 1024 * 1024 * 1024);
    char *c = s;
    if (s == NULL) { perror("malloc"); exit(EXIT_FAILURE); }
    for (i = 0; i < 5; i++)
    for (j = 0; j < 1024; j++)
    for (k = 0; k < 1024; k++)
    for (l = 0; l < 1024; l++)
    *c++ = '_';
    free(s);
    return 0;
    }
    eventually segfaults, despite I *have* checked "whether it works".
    (Understanding why it does is left as an exercise.)
    --
    Army1987 (Replace "NOSPAM" with "email")
    "Never attribute to malice that which can be adequately explained
    by stupidity." -- R. J. Hanlon (?)
    Army1987, Aug 5, 2007
    #7
  8. P

    jacob navia Guest

    pete wrote:
    > jacob navia wrote:
    >> Army1987 wrote:
    >>> On Fri, 03 Aug 2007 13:56:44 -0700, P wrote:
    >>>
    >>>> I want to write a platform independent program that would use up at
    >>>> least 5Gb of memory for testing purposes. How would I do that?
    >>> Am I actually reading "platform independent" and "5Gb" in the same
    >>> sentence?
    >>>
    >>> (fx:removes glasses and looks closer)
    >>> Yes, I am.
    >>> The laptop I'm using right now has 1GB RAM and 1GB swap partition.
    >>> How can you use 5GB of memory on it?

    >> Like this:
    >> #include <stdlib.h>
    >> int main(void)
    >> {
    >> char *s = malloc(5*1024*1024*1024);
    >> }
    >>
    >> This is platform independent.
    >>
    >> Whether it works is platform dependent.

    >
    > Whether or not (5*1024*1024*1024) is undefined, is platform dependent.
    >


    Yes. Actually I should have tested for

    if (32*sizeof(size_t) > 33 && s = malloc(...);
    jacob navia, Aug 5, 2007
    #8
  9. P

    jacob navia Guest

    Army1987 wrote:
    > On Sun, 05 Aug 2007 13:03:49 +0200, jacob navia wrote:
    >
    >> Army1987 wrote:
    >>> On Fri, 03 Aug 2007 13:56:44 -0700, P wrote:
    >>>
    >>>> I want to write a platform independent program that would use up at
    >>>> least 5Gb of memory for testing purposes. How would I do that?
    >>> Am I actually reading "platform independent" and "5Gb" in the same
    >>> sentence?
    >>>
    >>> (fx:removes glasses and looks closer)
    >>> Yes, I am.
    >>> The laptop I'm using right now has 1GB RAM and 1GB swap partition.
    >>> How can you use 5GB of memory on it?

    >> Like this:
    >> #include <stdlib.h>
    >> int main(void)
    >> {
    >> char *s = malloc(5*1024*1024*1024);
    >> }
    >>
    >> This is platform independent.
    >>
    >> Whether it works is platform dependent.

    > C&V, please?
    > All of the operands of * have type int. Where the heck does the
    > standard guarantees that number to fit in a int?
    > Indeed, on my platform,
    > #include <stdio.h>
    > #include <stdlib.h>
    > int main(void)
    > {
    > int i, j, k, l;
    > char *s = malloc(5 * 1024 * 1024 * 1024);
    > char *c = s;
    > if (s == NULL) { perror("malloc"); exit(EXIT_FAILURE); }
    > for (i = 0; i < 5; i++)
    > for (j = 0; j < 1024; j++)
    > for (k = 0; k < 1024; k++)
    > for (l = 0; l < 1024; l++)
    > *c++ = '_';
    > free(s);
    > return 0;
    > }
    > eventually segfaults, despite I *have* checked "whether it works".
    > (Understanding why it does is left as an exercise.)


    True. I should have added some test along the lines of
    if (CHAR_BIT*sizeof(size_t) > 33 && s = malloc(...);
    jacob navia, Aug 5, 2007
    #9
  10. P

    jacob navia Guest

    jacob navia wrote:
    > pete wrote:
    >> jacob navia wrote:
    >>> Army1987 wrote:
    >>>> On Fri, 03 Aug 2007 13:56:44 -0700, P wrote:
    >>>>
    >>>>> I want to write a platform independent program that would use up at
    >>>>> least 5Gb of memory for testing purposes. How would I do that?
    >>>> Am I actually reading "platform independent" and "5Gb" in the same
    >>>> sentence?
    >>>>
    >>>> (fx:removes glasses and looks closer)
    >>>> Yes, I am.
    >>>> The laptop I'm using right now has 1GB RAM and 1GB swap partition.
    >>>> How can you use 5GB of memory on it?
    >>> Like this:
    >>> #include <stdlib.h>
    >>> int main(void)
    >>> {
    >>> char *s = malloc(5*1024*1024*1024);
    >>> }
    >>>
    >>> This is platform independent.
    >>>
    >>> Whether it works is platform dependent.

    >>
    >> Whether or not (5*1024*1024*1024) is undefined, is platform dependent.
    >>

    >
    > Yes. Actually I should have tested for
    >
    > if (32*sizeof(size_t) > 33 && s = malloc(...);
    >


    AND SUBSTITUTE THAT "32" for CHAR_BIT!
    (normally 8)
    jacob navia, Aug 5, 2007
    #10
  11. P

    Army1987 Guest

    On Sun, 05 Aug 2007 13:47:44 +0200, jacob navia wrote:

    > Army1987 wrote:
    >> On Sun, 05 Aug 2007 13:03:49 +0200, jacob navia wrote:
    >>
    >>> Army1987 wrote:
    >>>> On Fri, 03 Aug 2007 13:56:44 -0700, P wrote:
    >>>>
    >>>>> I want to write a platform independent program that would use up at
    >>>>> least 5Gb of memory for testing purposes. How would I do that?
    >>>> Am I actually reading "platform independent" and "5Gb" in the same
    >>>> sentence?
    >>>>
    >>>> (fx:removes glasses and looks closer)
    >>>> Yes, I am.
    >>>> The laptop I'm using right now has 1GB RAM and 1GB swap partition.
    >>>> How can you use 5GB of memory on it?
    >>> Like this:
    >>> #include <stdlib.h>
    >>> int main(void)
    >>> {
    >>> char *s = malloc(5*1024*1024*1024);
    >>> }
    >>>
    >>> This is platform independent.
    >>>
    >>> Whether it works is platform dependent.

    >> C&V, please?
    >> All of the operands of * have type int. Where the heck does the
    >> standard guarantees that number to fit in a int?
    >> Indeed, on my platform,
    >> #include <stdio.h>
    >> #include <stdlib.h>
    >> int main(void)
    >> {
    >> int i, j, k, l;
    >> char *s = malloc(5 * 1024 * 1024 * 1024);
    >> char *c = s;
    >> if (s == NULL) { perror("malloc"); exit(EXIT_FAILURE); }
    >> for (i = 0; i < 5; i++)
    >> for (j = 0; j < 1024; j++)
    >> for (k = 0; k < 1024; k++)
    >> for (l = 0; l < 1024; l++)
    >> *c++ = '_';
    >> free(s);
    >> return 0;
    >> }
    >> eventually segfaults, despite I *have* checked "whether it works".
    >> (Understanding why it does is left as an exercise.)

    >
    > True. I should have added some test along the lines of
    > if (CHAR_BIT*sizeof(size_t) > 33 && s = malloc(...);

    If both the operands of * have type int, the result will have type
    int, and *it* will be converted to size_t. If the result of the
    multiplication doesn't fit in an int, the behavior is undefined.
    You mean malloc((size_t)5 * 1024 * 1024 * 1024)?
    Also, nothing forbids size_t from having padding bits, if it has
    it is well possible that 5GB doesn't fit in it even if CHAR_BIT *
    sizeof(size_t) > 33.
    --
    Army1987 (Replace "NOSPAM" with "email")
    "Never attribute to malice that which can be adequately explained
    by stupidity." -- R. J. Hanlon (?)
    Army1987, Aug 5, 2007
    #11
  12. P

    Richard Guest

    jacob navia <> writes:

    > jacob navia wrote:
    >> pete wrote:
    >>> jacob navia wrote:
    >>>> Army1987 wrote:
    >>>>> On Fri, 03 Aug 2007 13:56:44 -0700, P wrote:
    >>>>>
    >>>>>> I want to write a platform independent program that would use up at
    >>>>>> least 5Gb of memory for testing purposes. How would I do that?
    >>>>> Am I actually reading "platform independent" and "5Gb" in the same
    >>>>> sentence?
    >>>>>
    >>>>> (fx:removes glasses and looks closer)
    >>>>> Yes, I am.
    >>>>> The laptop I'm using right now has 1GB RAM and 1GB swap partition.
    >>>>> How can you use 5GB of memory on it?
    >>>> Like this:
    >>>> #include <stdlib.h>
    >>>> int main(void)
    >>>> {
    >>>> char *s = malloc(5*1024*1024*1024);
    >>>> }
    >>>>
    >>>> This is platform independent.
    >>>>
    >>>> Whether it works is platform dependent.
    >>>
    >>> Whether or not (5*1024*1024*1024) is undefined, is platform dependent.
    >>>

    >>
    >> Yes. Actually I should have tested for
    >>
    >> if (32*sizeof(size_t) > 33 && s = malloc(...);
    >>

    >
    > AND SUBSTITUTE THAT "32" for CHAR_BIT!
    > (normally 8)


    Is a char a byte now? I lose track.
    Richard, Aug 6, 2007
    #12
  13. P

    jacob navia Guest

    Richard wrote:
    > jacob navia <> writes:
    >
    >> jacob navia wrote:
    >>> pete wrote:
    >>>> jacob navia wrote:
    >>>>> Army1987 wrote:
    >>>>>> On Fri, 03 Aug 2007 13:56:44 -0700, P wrote:
    >>>>>>
    >>>>>>> I want to write a platform independent program that would use up at
    >>>>>>> least 5Gb of memory for testing purposes. How would I do that?
    >>>>>> Am I actually reading "platform independent" and "5Gb" in the same
    >>>>>> sentence?
    >>>>>>
    >>>>>> (fx:removes glasses and looks closer)
    >>>>>> Yes, I am.
    >>>>>> The laptop I'm using right now has 1GB RAM and 1GB swap partition.
    >>>>>> How can you use 5GB of memory on it?
    >>>>> Like this:
    >>>>> #include <stdlib.h>
    >>>>> int main(void)
    >>>>> {
    >>>>> char *s = malloc(5*1024*1024*1024);
    >>>>> }
    >>>>>
    >>>>> This is platform independent.
    >>>>>
    >>>>> Whether it works is platform dependent.
    >>>> Whether or not (5*1024*1024*1024) is undefined, is platform dependent.
    >>>>
    >>> Yes. Actually I should have tested for
    >>>
    >>> if (32*sizeof(size_t) > 33 && s = malloc(...);
    >>>

    >> AND SUBSTITUTE THAT "32" for CHAR_BIT!
    >> (normally 8)

    >
    > Is a char a byte now? I lose track.


    No, CHAR_BIT is the number of bits in a char.
    jacob navia, Aug 6, 2007
    #13
  14. P

    Army1987 Guest

    On Sun, 05 Aug 2007 13:48:13 +0200, jacob navia wrote:

    > jacob navia wrote:
    >> pete wrote:
    >>> jacob navia wrote:
    >>>> Army1987 wrote:
    >>>>> On Fri, 03 Aug 2007 13:56:44 -0700, P wrote:
    >>>>>
    >>>>>> I want to write a platform independent program that would use up at
    >>>>>> least 5Gb of memory for testing purposes. How would I do that?
    >>>>> Am I actually reading "platform independent" and "5Gb" in the same
    >>>>> sentence?
    >>>>>
    >>>>> (fx:removes glasses and looks closer)
    >>>>> Yes, I am.
    >>>>> The laptop I'm using right now has 1GB RAM and 1GB swap partition.
    >>>>> How can you use 5GB of memory on it?
    >>>> Like this:
    >>>> #include <stdlib.h>
    >>>> int main(void)
    >>>> {
    >>>> char *s = malloc(5*1024*1024*1024);
    >>>> }
    >>>>
    >>>> This is platform independent.
    >>>>
    >>>> Whether it works is platform dependent.
    >>>
    >>> Whether or not (5*1024*1024*1024) is undefined, is platform dependent.
    >>>

    >>
    >> Yes. Actually I should have tested for
    >>
    >> if (32*sizeof(size_t) > 33 && s = malloc(...);
    >>

    >
    > AND SUBSTITUTE THAT "32" for CHAR_BIT!
    > (normally 8)

    Or better
    if ((size_t)(-1) / 1024 / 1024 / 1024 > 5
    && s = malloc((size_t)5 * 1024 * 1024 * 1024)
    It will work regardless of padding bits in size_t.
    --
    Army1987 (Replace "NOSPAM" with "email")
    "Never attribute to malice that which can be adequately explained
    by stupidity." -- R. J. Hanlon (?)
    Army1987, Aug 6, 2007
    #14
  15. P

    santosh Guest

    Richard wrote:

    > jacob navia <> writes:
    >
    >> jacob navia wrote:
    >>> pete wrote:
    >>>> jacob navia wrote:
    >>>>> Army1987 wrote:
    >>>>>> On Fri, 03 Aug 2007 13:56:44 -0700, P wrote:
    >>>>>>
    >>>>>>> I want to write a platform independent program that would use up at
    >>>>>>> least 5Gb of memory for testing purposes. How would I do that?
    >>>>>> Am I actually reading "platform independent" and "5Gb" in the same
    >>>>>> sentence?
    >>>>>>
    >>>>>> (fx:removes glasses and looks closer)
    >>>>>> Yes, I am.
    >>>>>> The laptop I'm using right now has 1GB RAM and 1GB swap partition.
    >>>>>> How can you use 5GB of memory on it?
    >>>>> Like this:
    >>>>> #include <stdlib.h>
    >>>>> int main(void)
    >>>>> {
    >>>>> char *s = malloc(5*1024*1024*1024);
    >>>>> }
    >>>>>
    >>>>> This is platform independent.
    >>>>>
    >>>>> Whether it works is platform dependent.
    >>>>
    >>>> Whether or not (5*1024*1024*1024) is undefined, is platform dependent.
    >>>>
    >>>
    >>> Yes. Actually I should have tested for
    >>>
    >>> if (32*sizeof(size_t) > 33 && s = malloc(...);
    >>>

    >>
    >> AND SUBSTITUTE THAT "32" for CHAR_BIT!
    >> (normally 8)

    >
    > Is a char a byte now? I lose track.


    In C a char is always a byte.
    santosh, Aug 6, 2007
    #15
  16. On 2007-08-06 08:18, Richard <> wrote:
    > Is a char a byte now? I lose track.


    A "byte" in C-speak has been defined as the space occupied by a "char"
    at least since C89.
    In other contexts, a "byte" may be defined differently (e.g., as
    "exactly 8 bits").

    hp

    --
    _ | Peter J. Holzer | I know I'd be respectful of a pirate
    |_|_) | Sysadmin WSR | with an emu on his shoulder.
    | | | |
    __/ | http://www.hjp.at/ | -- Sam in "Freefall"
    Peter J. Holzer, Aug 6, 2007
    #16
  17. P

    Richard Guest

    "Peter J. Holzer" <> writes:

    > On 2007-08-06 08:18, Richard <> wrote:
    >> Is a char a byte now? I lose track.

    >
    > A "byte" in C-speak has been defined as the space occupied by a "char"
    > at least since C89.
    > In other contexts, a "byte" may be defined differently (e.g., as
    > "exactly 8 bits").


    So A byte is CHAR_BITS? Which is in 99.999% of the time 8 bits?
    Richard, Aug 6, 2007
    #17
  18. P

    santosh Guest

    Richard wrote:

    > "Peter J. Holzer" <> writes:
    >
    >> On 2007-08-06 08:18, Richard <> wrote:
    >>> Is a char a byte now? I lose track.

    >>
    >> A "byte" in C-speak has been defined as the space occupied by a "char"
    >> at least since C89.
    >> In other contexts, a "byte" may be defined differently (e.g., as
    >> "exactly 8 bits").

    >
    > So A byte is CHAR_BITS?


    In C, yes.

    > Which is in 99.999% of the time 8 bits?


    Maybe not quite that much, if you consider the large embedded world, where
    the bulk of C programming is currently happenning.
    santosh, Aug 6, 2007
    #18
  19. P

    Richard Guest

    santosh <> writes:

    > Richard wrote:
    >
    >> "Peter J. Holzer" <> writes:
    >>
    >>> On 2007-08-06 08:18, Richard <> wrote:
    >>>> Is a char a byte now? I lose track.
    >>>
    >>> A "byte" in C-speak has been defined as the space occupied by a "char"
    >>> at least since C89.
    >>> In other contexts, a "byte" may be defined differently (e.g., as
    >>> "exactly 8 bits").

    >>
    >> So A byte is CHAR_BITS?

    >
    > In C, yes.
    >
    >> Which is in 99.999% of the time 8 bits?

    >
    > Maybe not quite that much, if you consider the large embedded world, where
    > the bulk of C programming is currently happenning.
    >


    Venturing off topic here, how many of those system is a "byte" not 8
    bits?
    Richard, Aug 6, 2007
    #19
  20. P

    Flash Gordon Guest

    Richard wrote, On 06/08/07 11:55:
    > santosh <> writes:
    >
    >> Richard wrote:
    >>
    >>> "Peter J. Holzer" <> writes:
    >>>
    >>>> On 2007-08-06 08:18, Richard <> wrote:
    >>>>> Is a char a byte now? I lose track.
    >>>> A "byte" in C-speak has been defined as the space occupied by a "char"
    >>>> at least since C89.
    >>>> In other contexts, a "byte" may be defined differently (e.g., as
    >>>> "exactly 8 bits").
    >>> So A byte is CHAR_BITS?

    >> In C, yes.
    >>
    >>> Which is in 99.999% of the time 8 bits?

    >> Maybe not quite that much, if you consider the large embedded world, where
    >> the bulk of C programming is currently happenning.
    >>

    >
    > Venturing off topic here, how many of those system is a "byte" not 8
    > bits?


    I don't know numbers, but certainly the C compiler for the Texas
    Instruments TMS320C1x/2x/3x/4x/5x and I believe a lot of the other DSP
    chips from other manufacturers as well.
    --
    Flash Gordon
    Flash Gordon, Aug 6, 2007
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Anders
    Replies:
    2
    Views:
    406
    Anders
    Sep 21, 2005
  2. Replies:
    3
    Views:
    505
  3. akineko
    Replies:
    3
    Views:
    2,597
    Jesse Noller
    Jan 29, 2009
  4. Qi Lu
    Replies:
    0
    Views:
    388
    Qi Lu
    Jul 14, 2008
  5. Mike Stead
    Replies:
    3
    Views:
    261
    Roger Pack
    Mar 25, 2010
Loading...

Share This Page