malloc of large string works, but same sized array fails

Discussion in 'C Programming' started by mathog, Apr 14, 2014.

  1. mathog

    mathog Guest

    This is an odd one. Normally it doesn't matter if a string is allocated
    at run time or if it is set up as an array. In this case it does
    though. This causes a segfault at run time:

    #define MYMAXSTRING 20000000

    int main(int argc, char *argv[]){
    char bigstring[MYMAXSTRING];

    whereas this works with no problems

    #define MYMAXSTRING 20000000

    int main(int argc, char *argv[]){
    char *bigstring=NULL;


    bigstring=malloc(MYMAXSTRING);
    if(!bigstring){
    (void) fprintf(stderr,"Could not allocate initial memory\n");
    exit(EXIT_FAILURE);
    }


    When run in gdb it shows the segfault on the main() line. There is
    plenty of free memory on the system.

    It was compiled like:

    gcc -Wall -ansi -g -pedantic -o prog prog.c

    with gcc 4.2.3 (yes, that is pretty old).

    Why the difference?

    Thanks,

    David Mathog
     
    mathog, Apr 14, 2014
    #1
    1. Advertisements

  2. mathog

    Kaz Kylheku Guest

    All allocation happens at "run time".

    Even static arrays are allocated at run-time, at least on on a system where
    programs are not singly instantiated on some EEPROM chip, but loaded into
    memory.

    Something has to find space for the program and its static data; that happens
    at run time, though it is before your main function is called.
    This array is in autoamtic storage. Automatic storage is allocated at run time;
    often from a region of space called a "stack".

    Implementations often impose more stringent limits on automatic storage (local
    variables) than they do on dynamic storage (malloc).
    Are you on some kind of Unix? What is your system's limit on stack size?
     
    Kaz Kylheku, Apr 14, 2014
    #2
    1. Advertisements

  3. mathog

    mathog Guest

    That seems to be the issue:

    ulimit -a | grep stack

    stack size (kbytes, -s) 8192

    Thanks,

    David Mathog
     
    mathog, Apr 14, 2014
    #3
  4. On many systems, I would say that static data is allocated
    at link time. It is often given its run-time address (possibly
    with a load time offset) at link time.
    OK, "find space for", but the address is already assigned within
    that space. Note that it also has to "find space for" the stack,
    but you can still "allocate" on the stack.
    This problem occurs more often in Fortran programs, which more
    naturally allocate on the stack.

    Yesterday, in a different thread on a different news group, I was
    remembering an Alpha/OSF1 system that restricted static data
    to 64K, even though stack and malloc() could allocate GB.
    (Alpha is a 64 bit processor, but somehow uses 16 bits for static.)
    And, within the system limit, the shell can have its own limit.


    -- glen
     
    glen herrmannsfeldt, Apr 14, 2014
    #4
  5. mathog

    BartC Guest

    Is that a hard upper limit? Often the stack size is the initial size, which
    grows as needed. 8KB would be a ludicrous maximum stack size on a machine
    with presumably several GB of RAM. On my Windows machine, that 20MB
    allocation is no problem.

    Try ulimit -s unlimited
     
    BartC, Apr 14, 2014
    #5
  6. The "kbytes" indicates the units in which the number 8192 is expressed.
    The limit is 8 MB, not 8 KB.
    The problem with that is that anyone running your program will have to
    do the same thing to make it work. It's probably better to modify the
    program so it works with the default settings.
     
    Keith Thompson, Apr 14, 2014
    #6
  7. I remember the MS-DOS and OS/2 linker from MS which had a default
    something like 8K or so for way too long. At least with OS/2 where
    the stack was virtual.
    I suppose, but at some point it is a mistake.

    Historically, C programs tended to allocate large regions with
    malloc() and smaller ones with auto variables. One reason for that
    is that in K&R C and C89, auto arrays dimensions are compile time
    constants, and large ones tend to be run time variables.
    (The OP didn't explain where the large number came from.)

    Since C programs didn't tend to allocate large variables on
    the stack, the defaults were set small. The need for larger stack
    was rare enough not to bother most people.

    Fortran programs have tended to do more allocation on the stack,
    including larger arrays. Compilers will even generate large
    temporary arrays on the stack, sometimes surprising users.

    But at some point, it is a chicken and egg problem. People follow
    your suggestion and don't allocate large regions on the stack,
    shell writers use that as a reason to set a small default.

    -- glen
     
    glen herrmannsfeldt, Apr 15, 2014
    #7
  8. mathog

    Ike Naar Guest

    Just wondering, why not simply

    char *bigstring = malloc(MYMAXSTRING);

    ?
     
    Ike Naar, Apr 15, 2014
    #8
  9. or better still
    char *bigstring = malloc(MYMAXSTRING + 1);
     
    Malcolm McLean, Apr 15, 2014
    #9
  10. Is MYMAXSTRING the maximum *size* of the string or the maximum *length*
    of the string? How do you know?
     
    Keith Thompson, Apr 15, 2014
    #10
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.