fgets and problems reading into array

Discussion in 'C Programming' started by Eigenvector, Jul 26, 2003.

  1. Eigenvector

    Eigenvector Guest

    I've been dinking around with this code for a while now and I can't seem to
    figure out where the problem lies. The original problem was that I for some
    reason or another couldn't allocate few 80,000,000 element arrays. Normally
    that shouldn't be a problem but it was. So I tried to do some calloc calls
    and everything went south from there.

    Essentially I'm trying to read in a big file, stick each line into an array,
    then from then on do stuff to it. The do stuff part should work without a
    hitch, if I can just get that read in part to work.

    Here's the code.
    #include <stdio.h>
    #include <string.h>

    int main()
    {
    int i=0;
    int j=0;

    int eof_marker=0;

    char *array[1000000];
    char *out_array[1000000];

    char input_line[80];

    FILE *fp;
    FILE *fp_out;
    /*************************************************/

    fp=fopen("/var/tmp/cpu.out", "r");
    fp_out=fopen("/var/tmp/cpu.output2", "w");

    for (i=0; i<1000000; i++)
    array=calloc(80, sizeof(char));

    while(fgets(input_line, sizeof(input_line), fp) != NULL)
    {
    strcat(array, input_line); /* At this line the code core dumps */
    i++;
    }
    eof_marker=i;
    }

    Please forgive M$'s abominable text parsing, my neat indenting doesn't seem
    to have survived intact.

    I don't think I understand the logic behind malloc and calloc. I've read
    the FAQ and a few past threads from this group already - which is where my
    syntax comes from mostly. It just doesn't seem to be clicking.

    Rob
     
    Eigenvector, Jul 26, 2003
    #1
    1. Advertisements

  2. Eigenvector

    Artie Gold Guest


    ^^^^^^^^^^^^
    sizeof(char) is 1 by definition.

    ....but back to your problem. What is the value of `i' here?


    HTH,
    --ag
     
    Artie Gold, Jul 26, 2003
    #2
    1. Advertisements

  3. [snips]



    Three items on one line...

    1) why not allocate the length of the line actually read in? That way
    short lines don't waste space, long lines don't overflow.

    2) No check for calloc failure.

    3) sizeof(char)? Just use 1.

    */

    Assuming calloc properly zero-inits the buffer, that should be okay...
    except you don't even know if the allocation worked or not.

    'Course, the fact that i currently points past the _last_ element of the
    array might be an issue...
     
    Kelsey Bjarnason, Jul 26, 2003
    #3
  4. Eigenvector

    Eigenvector Guest



    Oh, crap!!!!!!! I didn't even think of that! I forgot to reset the value
    for i

    Please tell me that the value for 'i' was in fact the reason why my code
    bombed.

    You'll have to forgive some of the generic formatting and other strange
    anomolies in my code - some of it was straight plagarism just to figure out
    how the syntax for calloc worked. It'll get cleaned out when I actually get
    the code working.
     
    Eigenvector, Jul 26, 2003
    #4
  5. Eigenvector

    Burne C Guest



    I think "i" is the main problem, you didn't reset i = 0 before the while loop. Also, why do you use
    strcat instead of strcpy ? ( I know that it is not a problem, because calloc fills the array block
    with all zero. )

    BTW, you should add code to trap the failure of fopen and calloc.
     
    Burne C, Jul 26, 2003
    #5
  6. Eigenvector

    Randy Howard Guest

    You've already received several good responses to the program operation,
    but you should be aware that there is no guarantee that you can declare
    an array this large and have it work (portably). A fair number of
    compilers will bomb out on arrays that large. I believe the limit for
    the size of an array declared this way is 64KB in C99, and half that for
    C89, but I can't recall the "chapter and verse" on that as I can't
    remember the last time I even attempted to declare an array this large in
    this way.

    clc Faq entries 16.3 and 19.23 cover this, perhaps a bit obliquely.
    (http://www.eskimo.com/~scs/C-faq/faq.html)
     
    Randy Howard, Jul 27, 2003
    #6
  7. Eigenvector

    Eigenvector Guest

    Well that's kind of lame. I understand that not every computer has loads of
    memory, but putting an actual limit or even recognizing a limit seems
    artificial and unnecessary. I guess programmers need to make their code as
    efficient as possible at all times, but I intensely dislike those types of
    standards.

    64Kb is pretty damn small, maybe not for a PC, but certainly my 16 GB
    internal memory UNIX server shouldn't have this limit imposed on it. But
    like you said, its not guarenteed, and there are other more efficient ways
    of doing the job.
     
    Eigenvector, Jul 27, 2003
    #7
  8. Eigenvector wrote:

    The limit is not imposed on your implementation. It is imposed on programs
    which the programmer requires to be portable to any conforming C compiler.

    <snip>
     
    Richard Heathfield, Jul 27, 2003
    #8
  9. Eigenvector

    Kevin Easton Guest

    Don't think of it as telling you your array won't work if it's bigger
    than 64kB - instead think of it as promising that you that your array
    *will* work if it's smaller than 64kB.

    - Kevin.
     
    Kevin Easton, Jul 27, 2003
    #9
  10. [snips]

    It's not a limit; you can (perhaps) create objects terabytes in size.
    What it is is a statement that you cannot create objects larger than X and
    expect them to actually work across all conforming implementations.
     
    Kelsey Bjarnason, Jul 27, 2003
    #10
  11. Eigenvector

    Eigenvector Guest

    Okay, then perhaps I read it wrong. Then what the standard is saying is
    that 64 Kb is RAM size of a minimal system running C? something like that.

    Still, it seems artificial for C to be assuming system capacity in order for
    C standard conformity. What does the compiler or the language care about
    the upper bounds of memory allocation? What is so special about 64 Kb in
    the first place, at 64 Kb you might as well have 128 or 256 or 16 for that
    matter. Granted, not concerning yourself about resource allocation is what
    got us into that M$ Windows memory race that bloated computer systems
    everywhere - but still what is the 64 Kb concern - what is that 16 bits?
    Does that imply that C is still based in the 16 bit world?
     
    Eigenvector, Jul 28, 2003
    #11
  12. Eigenvector

    Eigenvector Guest

    I think I understand now. I was looking at it from the wrong perspective.

    I'm still a little hacked off that I couldn't get an 80 million element
    array to compile, but perhaps my OS manufacturer can patch that up.
    However, pertaining to my original problem, it works just fine now. A
    calloc call, resetting the value for i, and aways we go.

    Thanks all for the replies.
     
    Eigenvector, Jul 29, 2003
    #12
  13. I have found that even large systems can limit the size of static arrays. I
    believe it was on a DEC Alpha, running its preferred OS, with 64 bit
    pointers on a system with over 4GB of memory. I tried to create a static
    array of about 100,000 and it failed. malloc() would allocate many
    gigabytes, though.

    -- glen
     
    Glen Herrmannsfeldt, Jul 29, 2003
    #13
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.