basic memory question

Discussion in 'C++' started by solartimba, Oct 9, 2003.

  1. solartimba

    solartimba Guest

    I am a stats man that uses C++ to analyze stock data, but I have run
    into a problem regarding memory usage.

    First, though, I need to give a little background info. I want to
    write a program that will open up 132 files containing daily stock
    market data. I estimate that each file contains around 91,000 bytes.
    Multiplied by 132, this equals 12 million bytes. Given that I have
    512 megs of memory, I thought that I would have no problem having all
    this data in memory at the same time. But just to be sure before I
    wrote all the code, I did a simple test of creating a very large array
    to find out what size would cause the system to crash. The test
    program's code is below:


    int main()
    {
    float array[259100]={4};
    array[258999]=2222;
    cout << "\nARRAY 1 " << array[258999];

    return 0;

    } //end main()

    This program crashes if the array size is increased beyond 259,000.
    Why? The array only takes up 1.04 million bytes.

    So, my question is: what do I need to do to have all that data in
    memory at the same time?

    Thanks
    solartimba, Oct 9, 2003
    #1
    1. Advertising

  2. "solartimba" <> wrote in message
    news:...
    > I am a stats man that uses C++ to analyze stock data, but I have run
    > into a problem regarding memory usage.

    ....
    > But just to be sure before I
    > wrote all the code, I did a simple test of creating a very large array
    > to find out what size would cause the system to crash. The test
    > program's code is below:

    ....
    > int main()
    > {
    > float array[259100]={4};
    > array[258999]=2222;
    > cout << "\nARRAY 1 " << array[258999];
    >
    > return 0;
    >
    > } //end main()


    This function allocates the array as a local variable
    of a fixed size (which typically means it is stored on
    the stack if you know waht this is).
    Local variables typically cannot be as large as
    arrays allocated dynamically (on the heap) using
    'new' or a standard container.

    > This program crashes if the array size is increased beyond 259,000.
    > Why? The array only takes up 1.04 million bytes.
    >
    > So, my question is: what do I need to do to have all that data in
    > memory at the same time?


    Use a container, such as std::vector:

    #include <vector>
    #include <iostream>

    int main()
    {
    std::vector<float> array;
    array.resize(1000000); // will just work
    array[258999] = 2222;
    std::cout << "\nARRAY 1 " << array[258999];
    return 0;
    }

    If you find this code hard to understand,
    I would recommend the book "Accelerated C++".
    It should help for the type of thing you
    are doing...


    I hope this helps,
    Ivan
    --
    http://ivan.vecerina.com
    http://www.brainbench.com <> Brainbench MVP for C++
    Ivan Vecerina, Oct 9, 2003
    #2
    1. Advertising

  3. solartimba

    Unforgiven Guest

    solartimba wrote:
    > I am a stats man that uses C++ to analyze stock data, but I have run
    > into a problem regarding memory usage.
    >
    > First, though, I need to give a little background info. I want to
    > write a program that will open up 132 files containing daily stock
    > market data. I estimate that each file contains around 91,000 bytes.
    > Multiplied by 132, this equals 12 million bytes. Given that I have
    > 512 megs of memory, I thought that I would have no problem having all
    > this data in memory at the same time. But just to be sure before I
    > wrote all the code, I did a simple test of creating a very large array
    > to find out what size would cause the system to crash. The test
    > program's code is below:
    >
    >
    > int main()
    > {
    > float array[259100]={4};
    > array[258999]=2222;


    You are allocating arrays on the stack. Stack space for a program is
    (usually) limited by the compiler. You must either increase the stack size
    (look at your compiler's documentation for that) or allocate the array on
    the heap. The best and safest way to do that is to use std::vector.

    --
    Unforgiven

    A: Top Posting!
    Q: What is the most annoying thing on Usenet?
    Unforgiven, Oct 9, 2003
    #3
  4. solartimba

    klaas Guest

    solartimba wrote:

    > I am a stats man that uses C++ to analyze stock data, but I have run
    > into a problem regarding memory usage.
    >
    > First, though, I need to give a little background info. I want to
    > write a program that will open up 132 files containing daily stock
    > market data. I estimate that each file contains around 91,000 bytes.
    > Multiplied by 132, this equals 12 million bytes. Given that I have
    > 512 megs of memory, I thought that I would have no problem having all
    > this data in memory at the same time. But just to be sure before I
    > wrote all the code, I did a simple test of creating a very large array
    > to find out what size would cause the system to crash. The test
    > program's code is below:
    >
    >

    well probably ypour floating point number does not fit in a byte right?
    probaly it fits in a word of four bytes. If you then multiply by four
    you'll get the right amount right?
    klaas, Oct 9, 2003
    #4
  5. solartimba

    Jerry Coffin Guest

    In article <>,
    says...
    > I am a stats man that uses C++ to analyze stock data, but I have run
    > into a problem regarding memory usage.
    >
    > First, though, I need to give a little background info. I want to
    > write a program that will open up 132 files containing daily stock
    > market data. I estimate that each file contains around 91,000 bytes.
    > Multiplied by 132, this equals 12 million bytes. Given that I have
    > 512 megs of memory, I thought that I would have no problem having all
    > this data in memory at the same time. But just to be sure before I
    > wrote all the code, I did a simple test of creating a very large array
    > to find out what size would cause the system to crash.


    First of all, you almost certainly want to use memory mapping to handle
    this. It's theoretically not portable, but nearly every reasonably
    modern OS supports it.

    The problem you're running into is that many systems limit the stack
    size to (considerably) less than all of memory. There are a couple of
    ways of handling this: you can allocate the space statically or on the
    heap, for a couple of the most obvious methods -- The easiest way is
    probably to use std::vector instead of an array, in which case the space
    will be allocated on the heap automatically.

    --
    Later,
    Jerry.

    The universe is a figment of its own imagination.
    Jerry Coffin, Oct 10, 2003
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. soni29
    Replies:
    6
    Views:
    396
    Kevin Goodsell
    Sep 5, 2003
  2. Engineer
    Replies:
    6
    Views:
    603
    Jeremy Bowers
    May 1, 2005
  3. Replies:
    0
    Views:
    417
  4. voidtwerp

    basic class memory use question

    voidtwerp, Jul 4, 2006, in forum: C++
    Replies:
    4
    Views:
    271
    Jim Langston
    Jul 4, 2006
  5. Jan Riechers
    Replies:
    6
    Views:
    169
    88888 Dihedral
    Jul 23, 2012
Loading...

Share This Page