How to parse huge file with timestamps

Discussion in 'Java' started by jobs239@hotmail.com, Nov 30, 2005.

  1. Guest

    I need to parse a text file where each line is timestamped.
    Each line has the form "timestamp, value1, value2"
    I have to get max value of value1 between two timestamps and the the
    avg value of value2 between another two timestamps. What would be an
    efficient way to do this in terms of memory and processing time?
    What comes to mind now is to read each line and store values in a
    vector of objects. Then traverse that vector later to find the starting
    timestamp and ending timestamp and process data in between. But it
    doesn't sound like the most efficient way given that the files are huge.
     
    , Nov 30, 2005
    #1
    1. Advertising

  2. Roedy Green Guest

    On 30 Nov 2005 14:04:10 -0800, wrote, quoted or
    indirectly quoted someone who said :

    > I have to get max value of value1 between two timestamps and the the
    >avg value of value2 between another two timestamps. What would be an
    >efficient way to do this in terms of memory and processing time?


    you process the file sequentially, convert your timestamps to longs,
    subtract and keep the longest value you find in variable called
    longest.

    if (diff > longest ) longest = diff;

    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
     
    Roedy Green, Nov 30, 2005
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    0
    Views:
    464
  2. Replies:
    3
    Views:
    553
  3. spaceman-spiff

    Trying to parse a HUGE(1gb) xml file

    spaceman-spiff, Dec 20, 2010, in forum: Python
    Replies:
    41
    Views:
    2,319
  4. spaceman-spiff

    Re: Trying to parse a HUGE(1gb) xml file

    spaceman-spiff, Dec 20, 2010, in forum: Python
    Replies:
    3
    Views:
    733
    John Nagle
    Dec 22, 2010
  5. nukeymusic

    replacing timestamps in file [newbie]

    nukeymusic, Dec 14, 2011, in forum: Python
    Replies:
    2
    Views:
    164
    Chris Angelico
    Dec 14, 2011
Loading...

Share This Page