How to parse huge file with timestamps

J

jobs239

I need to parse a text file where each line is timestamped.
Each line has the form "timestamp, value1, value2"
I have to get max value of value1 between two timestamps and the the
avg value of value2 between another two timestamps. What would be an
efficient way to do this in terms of memory and processing time?
What comes to mind now is to read each line and store values in a
vector of objects. Then traverse that vector later to find the starting
timestamp and ending timestamp and process data in between. But it
doesn't sound like the most efficient way given that the files are huge.
 
R

Roedy Green

I have to get max value of value1 between two timestamps and the the
avg value of value2 between another two timestamps. What would be an
efficient way to do this in terms of memory and processing time?

you process the file sequentially, convert your timestamps to longs,
subtract and keep the longest value you find in variable called
longest.

if (diff > longest ) longest = diff;
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Staff online

Members online

Forum statistics

Threads
473,767
Messages
2,569,571
Members
45,045
Latest member
DRCM

Latest Threads

Top