G
Guest
Hi
I am deciding how to animate some numerical data. So far, I have kept
the graphics and science(number crunching) codes separate which I
like.
The main issue confronting me is how to handle lots of data (usually
6000 data points, 4 quantities at each point, 24,000 numbers total)
in terms of optimization vs. effort. The 3 current options before me
are:
1) Have the science code write data to a file at each time step, and
then the graphics read it in at each step.
Advantage
1a) Minimal coding with just a little modification of existing code
1b) Maintains the separation between science and graphics
1c) Straightforward solution
Disadvantage
1a) It seems there is a high overhead. I've done some limited tests
and it takes about 1 sec to write and then read to the data
file.
Each numerical time step takes 1 sec. So I'll be doubling the
time
to run, and with 3600 steps, that goes from 1 to 2 hours(And 1
more
hour for animation).
1b) Fragment the drive by repeated write/re-write. Actually, I
don't
know if this is an issue, but I worry.
2) Combine the codes into a big code so that there is no reading or
writing.
Advantage
2a) No input/output overhead
2b) No time penalty related to 2a)
Disadvantage
1a) Lose everything about 1a), 1b), and 1c).
3) Someone said that if I wrote to stdout and read from stdin, and
pipe one
program through another, the data would stay in memory without
having to
be written out. In other words:
science-a.out | animate-a.out
I did some tests though, and it still seems to take a second to
write and
read the data.
I tried using the code profiler with the "-pg" flag but can't quite
set it up
to get anything.
So my questions are:
1) Should 3) be giving me better results than I seem to be getting?
2) Can you think of anything else I should try?
My machine is a 1.6 GhZ Pentium 4, 1 processor, and I'm coding on GNU/
Linux.
Graphics with OpenGL/Mesa, animation with ffmpeg.
I am guessing that without the read/write overhead, a standard
problem
would take 2 hours to run. With the overhead, it would take 3 hours.
Thanks.
San Le
slffea.com
I am deciding how to animate some numerical data. So far, I have kept
the graphics and science(number crunching) codes separate which I
like.
The main issue confronting me is how to handle lots of data (usually
6000 data points, 4 quantities at each point, 24,000 numbers total)
in terms of optimization vs. effort. The 3 current options before me
are:
1) Have the science code write data to a file at each time step, and
then the graphics read it in at each step.
Advantage
1a) Minimal coding with just a little modification of existing code
1b) Maintains the separation between science and graphics
1c) Straightforward solution
Disadvantage
1a) It seems there is a high overhead. I've done some limited tests
and it takes about 1 sec to write and then read to the data
file.
Each numerical time step takes 1 sec. So I'll be doubling the
time
to run, and with 3600 steps, that goes from 1 to 2 hours(And 1
more
hour for animation).
1b) Fragment the drive by repeated write/re-write. Actually, I
don't
know if this is an issue, but I worry.
2) Combine the codes into a big code so that there is no reading or
writing.
Advantage
2a) No input/output overhead
2b) No time penalty related to 2a)
Disadvantage
1a) Lose everything about 1a), 1b), and 1c).
3) Someone said that if I wrote to stdout and read from stdin, and
pipe one
program through another, the data would stay in memory without
having to
be written out. In other words:
science-a.out | animate-a.out
I did some tests though, and it still seems to take a second to
write and
read the data.
I tried using the code profiler with the "-pg" flag but can't quite
set it up
to get anything.
So my questions are:
1) Should 3) be giving me better results than I seem to be getting?
2) Can you think of anything else I should try?
My machine is a 1.6 GhZ Pentium 4, 1 processor, and I'm coding on GNU/
Linux.
Graphics with OpenGL/Mesa, animation with ffmpeg.
I am guessing that without the read/write overhead, a standard
problem
would take 2 hours to run. With the overhead, it would take 3 hours.
Thanks.
San Le
slffea.com