clocking subprocesses

[barnburnr]

Hi,

I've seen several threads on this subject, but haven't (yet) run
across one that answers my specific questions. This should be really
easy for someone, so here goes:

I'm running some numerical simulations under Ubuntu, and using Python
as my scripting language to automatically manage input and output. I
need to have a precise performance measurement (CPU time) of how long
it takes to run my simulations.

Right now I have something like:

stime = time.time()
subprocess.call(["./mysim","args"])
ftime = time.time()
print ftime-stime

However, time.time() only gives wall-clock time, so I'm also measuring
the time it takes to run other processes running at the same time.
What I'd rather have is:

stime = time.clock()
subprocess.call(["./mysim","args"])
ftime = time.clock()
print ftime-stime

But this, of course, usually outputs 0, because time.clock() does not
count the CPU ticks of the subprocess.

So, long story short, I need to get CPU time of something I call using
subprocess.call(). I don't want to have to profile my code, since it
will significantly reduce computation time.

Thanks for the advice.

Kevin

 

[Preston Landers]

So, long story short, I need to get CPU time of something I call using
subprocess.call().  

Run your command through the "time" program. You can parse the output
format of "time", or set a custom output format. This mostly applies
to Unix-like systems but there is probably an equivalent somewhere on
Windows.

Preston

 

[barnburnr]

Run your command through the "time" program. You can parse the output
format of "time", or set a custom output format. This mostly applies
to Unix-like systems but there is probably an equivalent somewhere on
Windows.

Preston

Thanks for the quick answer. That seems to work, though, I'll write a
timesubprocess() function which runs the program through time and
spits the formatted out to a file, then parses that file, then returns
the execution time. There doesn't appear to be a more elegant way to
do this.

Kevin

 

[Matt Nordhoff]

Thanks for the quick answer. That seems to work, though, I'll write a
timesubprocess() function which runs the program through time and
spits the formatted out to a file, then parses that file, then returns
the execution time. There doesn't appear to be a more elegant way to
do this.

Kevin

subprocess can do that easily.

What about something like this?

def timecall(args):
p = subprocess.Popen(['time', '--format', '%e %U %S'] + args,
stderr=subprocess.PIPE)
p.wait()
timings = p.stderr.readline().split()
assert len(timings) == 3
timings = tuple(float(t) for t in timings)
return timings

It returns (real, user, sys), in seconds. The program's stdout goes to
sys.stdout, I don't know where the program's stderr goes, and it really
doesn't handle errors, but it's got the basic idea.
--

 

[Karthik Gurusamy]

Hi,

I've seen several threads on this subject, but haven't (yet) run
across one that answers my specific questions. This should be really
easy for someone, so here goes:

I'm running some numerical simulations under Ubuntu, and using Python
as my scripting language to automatically manage input and output. I
need to have a precise performance measurement (CPU time) of how long
it takes to run my simulations.

Right now I have something like:

stime = time.time()
subprocess.call(["./mysim","args"])
ftime = time.time()
print ftime-stime

However, time.time() only gives wall-clock time, so I'm also measuring
the time it takes to run other processes running at the same time.
What I'd rather have is:

stime = time.clock()
subprocess.call(["./mysim","args"])
ftime = time.clock()
print ftime-stime

But this, of course, usually outputs 0, because time.clock() does not
count the CPU ticks of the subprocess.

So, long story short, I need to get CPU time of something I call using
subprocess.call(). I don't want to have to profile my code, since it
will significantly reduce computation time.

Use os.times(). It returns a 5-tuple and what you want is child cpu
times.

times(...)
times() -> (utime, stime, cutime, cstime, elapsed_time)

Return a tuple of floating point numbers indicating process times.

cutime+cstime will give you the total CPU used by child (your
simulation).

Karthik