Exception raising, and performance implications.

[leo]

Hello all -

I was wondering about the performance implications of explicitly
raising exceptions to get information about the current frame.
Something like what the inspect module does, with:

---
def currentframe():
"""Return the frame object for the caller's stack frame."""
try:
raise 'catch me'
except:
return sys.exc_traceback.tb_frame.f_back
---

I come from a java background, where Exceptions are a big Avoid Me, but
are the performance implications the same in Python? We're expecting a
big load on our app (100,000 users/hour) , so we'd like to be as tuned
as possible.

Thanks,
leo

 

[Paul Rubin]

I come from a java background, where Exceptions are a big Avoid Me, but
are the performance implications the same in Python?

Well, you could measure it experimentally pretty easily, but anyway,
Python exceptions are much less expensive than Java exceptions.

 

[jepler]

As for performance, you'll need to benchmark it.

However, I think the functionality you're asking for is available as
inspect.currentframe(), and if the implementation is in "C" it may have a tiny
performance advantage over the Python version.

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDQah+Jd01MZaTXX0RAv0KAJ4/H3m7sSaszecPpA4hN6DX+hkN9QCfYy9F
vc/97Vn8K+A30ZNAVKCi4R0=
=OyAA
-----END PGP SIGNATURE-----

 

[Tony Nelson]

Hello all -

I was wondering about the performance implications of explicitly
raising exceptions to get information about the current frame.
Something like what the inspect module does, with:

Python uses exceptions internally, using StopIteration to terminate the
iterator in a for: loop.

---
def currentframe():
"""Return the frame object for the caller's stack frame."""
try:
raise 'catch me'
except:
return sys.exc_traceback.tb_frame.f_back
---

This also does a traceback; you might want to measure the cost of that.

I come from a java background, where Exceptions are a big Avoid Me, but
are the performance implications the same in Python? We're expecting a
big load on our app (100,000 users/hour) , so we'd like to be as tuned
as possible.

Switching to Python, eh? Remember to measure, measure, measure!
________________________________________________________________________
TonyN.:' *firstname*nlsnews@georgea*lastname*.com
' <http://www.georgeanelson.com/>

 

[jepler]

I come from a java background, where Exceptions are a big Avoid Me, but
are the performance implications the same in Python? We're expecting a
big load on our app (100,000 users/hour) , so we'd like to be as tuned
as possible.

I don't know what you do for each user, but here's a data point: My 1.8GHz
Sempron (firmly in the "budget" category) uses a fairly unoptimized piece of
software called aether[1] to serve my blog and a few other things. It uses
Apache and good old fashioned cgi-bin one-process-per-request to serve up most
pages. It parses a substantial amount of Python code each time with execfile
(not using byte-compiled files). It still does this in 87ms on average (albeit
for a simple page), or about 41k requests per hour.

By simply buying a faster CPU, or by avoiding execfile, or by using a
higher-performance technology than CGI, or with judicious use of caching, I'm
sure I could reach 100,000 requests per hour, and by using several of the
techniques together I might be able to reach 400k requests per hour. Probably
it would be after these fairly obvious, high-level optimization ideas were
exhausted that I would look at the code at a microscopic level for optimization
ideas.

But because my website is stuck on the slow end of a DSL line, there's not much
point to any of this.

Jeff
[1] http://www.logarithmic.net/pfh/aether (not my site)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDQcd0Jd01MZaTXX0RAqMEAJ9UDEYsW+v9toSGbcepC1oH/PEeNACdHgQk
tcUZnt/nBNFW+nNWhOKG29E=
=56lv
-----END PGP SIGNATURE-----

 

[leo]

However, I think the functionality you're asking for is available as

inspect.currentframe(), and if the implementation is in "C" it may have a tiny
performance advantage over the Python version.

You're absolutely right, in fact the code snippet from my OP was taken
directly from inspect.currentframe. We're intending on using this in
production, and I'm trying to gauge what the implications may be.

Python uses exceptions internally, using StopIteration to terminate the
iterator in a for: loop.

Wow, I was unaware of this. If Python internally uses exceptions, maybe
they aren't as detrimental as I thought.

That said, I will be judiciously profiling my code and measuring as
much as possible, I just wanted to throw this question out to the NG in
case anyone had done this before (and so I can put off learning the
profiler a little bit longer )

Thanks all for the replies.

 

[Steve Holden]

You're absolutely right, in fact the code snippet from my OP was taken
directly from inspect.currentframe. We're intending on using this in
production, and I'm trying to gauge what the implications may be.




Wow, I was unaware of this. If Python internally uses exceptions, maybe
they aren't as detrimental as I thought.

That said, I will be judiciously profiling my code and measuring as
much as possible, I just wanted to throw this question out to the NG in
case anyone had done this before (and so I can put off learning the
profiler a little bit longer )

Thanks all for the replies.

Do note, however, that detecting an exception inside the C framework of
the interpreter carries less overhead than detecting an exception in
Python itself.

That said, exceptions are probably rather more "lightweight" than you
might imagine, so benchmarking (the profiler may not be best - have you
come across "timeit.py"?) is the best way to go.

regards
Steve

 

[Phillip J. Eby]

You're absolutely right, in fact the code snippet from my OP was taken
directly from inspect.currentframe. We're intending on using this in
production, and I'm trying to gauge what the implications may be.

Use sys._getframe() instead; it doesn't raise an exception.

Wow, I was unaware of this. If Python internally uses exceptions, maybe
they aren't as detrimental as I thought.

Exceptions raised from C code and caught by C code use considerably
less resources, so a "for" loop catching a StopIteration raised by a
built-in iterator will have better performance than Python code
catching an exception raised in Python code. So, it's not necessarily
the case that the "for" loop scenario matches your scenario(s). Always
measure.

 

[Tom Anderson]

Well, you could measure it experimentally pretty easily, but anyway,
Python exceptions are much less expensive than Java exceptions.

Really? How come? What is it that stops java using the same technique as
python? There's been quite a lot of work put into making java fast, so
it'd be interesting if we had something they didn't.

tom