Exception raising, and performance implications.

Discussion in 'Python' started by leo, Oct 3, 2005.

  1. leo

    leo Guest

    Hello all -

    I was wondering about the performance implications of explicitly
    raising exceptions to get information about the current frame.
    Something like what the inspect module does, with:

    ---
    def currentframe():
    """Return the frame object for the caller's stack frame."""
    try:
    raise 'catch me'
    except:
    return sys.exc_traceback.tb_frame.f_back
    ---

    I come from a java background, where Exceptions are a big Avoid Me, but
    are the performance implications the same in Python? We're expecting a
    big load on our app (100,000 users/hour) , so we'd like to be as tuned
    as possible.

    Thanks,
    leo
     
    leo, Oct 3, 2005
    #1
    1. Advertising

  2. leo

    Paul Rubin Guest

    "leo" <> writes:
    > I come from a java background, where Exceptions are a big Avoid Me, but
    > are the performance implications the same in Python?


    Well, you could measure it experimentally pretty easily, but anyway,
    Python exceptions are much less expensive than Java exceptions.
     
    Paul Rubin, Oct 3, 2005
    #2
    1. Advertising

  3. leo

    Guest

    As for performance, you'll need to benchmark it.

    However, I think the functionality you're asking for is available as
    inspect.currentframe(), and if the implementation is in "C" it may have a tiny
    performance advantage over the Python version.

    Jeff

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.1 (GNU/Linux)

    iD8DBQFDQah+Jd01MZaTXX0RAv0KAJ4/H3m7sSaszecPpA4hN6DX+hkN9QCfYy9F
    vc/97Vn8K+A30ZNAVKCi4R0=
    =OyAA
    -----END PGP SIGNATURE-----
     
    , Oct 3, 2005
    #3
  4. leo

    Tony Nelson Guest

    In article <>,
    "leo" <> wrote:

    > Hello all -
    >
    > I was wondering about the performance implications of explicitly
    > raising exceptions to get information about the current frame.
    > Something like what the inspect module does, with:


    Python uses exceptions internally, using StopIteration to terminate the
    iterator in a for: loop.

    > ---
    > def currentframe():
    > """Return the frame object for the caller's stack frame."""
    > try:
    > raise 'catch me'
    > except:
    > return sys.exc_traceback.tb_frame.f_back
    > ---


    This also does a traceback; you might want to measure the cost of that.

    > I come from a java background, where Exceptions are a big Avoid Me, but
    > are the performance implications the same in Python? We're expecting a
    > big load on our app (100,000 users/hour) , so we'd like to be as tuned
    > as possible.


    Switching to Python, eh? Remember to measure, measure, measure!
    ________________________________________________________________________
    TonyN.:' *firstname*nlsnews@georgea*lastname*.com
    ' <http://www.georgeanelson.com/>
     
    Tony Nelson, Oct 3, 2005
    #4
  5. leo

    Guest

    On Mon, Oct 03, 2005 at 02:34:40PM -0700, leo wrote:
    > I come from a java background, where Exceptions are a big Avoid Me, but
    > are the performance implications the same in Python? We're expecting a
    > big load on our app (100,000 users/hour) , so we'd like to be as tuned
    > as possible.


    I don't know what you do for each user, but here's a data point: My 1.8GHz
    Sempron (firmly in the "budget" category) uses a fairly unoptimized piece of
    software called aether[1] to serve my blog and a few other things. It uses
    Apache and good old fashioned cgi-bin one-process-per-request to serve up most
    pages. It parses a substantial amount of Python code each time with execfile
    (not using byte-compiled files). It still does this in 87ms on average (albeit
    for a simple page), or about 41k requests per hour.

    By simply buying a faster CPU, or by avoiding execfile, or by using a
    higher-performance technology than CGI, or with judicious use of caching, I'm
    sure I could reach 100,000 requests per hour, and by using several of the
    techniques together I might be able to reach 400k requests per hour. Probably
    it would be after these fairly obvious, high-level optimization ideas were
    exhausted that I would look at the code at a microscopic level for optimization
    ideas.

    But because my website is stuck on the slow end of a DSL line, there's not much
    point to any of this.

    Jeff
    [1] http://www.logarithmic.net/pfh/aether (not my site)

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.1 (GNU/Linux)

    iD8DBQFDQcd0Jd01MZaTXX0RAqMEAJ9UDEYsW+v9toSGbcepC1oH/PEeNACdHgQk
    tcUZnt/nBNFW+nNWhOKG29E=
    =56lv
    -----END PGP SIGNATURE-----
     
    , Oct 4, 2005
    #5
  6. leo

    leo Guest

    > However, I think the functionality you're asking for is available as
    > inspect.currentframe(), and if the implementation is in "C" it may have a tiny
    > performance advantage over the Python version.


    You're absolutely right, in fact the code snippet from my OP was taken
    directly from inspect.currentframe. We're intending on using this in
    production, and I'm trying to gauge what the implications may be.

    > Python uses exceptions internally, using StopIteration to terminate the
    > iterator in a for: loop.


    Wow, I was unaware of this. If Python internally uses exceptions, maybe
    they aren't as detrimental as I thought.

    That said, I will be judiciously profiling my code and measuring as
    much as possible, I just wanted to throw this question out to the NG in
    case anyone had done this before (and so I can put off learning the
    profiler a little bit longer :) )

    Thanks all for the replies.
     
    leo, Oct 4, 2005
    #6
  7. leo

    Steve Holden Guest

    leo wrote:
    >>However, I think the functionality you're asking for is available as
    >>inspect.currentframe(), and if the implementation is in "C" it may have a tiny
    >>performance advantage over the Python version.

    >
    >
    > You're absolutely right, in fact the code snippet from my OP was taken
    > directly from inspect.currentframe. We're intending on using this in
    > production, and I'm trying to gauge what the implications may be.
    >
    >
    >>Python uses exceptions internally, using StopIteration to terminate the
    >>iterator in a for: loop.

    >
    >
    > Wow, I was unaware of this. If Python internally uses exceptions, maybe
    > they aren't as detrimental as I thought.
    >
    > That said, I will be judiciously profiling my code and measuring as
    > much as possible, I just wanted to throw this question out to the NG in
    > case anyone had done this before (and so I can put off learning the
    > profiler a little bit longer :) )
    >
    > Thanks all for the replies.
    >

    Do note, however, that detecting an exception inside the C framework of
    the interpreter carries less overhead than detecting an exception in
    Python itself.

    That said, exceptions are probably rather more "lightweight" than you
    might imagine, so benchmarking (the profiler may not be best - have you
    come across "timeit.py"?) is the best way to go.

    regards
    Steve
    --
    Steve Holden +44 150 684 7255 +1 800 494 3119
    Holden Web LLC www.holdenweb.com
    PyCon TX 2006 www.python.org/pycon/
     
    Steve Holden, Oct 4, 2005
    #7
  8. leo wrote:
    >
    > You're absolutely right, in fact the code snippet from my OP was taken
    > directly from inspect.currentframe. We're intending on using this in
    > production, and I'm trying to gauge what the implications may be.


    Use sys._getframe() instead; it doesn't raise an exception.


    > Wow, I was unaware of this. If Python internally uses exceptions, maybe
    > they aren't as detrimental as I thought.


    Exceptions raised from C code and caught by C code use considerably
    less resources, so a "for" loop catching a StopIteration raised by a
    built-in iterator will have better performance than Python code
    catching an exception raised in Python code. So, it's not necessarily
    the case that the "for" loop scenario matches your scenario(s). Always
    measure.
     
    Phillip J. Eby, Oct 4, 2005
    #8
  9. leo

    Tom Anderson Guest

    On Mon, 3 Oct 2005, it was written:

    > "leo" <> writes:
    >
    >> I come from a java background, where Exceptions are a big Avoid Me, but
    >> are the performance implications the same in Python?

    >
    > Well, you could measure it experimentally pretty easily, but anyway,
    > Python exceptions are much less expensive than Java exceptions.


    Really? How come? What is it that stops java using the same technique as
    python? There's been quite a lot of work put into making java fast, so
    it'd be interesting if we had something they didn't.

    tom

    --
    What we learn about is not nature itself, but nature exposed to our methods of questioning. -- Werner Heisenberg
     
    Tom Anderson, Oct 5, 2005
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mark
    Replies:
    0
    Views:
    377
  2. Tom  Kerigan
    Replies:
    2
    Views:
    463
    Peter Flynn
    Oct 25, 2005
  3. Calvin Spealman

    Ideas for yielding and exception raising

    Calvin Spealman, Jun 5, 2004, in forum: Python
    Replies:
    3
    Views:
    319
    Peter Hansen
    Jun 8, 2004
  4. Replies:
    2
    Views:
    359
    Steve C. Orr [MVP, MCSD]
    Nov 11, 2006
  5. GreenLight
    Replies:
    3
    Views:
    201
    Anno Siegel
    May 1, 2004
Loading...

Share This Page