Re: Any way to turn off exception handling? (debugging)

Discussion in 'Python' started by Paul Rubin, Feb 11, 2010.

  1. Paul Rubin

    Paul Rubin Guest

    mk <> writes:
    >> Um... run your code in a debugger.

    >
    > ..except the code in question is multithreaded and pdb is no good for
    > that, and last time I checked, yappi was broken.


    Try winpdb.org.
    Paul Rubin, Feb 11, 2010
    #1
    1. Advertising

  2. Paul Rubin

    mk Guest


    > the uncommon, the exceptional, case. If one could somehow turn off
    > exceptions, you can't even do a for loop: every for loop would become
    > infinite-- exceptions are how Python signals the end of an iterator.
    > Think about that, *every* for loop in Python suddenly breaks.


    Ouch.

    > Sure I can, but how do I get out of Python the info *what called
    > particular class/instance*? That is, how do I know who's the caller?
    >
    >
    > Um... run your code in a debugger.


    ...except the code in question is multithreaded and pdb is no good for
    that, and last time I checked, yappi was broken.

    >
    > That's what debugger's are -for-.
    >
    > "pdb" is a simple one. Various IDE's have more advanced ones.


    I tried using pdb on paramiko and my own threaded code, lost some hair
    in the process, and gave up. It's good for tracing the main thread,
    basically, not for threading.Threads.

    > I think you're better served showing us what, "I'm getting an exception
    > (on socket) handled..." actually means, what is actually happening with
    > real code and output, and getting pointers on where to look, then trying
    > to gather material to support an honestly impossible change :)


    I'm using a threaded library paramiko (probably by now half of this
    group is sick of hearing me saying this again). 2 of (rather broken)
    hosts (Solaris, Linux) are really, really slow when responding over SSH,
    like a minute for a key exchange. In paramiko, timeout doesn't work bc
    the timeout specified there is timeout for a socket and not for SSH.

    When calling transport.socket.close() via threading.Timer like this:

    def printandclose(self):
    print "\ntrying to close transport.sock"
    self.trans.sock.close()
    self.flag = True


    self.flag = False
    timer = threading.Timer(4, self.printandclose)
    timer.start()
    self.trans.start_client()
    timer.cancel()

    .... I'm getting this on the console:

    ERROR (9, 'Bad file descriptor')

    According to a guy who is very knowledgable on the library (thanks for
    all the help, James) this is result of catching socket.error somewhere
    in paramiko code, like "except socket.error".

    I need to find that place. On top of lack of proper handling of
    timeouts, this is important bc when calling paramiko SSHClient.connect()
    (which in turn uses the same machinery as transport.start_client IIRC)
    paramiko uses close to 100% of CPU when negotiating with such broken or
    unresponsive hosts.

    For my reasons, I need to get this fixed.

    Regards,
    mk
    mk, Feb 11, 2010
    #2
    1. Advertising

  3. Paul Rubin

    Peter Otten Guest

    mk wrote:

    >
    >> the uncommon, the exceptional, case. If one could somehow turn off
    >> exceptions, you can't even do a for loop: every for loop would become
    >> infinite-- exceptions are how Python signals the end of an iterator.
    >> Think about that, *every* for loop in Python suddenly breaks.

    >
    > Ouch.
    >
    >> Sure I can, but how do I get out of Python the info *what called
    >> particular class/instance*? That is, how do I know who's the caller?
    >>
    >>
    >> Um... run your code in a debugger.

    >
    > ..except the code in question is multithreaded and pdb is no good for
    > that, and last time I checked, yappi was broken.
    >
    >>
    >> That's what debugger's are -for-.
    >>
    >> "pdb" is a simple one. Various IDE's have more advanced ones.

    >
    > I tried using pdb on paramiko and my own threaded code, lost some hair
    > in the process, and gave up. It's good for tracing the main thread,
    > basically, not for threading.Threads.
    >
    >> I think you're better served showing us what, "I'm getting an exception
    >> (on socket) handled..." actually means, what is actually happening with
    >> real code and output, and getting pointers on where to look, then trying
    >> to gather material to support an honestly impossible change :)

    >
    > I'm using a threaded library paramiko (probably by now half of this
    > group is sick of hearing me saying this again). 2 of (rather broken)
    > hosts (Solaris, Linux) are really, really slow when responding over SSH,
    > like a minute for a key exchange. In paramiko, timeout doesn't work bc
    > the timeout specified there is timeout for a socket and not for SSH.
    >
    > When calling transport.socket.close() via threading.Timer like this:
    >
    > def printandclose(self):
    > print "\ntrying to close transport.sock"
    > self.trans.sock.close()
    > self.flag = True
    >
    >
    > self.flag = False
    > timer = threading.Timer(4, self.printandclose)
    > timer.start()
    > self.trans.start_client()
    > timer.cancel()
    >
    > ... I'm getting this on the console:
    >
    > ERROR (9, 'Bad file descriptor')
    >
    > According to a guy who is very knowledgable on the library (thanks for
    > all the help, James) this is result of catching socket.error somewhere
    > in paramiko code, like "except socket.error".
    >
    > I need to find that place. On top of lack of proper handling of
    > timeouts, this is important bc when calling paramiko SSHClient.connect()
    > (which in turn uses the same machinery as transport.start_client IIRC)
    > paramiko uses close to 100% of CPU when negotiating with such broken or
    > unresponsive hosts.
    >
    > For my reasons, I need to get this fixed.


    You could try to shadow the exception class with None:

    >>> ZeroDivisionError = None
    >>> try:

    .... 1/0
    .... except ZeroDivisionError:
    .... print "caught"
    ....
    Traceback (most recent call last):
    File "<stdin>", line 2, in <module>
    ZeroDivisionError: integer division or modulo by zero

    Applying that technique on a module containing

    try:
    ...
    except socket.error:
    ...

    #untested
    import socket

    class SocketWrapper:
    def __getattr__(self, name):
    return getattr(socket, name)
    error = None

    import module_using_socket
    module_using_socket.socket = SocketWrapper()

    Peter
    Peter Otten, Feb 11, 2010
    #3
  4. Paul Rubin

    mk Guest

    Paul Rubin wrote:
    > mk <> writes:
    >>> Um... run your code in a debugger.

    >> ..except the code in question is multithreaded and pdb is no good for
    >> that, and last time I checked, yappi was broken.

    >
    > Try winpdb.org.


    This is a treasure! In minutes I've had this attached to remote process
    and debugging threads.

    Muchos gracias Paul!

    Regards,
    mk
    mk, Feb 11, 2010
    #4
  5. Paul Rubin

    mk Guest

    Peter Otten wrote:

    > try:
    > ...
    > except socket.error:
    > ...
    >
    > #untested
    > import socket
    >
    > class SocketWrapper:
    > def __getattr__(self, name):
    > return getattr(socket, name)
    > error = None
    >
    > import module_using_socket
    > module_using_socket.socket = SocketWrapper()


    Very interesting solution. Thanks!

    Regards,
    mk
    mk, Feb 11, 2010
    #5
  6. Paul Rubin

    Vinay Sajip Guest

    On Feb 11, 7:12 pm, mk <> wrote:
    > Peter Otten wrote:
    > > try:
    > >    ...
    > > except socket.error:
    > >    ...

    >
    > > #untested
    > > import socket

    >
    > > class SocketWrapper:
    > >     def __getattr__(self, name):
    > >         return getattr(socket, name)
    > >     error = None

    >
    > > import module_using_socket
    > > module_using_socket.socket = SocketWrapper()

    >
    > Very interesting solution. Thanks!
    >
    > Regards,
    > mk


    On Feb 11, 2:12 pm, mk <> wrote:
    > Peter Otten wrote:
    > > try:
    > > ...
    > > except socket.error:
    > > ...

    >
    > > #untested
    > > import socket

    >
    > > class SocketWrapper:
    > > def __getattr__(self, name):
    > > return getattr(socket, name)
    > > error = None

    >
    > > import module_using_socket
    > > module_using_socket.socket = SocketWrapper()

    >
    > Very interesting solution. Thanks!
    >
    > Regards,
    > mk


    You could refine Peter's suggestion further. For example, if you don't
    want to change the flow of execution (as Peter's suggestion does) but
    just log the occurrences, then you could try something like this:

    # socktest.py

    import socket

    def test():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    try:
    s.connect(('localhost', 9999))
    except socket.error, e:
    print '-- handler printing:', e

    # sockwrap.py

    import socket
    import traceback
    import logging

    class SocketWrapper:
    def __getattr__(self, name):
    return getattr(socket, name)

    @property
    def error(self):
    # In practice you could log to a specific logger
    logging.exception('A socket error occurred')
    return getattr(socket, 'error')

    def wrap(mod):
    mod.socket = SocketWrapper()

    # Now try it out

    >>> import socktest, sockwrap
    >>> sockwrap.wrap(socktest)
    >>> socktest.test()

    ERROR:root:A socket error occurred
    Traceback (most recent call last):
    File "socktest.py", line 7, in test
    s.connect(('localhost', 9999))
    File "<string>", line 1, in connect
    error: (10061, 'Connection refused')
    -- handler printing: (10061, 'Connection refused')
    >>>


    Regards,

    Vinay Sajip
    Vinay Sajip, Feb 12, 2010
    #6
  7. Paul Rubin

    Peter Otten Guest

    Duncan Booth wrote:

    > Peter Otten <> wrote:
    >
    >> You could try to shadow the exception class with None:


    > This works in Python 2.x but will break in Python 3. None is not a valid
    > exception specification and Python 3 will check for that and complain.


    > A better solution is to use an empty tuple as that is a valid exception
    > specification so will work in both Python 2.x and 3.x:


    Out of curiosity I tried a custom __subclasscheck__, but didn't get it to
    work in 3.1:

    $ cat instance_check.py
    class ZeroDivisionError(BaseException):
    class __metaclass__(type):
    def __subclasscheck__(self, other):
    print "performing subclass check --> %s" % catch
    return catch

    if __name__ == "__main__":
    for catch in [True, False]:
    print "catch = %s" % catch
    try:
    1/0
    except ZeroDivisionError:
    print "caught"

    $ python2.6 instance_check.py
    catch = True
    performing subclass check --> True
    caught
    catch = False
    performing subclass check --> False
    Traceback (most recent call last):
    File "instance_check.py", line 11, in <module>
    1/0
    ZeroDivisionError: integer division or modulo by zero
    $

    Peter
    Peter Otten, Feb 12, 2010
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. mortb
    Replies:
    2
    Views:
    421
    Peter Blum
    May 10, 2004
  2. Replies:
    2
    Views:
    383
    =?utf-8?B?SGFyYWxkIHZhbiBExLNr?=
    Aug 29, 2006
  3. robert112
    Replies:
    1
    Views:
    425
    Juan T. Llibre
    Apr 26, 2007
  4. mk
    Replies:
    0
    Views:
    195
  5. Immortal Nephi

    Turn off exception

    Immortal Nephi, Jul 27, 2010, in forum: C++
    Replies:
    2
    Views:
    390
    James Kanze
    Jul 30, 2010
Loading...

Share This Page