spliting a list by nth items

D

Dan Perl

Steve Holden said:
As a recent convert to Wing I think you aren't giving them enough credit:
remember, this only happens in the 2.0 BETA product (or, if it also
happens in earlier versions, they are definitely planning to minimize the
effects in the production version).

According to the documentation this was happening in much earlier,
production, versions. You are saying "they are definitely planning to
minimize the effects in the production version". Can you please qualify
that? Minimize how? What does "minimizing the effects" mean anyway?
Well, I'm sure I remember reading somewhere that before they go into
production they plan to add a standard list of such exceptions for the
various Python versions, so that the standard libraries are far less
troublesome in this respect.

A "standard" list? I don't see what's "standard" about that. And the
"standard libraries" are not "troublesome", the IDE is. On the other hand,
I suppose that this would be an "effect minimization". I don't know what
their plans are, but they could include a list of all the possible false
positives and automatically ignore them. Forgive me, but that is a hack and
I will not have any respect for that.

You are using the same kind of marketing spin speak that Wingware is using.
I can understand that you like the tool and you feel you have to defend it.
But I think this is the wrong way to do it, both for you and for Wingware.

I probably shouldn't get so worked up about it. To be fair, since I tried
the tool I have discovered that Wing IDE also has the option to disable that
behavior and to debug in a mode where unhandled exceptions are just treated
like a normal run would treat them.
Plus, IIRC, all you have to do is check an "ignore this exception" box to
have a specific exception ignored everafter, which didn't seem like a huge
deal to me.

It's just too bad I don't have a checkbox for "Ignore marketing spin at this
location".

Dan
 
S

Steve Holden

Dan said:
[...]
As a recent convert to Wing I think you aren't giving them enough credit:
remember, this only happens in the 2.0 BETA product (or, if it also
happens in earlier versions, they are definitely planning to minimize the
effects in the production version).


According to the documentation this was happening in much earlier,
production, versions. You are saying "they are definitely planning to
minimize the effects in the production version". Can you please qualify
that? Minimize how? What does "minimizing the effects" mean anyway?
All I was doing was pointing out htat they are aware of hte phenomenon,
and that I (unlike you, apparently) am prepared to live with a minor nit
in order to get the advantages the tool gives me.
A "standard" list? I don't see what's "standard" about that. And the
"standard libraries" are not "troublesome", the IDE is. On the other hand,
I suppose that this would be an "effect minimization". I don't know what
their plans are, but they could include a list of all the possible false
positives and automatically ignore them. Forgive me, but that is a hack and
I will not have any respect for that.
Well, fine.
You are using the same kind of marketing spin speak that Wingware is using.
I can understand that you like the tool and you feel you have to defend it.
But I think this is the wrong way to do it, both for you and for Wingware.
Nope, I don;t feel I have to defend it. The tool does what it does, I
was merely sayiong that some people (such as me) can work with it as it
is. You apparently can't. Fine.
I probably shouldn't get so worked up about it. To be fair, since I tried
the tool I have discovered that Wing IDE also has the option to disable that
behavior and to debug in a mode where unhandled exceptions are just treated
like a normal run would treat them.
Well, yes, but I thought we were focusing ont he deficiencies of the
tool, not the advantages :)
It's just too bad I don't have a checkbox for "Ignore marketing spin at this
location".
Indeed. Hope you feel better now.
regards
Steve
 
I

Ian J Cottee

Dan said:
Right off the bat, the trial period was only 10 days. I've evaluated quite
a few tools that had a free trial period and I don't think I can remember of
ever having only 10 days before. Then, the GUI is kind of awkward in a few
places (the "project tree view", or something like that, and the toolbars).

In fairness, the initial period is 10 days but it can be renewed at the
end of that for another 10 days. The renewal procedure is done online
and takes about 10 seconds. I also *think* that during the beta period
you can repeatedly renew anyway.

I agree with your comments about the GUI - it's just so busy but I feel
that with all the alternatives. Apart from Emacs of course ;-).

Ian
 
M

Marco Aschwanden

I myself felt disturbed by your comments on WingIDE. But yes, I do
understand your nagging on "false exceptions" which can be turned off. You
turned it down because of this and because the short trial period (which
can be prolonged with 1 button click)... well, it didn't seem fair towards
the tool. It has its shortcomings but the ones you mention are ridiculous.

It beats Komodo 3.01 in some aspect:
- it shines when it comes to auto-completion. No need to feed some
"intelligent database" for each and every file you introduce like in
Komodo (first crash while building the database).
- in my case it works seamlessly with wxPython. Komodo crashed constantly.
Auto-completion with wxPython did never work (tried on 2 machines)
- WingIDE is much more robust although beta than Komodo, which crashed
about 5 times during 3h of work (Win2K; about 20 times because of wxPython
are not taken into account). WingIDE does not crash since around patch 3
(we are now at patch 8).


So I would say:

Use Komodo if you:
- are programming as well in other languages that Komodo supports (PHP,
Perl, Python, HTML, ...)
- need built-in cvs access
- can live with an IDE that crashes from time to time

Use Wingware if you:
- need a good "intelligent" auto-completion even for wxPython, pyQt, ...
- need an extensible IDE (you may in future add your own tools easily)
- need an IDE that focusses on Python and that tries to do the best on
this single language
- can live with an IDE that does not look as polished as Komodo

This comparison is not perfect. But you may realize that I turned it down
for it constantly crashing on me (which might be a problem of me and my 2
machines) and the useless auto-completion (constantly feeding the
database; does not work with wxPython). Those 2 points were in my case the
show stopper. Sad, because the idea behind Komodo is just great, the looks
is fantastic... I will give it another chance with 4.0...

Happy wing user,
Marco
 
D

Dan Perl

Marco Aschwanden said:
I myself felt disturbed by your comments on WingIDE. But yes, I do
understand your nagging on "false exceptions" which can be turned off. You
turned it down because of this and because the short trial period (which
can be prolonged with 1 button click)... well, it didn't seem fair towards
the tool. It has its shortcomings but the ones you mention are ridiculous.

The 10 day trial period just gave me a bad first impression and the
explanations for the false positives just left me with a bad taste. I gave
up on Wing because of the false positives (their simple existence, even
without the explanations) and the lack of a class browser. I encountered
the false positives first and I almost gave up right there. But it was just
the first day and I thought I should look more. Like you say, I wanted to
be fair. When I found that the class browser is not supported in the
Personal edition I threw my arms in the air.

To be fair again to Wing, I gave up on it based on the false positives
before finding out that I could have switched to another debug mode. But
that's their fault too because the widget that came up with the false
positive just says you can ignore the exception and then there was something
else just praising the tool, but there was no mention of the different debug
modes. I found that option only once I searched the manual. Maybe it's
just me, but I'm not reading the manuals when I'm trying out several tools.

My "nagging" on the false positives is mostly on the spin that Wingware puts
in their explanations. It's one thing to explain it the way that Stephan
did, that it was better to offer something imperfect if that was also
providing some important advantages and then also offer the other
alternative ("We weighed having a necessarily imperfect but useful feature
with not having the feature at all."). And it's another thing to insist
that this is not a problem and it is actually good for you. This doesn't
seem to bother other people as much as it bothered me, but I do have a beef
with it.

I'm curious to know what disturbed you in my comments.
It beats Komodo 3.01 in some aspect:
- it shines when it comes to auto-completion. No need to feed some
"intelligent database" for each and every file you introduce like in
Komodo (first crash while building the database).
- in my case it works seamlessly with wxPython. Komodo crashed constantly.
Auto-completion with wxPython did never work (tried on 2 machines)
- WingIDE is much more robust although beta than Komodo, which crashed
about 5 times during 3h of work (Win2K; about 20 times because of wxPython
are not taken into account). WingIDE does not crash since around patch 3
(we are now at patch 8).


So I would say:

Use Komodo if you:
- are programming as well in other languages that Komodo supports (PHP,
Perl, Python, HTML, ...)
- need built-in cvs access
- can live with an IDE that crashes from time to time

Use Wingware if you:
- need a good "intelligent" auto-completion even for wxPython, pyQt, ...
- need an extensible IDE (you may in future add your own tools easily)
- need an IDE that focusses on Python and that tries to do the best on
this single language
- can live with an IDE that does not look as polished as Komodo

This comparison is not perfect. But you may realize that I turned it down
for it constantly crashing on me (which might be a problem of me and my 2
machines) and the useless auto-completion (constantly feeding the
database; does not work with wxPython). Those 2 points were in my case the
show stopper. Sad, because the idea behind Komodo is just great, the looks
is fantastic... I will give it another chance with 4.0...

I used Komodo for about a month and it never crashed. As a matter of fact,
when the license expired, I let it run for several days and then I just had
to reboot my machine for other reasons (I am using Win XP, BTW). And I have
used it with wxPython already, although probably not as extensively as you
have.

I actually prefer the auto-completion in Komodo. Yes, it doesn't always
work because sometimes it has to learn first, but on the other hand I felt
that Wing tries to do too much. I don't need auto-completion for every
little name that I'm using and Wing's auto-completion was rather annoying me
even after I adjusted the pause period. I didn't find any option to disable
auto-completion selectively for things like locals. I also didn't see a
function usage showing up in Wing the way I am getting it in Komodo (and I
want that), but maybe that was a preference that I had to turn on and I
didn't know about it. I later saw something similar in a screen shot but
maybe that's also not in Wing's Personal edition.

So your 2 show stoppers do not apply for me. As for your other arguments, I
am using Komodo only for Python and not for any other languages and I am
using winCvs for CVS access. Still, during one month of use (more than a
month now), Komodo satisfied all my needs.
 
D

David Bolen

Dan Perl said:
The 10 day trial period just gave me a bad first impression and the
explanations for the false positives just left me with a bad taste. I gave
up on Wing because of the false positives (their simple existence, even
without the explanations) and the lack of a class browser. I encountered
the false positives first and I almost gave up right there. But it was just
the first day and I thought I should look more. Like you say, I wanted to
be fair. When I found that the class browser is not supported in the
Personal edition I threw my arms in the air.

For whatf it's worth, the "false positives" you refer to was a
significant reason that I really liked Wing.

If you ever use any code that runs Python through an extension
boundary (my key use is with wxPython, where all of your GUI objects
and event handlers run from within the wxPython extension), Wing is
one of the few debuggers that catches exceptions in that code. That's
because normally the exceptions are handled by the wxPython extension
module, and then suppressed so that they don't kill the main event
loop.

In my experience, there are minimal other false positives that a
normal application triggers (a consistent one I recall was in the sre
code), although the current 2.0 beta may still be getting tuned with
respect to its default "known" list, and I was more than willing to
mark those to be ignored in order to be able to be sure of catching
any exceptions within my own code, regardless of whether it was
triggered through an extension or not.
(...)
My "nagging" on the false positives is mostly on the spin that Wingware puts
in their explanations. It's one thing to explain it the way that Stephan
did, that it was better to offer something imperfect if that was also
providing some important advantages and then also offer the other
alternative ("We weighed having a necessarily imperfect but useful feature
with not having the feature at all."). And it's another thing to insist
that this is not a problem and it is actually good for you. This doesn't
seem to bother other people as much as it bothered me, but I do have a beef
with it.

The actually good for you part is arguably true, since otherwise you
have the potential to be generating exceptions in your own code and
not even know it. But I agree that different people will react
differently (note how differently you and I reacted to this feature),
so the more that can be done to explain the feature (and its
configurability) the better.

-- David
 
D

Dan Perl

David Bolen said:
For whatf it's worth, the "false positives" you refer to was a
significant reason that I really liked Wing.

If you ever use any code that runs Python through an extension
boundary (my key use is with wxPython, where all of your GUI objects
and event handlers run from within the wxPython extension), Wing is
one of the few debuggers that catches exceptions in that code. That's
because normally the exceptions are handled by the wxPython extension
module, and then suppressed so that they don't kill the main event
loop.

I'm new to wxPython so I cannot relate yet to what you are saying. The
wxPython extension module may catch the exceptions and suppress them, but
doesn't it generate a message for that or something? I hope that they are
not hiding the exception completely (and that they generate something) and
then you can still debug the problem with well placed breakpoints. Am I
wrong here or is there something I'm missing?
In my experience, there are minimal other false positives that a
normal application triggers (a consistent one I recall was in the sre
code), although the current 2.0 beta may still be getting tuned with
respect to its default "known" list, and I was more than willing to
mark those to be ignored in order to be able to be sure of catching
any exceptions within my own code, regardless of whether it was
triggered through an extension or not.

The false positive that I encountered was indeed in 'sre', invoked by
pickle. I think it was an IndexError which should be a normal thing if they
are using it to stop at the end of an iteration. So I'm not convinced this
would constitute "minimal" false positives.

Maybe if I would have been convinced first with a personal experience about
the advantages of Wing's exception detection at the point of raise, then I
would have accepted the false positives more easily. But seeing the false
exception first (actually right away and with positive spins on it) and then
finding out the right technical explanation later, it made it much harder to
swallow.

Here is my suggestion to Wingware. Make the "Never Stop" exception
reporting mode the default in the Personal edition or at least for the trial
download. A tutorial could then lead to the "Stop on Unhandled" mode.
The actually good for you part is arguably true, since otherwise you
have the potential to be generating exceptions in your own code and
not even know it. But I agree that different people will react
differently (note how differently you and I reacted to this feature),
so the more that can be done to explain the feature (and its
configurability) the better.

I would need to first see an example of generating an exception without
knowing it and that exception still being relevant although it is being
handled. I can imagine something not working right and an exception being
one more indication for the problem, but then there are still other good
ways to debug the problem. And if things are working, I don't see how
hidden exceptions may be of interest. I have very little experience with
GUIs but I realize that unit testing is much more difficult for GUIs. Does
that make it necessary to "hunt for exceptions" like that instead of testing
for results correctness? It may also be just my short experience with
Python and especially my total lack of experience with complex systems in
Python (e.g, Zope), so I'm ready to listen to an argument on that point.

Dan
 
D

David Bolen

Dan Perl said:
I'm new to wxPython so I cannot relate yet to what you are saying. The
wxPython extension module may catch the exceptions and suppress them, but
doesn't it generate a message for that or something? I hope that they are
not hiding the exception completely (and that they generate something) and
then you can still debug the problem with well placed breakpoints. Am I
wrong here or is there something I'm missing?

Yes, the wxPython layer will generally output an exception, but only
if there's a console available to receive it.

Even if you do have a console showing exceptions, not all exceptions
are cleanly reproduceable or predictable. I may have code that
typically runs just great, but that occasionally generates an
exception. Breakpoints may not work if the code runs many times
successfully before failing (due to varieties in calling parameters at
runtime). Perhaps a watchpoint might work but you'd have to know
enough about the failure to establish the watchpoint parameters.

Instead, with Wing, just start it up, let it run (or if you need to,
run it independently of Wing but with the stub loaded) until the
exception occurs. No hassles or up front work.
The false positive that I encountered was indeed in 'sre', invoked by
pickle. I think it was an IndexError which should be a normal thing if they
are using it to stop at the end of an iteration. So I'm not convinced this
would constitute "minimal" false positives.

Did you get any others? I was talking "minimal" in terms of volume,
not necessarily significance (which may be more subjective). I didn't
have much of an issue choosing to ignore the sre case (after which it
never showed up again for my project).
Maybe if I would have been convinced first with a personal experience about
the advantages of Wing's exception detection at the point of raise, then I
would have accepted the false positives more easily. But seeing the false
exception first (actually right away and with positive spins on it) and then
finding out the right technical explanation later, it made it much harder to
swallow.

That could well be. When I was testing Wing originally being able to
effectively work with legacy wxPython apps was a high priority for me,
both in terms of the auto-completion and in terms of exception
handling (the latter of which I had been disappointed to find not
working in other IDEs I had tried to that point). So I was probably
predisposed to not worry too much about the side-effects of
functionality that at least got me the exception processing I wanted.

(...)
I would need to first see an example of generating an exception without
knowing it and that exception still being relevant although it is being
handled.

I'd split that into two parts - (a) generating the exception without
knowing it, and (b) it being relevant even if handled.

For (a), I can't seem to come up with a good concise example, so maybe
I'll just agree that you may know about, but not be able to act on it
in a useful fashion. (The cases I was thinking of were GUI apps
without a console, or apps generating output where a traceback might
be missed, but I can come up with reasonable solutions in those cases
if you knew you were looking for an exception, so I'm willing to bet
you can too. The other case was with a locally developed extension, but
that's susceptible to the argument of just fixing the local extension :))

Note however, that knowing about it in many cases may only be through
a stderr traceback, so to know it other than manually inspecting it at
runtime would require trapping stderr in general and logging that
somewhere, and then having something to recognize tracebacks in that
log.

(b) is certainly possible though, although at the risk of overuse I'll
fall back to wxPython again. In wxPython any exception that occurs in
an event handler will be handled by the wxPython extension, and is
thus invisible to top level code. So if you have an exception in an
event handler, it interrupts the exception handler, which presumably
no longer does what it is supposed to, so that's relevant. The
exception will not, however, stop the application, nor bubble up to
any top level application exception handler. The only place you'd see
an indication of it would be in the traceback to stderr - your
application has no opportunity to handle it otherwise (unless you have
exception handlers in each individual event handler).

Now in normal operation that's actually a very attractive part of
wxPython. By and large you can create problems, and the application
keeps running. Perhaps one element of the GUI might not react
properly or some button won't work, but the application survives. But
when trying to troubleshoot that problem, it's often most convenient
to interact with the application as a user until the problem occurs,
letting the debugger simply catch things at the point when it happens.
I can imagine something not working right and an exception being
one more indication for the problem, but then there are still other good
ways to debug the problem.

I'd certainly agree with that. As with everything, the debugger is
just one element in the toolkit. But even if you have an exception
being generated, there is real value to being able to stop and
dynamically inspect the state of the system at the point of exception,
particularly when an exception is sporadic or difficult to reproduce.

Having other methods to also debug it doesn't negate the value of having
a debugger that can run an application until such an event takes place and
then give you direct access to the state of the system at that point.
And if things are working, I don't see how
hidden exceptions may be of interest. I have very little experience with
GUIs but I realize that unit testing is much more difficult for GUIs. Does
that make it necessary to "hunt for exceptions" like that instead of testing
for results correctness? It may also be just my short experience with
Python and especially my total lack of experience with complex systems in
Python (e.g, Zope), so I'm ready to listen to an argument on that point.

I definitely have multiple categories of code. New projects that can
make use of TDD approaches do so and have far less of a requirement
(IMO so far) of debuggers in general. They're generally also more
amenable to "print" debugging since issues arise that tend to be more
focused, and occurring in well identified areas of the code under well
identified conditions.

Older projects not written in a TDD fashion, or those for which unit
testing is difficult (in my environment typically GUI, threading, and
network in some mixture) often run into scenarios where you need a
lengthy runtime or real world usage to tickle sporadic or
non-deterministic bugs. In those cases, a solid debugger with the
ability to stop at any moment when a problem arises can be worth its
weight in gold - even if such occurrances may be few and far between.
I believe Alex Martelli recently posted about such a case in a web
application, for example, where a debugger ended up very useful in
addition to the other techniques being applied.

-- David
 
J

John J. Lee

Dan Perl said:
Marco Aschwanden said:
I myself felt disturbed by your comments on WingIDE. But yes, I do
understand your nagging on "false exceptions" which can be turned off. You
turned it down because of this and because the short trial period (which
can be prolonged with 1 button click)... well, it didn't seem fair towards
the tool. It has its shortcomings but the ones you mention are ridiculous.
[...]
To be fair again to Wing, I gave up on it based on the false positives
before finding out that I could have switched to another debug mode. But
that's their fault too because the widget that came up with the false
positive just says you can ignore the exception and then there was something
else just praising the tool, but there was no mention of the different debug
modes. I found that option only once I searched the manual. Maybe it's
just me, but I'm not reading the manuals when I'm trying out several tools.

My "nagging" on the false positives is mostly on the spin that Wingware puts
in their explanations. It's one thing to explain it the way that Stephan
did, that it was better to offer something imperfect if that was also
providing some important advantages and then also offer the other
alternative ("We weighed having a necessarily imperfect but useful feature
with not having the feature at all."). And it's another thing to insist
that this is not a problem and it is actually good for you. This doesn't
seem to bother other people as much as it bothered me, but I do have a beef
with it.

Dan, I suspect that (though perhaps their marketing may have room for
improvement here ;-) there may have been an unstated assumption on the
part of the Wing IDE people that, "obviously", the false-positives
issue is only a "nit" if one has a hard debugging problem on one's
hands, and that "obviously", one turns this feature off under all
other circumstances. Not necessarily obvious at all, of course.

Perhaps, like many of the rest of us, they only wheel out their
debugger when they hit a really *nasty* problem, so the exception
issue simply never arises as anything other than a minor wart in what
I can see could be a very handy feature. That would explain why this
feature is on by default. Another explanation is that it's turned on
by default just to advertise the feature -- if you don't bump into it
like this, you might never realise it existed.


John
 
D

Dan Perl

David Bolen said:
...................
Even if you do have a console showing exceptions, not all exceptions
are cleanly reproduceable or predictable. I may have code that
typically runs just great, but that occasionally generates an
exception. Breakpoints may not work if the code runs many times
successfully before failing (due to varieties in calling parameters at
runtime). Perhaps a watchpoint might work but you'd have to know
enough about the failure to establish the watchpoint parameters.

Instead, with Wing, just start it up, let it run (or if you need to,
run it independently of Wing but with the stub loaded) until the
exception occurs. No hassles or up front work.

Okay, I can see this point. The idea of the loaded stub sounds especially
interesting because you can just run the application normally, independent
of an IDE. In my experience with sporadic problems like the ones you are
describing, debuggers are usually not much help, except after the fact, for
instance debugging a core of a C/C++ program. If the problem is caused by a
race condition, a debugger can be useless because it affects the scheduling
of threads. How does the loaded stub work in cases like that? Does it
affect threads in any way?
Did you get any others? I was talking "minimal" in terms of volume,
not necessarily significance (which may be more subjective). I didn't
have much of an issue choosing to ignore the sre case (after which it
never showed up again for my project).

It was the only one. My point was that even just this particular exception
should happen very often. So even volume, not only significance, is
relative. I, for one, was not comfortable ignoring an exception, especially
without a very clear and detailed explanation on why the false positive
happened in the first place and what happens with the exception if I ignore
it.
(...)

I'd split that into two parts - (a) generating the exception without
knowing it, and (b) it being relevant even if handled.

For (a), I can't seem to come up with a good concise example, so maybe
I'll just agree that you may know about, but not be able to act on it
in a useful fashion. (The cases I was thinking of were GUI apps
without a console, or apps generating output where a traceback might
be missed, but I can come up with reasonable solutions in those cases
if you knew you were looking for an exception, so I'm willing to bet
you can too. The other case was with a locally developed extension, but
that's susceptible to the argument of just fixing the local extension :))

Note however, that knowing about it in many cases may only be through
a stderr traceback, so to know it other than manually inspecting it at
runtime would require trapping stderr in general and logging that
somewhere, and then having something to recognize tracebacks in that
log.

(b) is certainly possible though, although at the risk of overuse I'll
fall back to wxPython again. In wxPython any exception that occurs in
an event handler will be handled by the wxPython extension, and is
thus invisible to top level code. So if you have an exception in an
event handler, it interrupts the exception handler, which presumably
no longer does what it is supposed to, so that's relevant. The
exception will not, however, stop the application, nor bubble up to
any top level application exception handler. The only place you'd see
an indication of it would be in the traceback to stderr - your
application has no opportunity to handle it otherwise (unless you have
exception handlers in each individual event handler).

Now in normal operation that's actually a very attractive part of
wxPython. By and large you can create problems, and the application
keeps running. Perhaps one element of the GUI might not react
properly or some button won't work, but the application survives. But
when trying to troubleshoot that problem, it's often most convenient
to interact with the application as a user until the problem occurs,
letting the debugger simply catch things at the point when it happens.

You're making a very good case, assuming that the exception is at the root
of the problem. However, it's hard to say how many such problems are caused
by an exception. Even if there is an exception, the root problem may be way
before that and you still have to go and do some normal debugging. So many
times I've had memory corruption crashing an application in C++ and I had to
look back at a method that was called WAY BEFORE the crash because that was
usually corrupting memory at exit.

So, in your example of a button, my general debugging approach would still
be to put a breakpoint in the event handler of the button, because the root
of the problem can be so many different things.

In your experience with Python, what percentage of problems manifested
themselves with exceptions (especially right at the root of the problem) as
opposed to simply flawed logic that gets through without any exceptions?

I think we are actually very much agreeing. The main difference between us
is how much weight we put in debugging exceptions versus debugging all kinds
of problems and how much we are willing to put up with in exchange for that
precious advantage in a few cases.

You are saying a solid debugger "can be worth its weight in gold". How much
does Wing IDE weigh? ;-)

Dan
 
D

Dan Perl

John J. Lee said:
Dan, I suspect that (though perhaps their marketing may have room for
improvement here ;-) there may have been an unstated assumption on the
part of the Wing IDE people that, "obviously", the false-positives
issue is only a "nit" if one has a hard debugging problem on one's
hands, and that "obviously", one turns this feature off under all
other circumstances. Not necessarily obvious at all, of course.

Perhaps, like many of the rest of us, they only wheel out their
debugger when they hit a really *nasty* problem, so the exception
issue simply never arises as anything other than a minor wart in what
I can see could be a very handy feature. That would explain why this
feature is on by default. Another explanation is that it's turned on
by default just to advertise the feature -- if you don't bump into it
like this, you might never realise it existed.

If Stephan is still reading this thread he may contradict this, but my
suspicion is that your last explanation is the real one. Such an approach
can also backfire like it did in my case.
 
D

David Bolen

Dan Perl said:
(...) In my experience with sporadic problems like the ones you are
describing, debuggers are usually not much help, except after the fact, for
instance debugging a core of a C/C++ program. If the problem is caused by a
race condition, a debugger can be useless because it affects the scheduling
of threads. How does the loaded stub work in cases like that? Does it
affect threads in any way?

Yes, once the debugger is actually activated (you start up the IDE and
it attaches) it's like a normal Python debugger, so it uses the trace
hook and will have some impact on runtime. You can finesse this a little by
controlling where you import the stub and/or when you connect to the process.

Of course, in my experience just about anything you do to try to work
with a subtle threading or race condition bug is just as likely to
affect the problem (including adding a new line of code), so you can
still hope that the problem will simply exhibit itself as well - even
if slightly differently - with the debugger attached. Obviously, no
guarantees.

(the "sre" exception)
It was the only one. My point was that even just this particular exception
should happen very often. So even volume, not only significance, is
relative. I, for one, was not comfortable ignoring an exception, especially
without a very clear and detailed explanation on why the false positive
happened in the first place and what happens with the exception if I ignore
it.

I don't really disagree - it would probably be good if this was a
well-defined one that Wing came configured to ignore.
(...)

You're making a very good case, assuming that the exception is at the root
of the problem. However, it's hard to say how many such problems are caused
by an exception. Even if there is an exception, the root problem may be way
before that and you still have to go and do some normal debugging. So many
times I've had memory corruption crashing an application in C++ and I had to
look back at a method that was called WAY BEFORE the crash because that was
usually corrupting memory at exit.

Yep, although if you're lucky the form of the corruption at the point
of error can help point the way. Also, since pure memory corruption
tends to be a rare occurrence in Python (buggy extension modules
notwithstanding), even if an exception is due to malformed code
earlier on, the actual memory state of the process is generally in
good enough shape to go poking around reliably in all your data
structures once the exception triggers.
So, in your example of a button, my general debugging approach would still
be to put a breakpoint in the event handler of the button, because the root
of the problem can be so many different things.

Sure, but if the button works sometimes and not others, it's just more
of a pain to hit the breakpoint over and over again until it happens
to get to a state where the problem would occur.
In your experience with Python, what percentage of problems manifested
themselves with exceptions (especially right at the root of the problem) as
opposed to simply flawed logic that gets through without any exceptions?

I'd have to separate by older and newer code again. The newer code
following a TDD pattern definitely has different characteristics than
older legacy code written using more traditional methods. The newer
stuff is much less likely to generate either kind of problem except
during active development at which point the current test generally
points right to the area of potential problem.

But I'd have to say that at least for me, for more traditional
methods, I do think there's a higher percentage of problems leading to
exceptions than to more subtle failures. I'm not entirely sure why
except that I expect bugs that in other environments might lead to
subtle mistakes (or harder to track problems like eventual crashes
from prior memory corruption) quickly end up in some state where you
are trying to interpret any object incorrectly in your code and yield
an exception. For example, a wrong pathway not constructing an
appropriate object type or contents which is then passed into another
function anticipating it. In effect, the "duck typing" approach that
is commonly used can yield exceptions in pretty quick order if you
start messing up the objects involved in your system.

Note however that I'm not saying all those exceptions require a
debugger or anything more than logging them and/or observing them
during development.
I think we are actually very much agreeing. The main difference between us
is how much weight we put in debugging exceptions versus debugging all kinds
of problems and how much we are willing to put up with in exchange for that
precious advantage in a few cases.

Yes (to the agreement). And don't let me over-emphasize things, as
I'm not necessarily suggesting that the "wait for the exception" is
extremely important. It is, however, useful enough to me that I
consider supporting it to be a "pro" feature of any Python debugger,
and at least for me, yes worth some consequences such as the
occasional false positive.
You are saying a solid debugger "can be worth its weight in gold". How much
does Wing IDE weigh? ;-)

Heh - good question. 'Course, with gold where it is at the moment, it
probably wouldn't have to weigh too much anyway :)

-- David
 
D

Dan Perl

David Bolen said:
Yes, once the debugger is actually activated (you start up the IDE and
it attaches) it's like a normal Python debugger, so it uses the trace
hook and will have some impact on runtime. You can finesse this a little
by
controlling where you import the stub and/or when you connect to the
process.

Interesting. So it's actively attached to the process, not just listening
to it. I would like to find out more about how this debug stub works. The
Wing IDE reference manual doesn't get into details like that, but I can see
that one can also set breakpoints remotely. That should mean that the stub
does quite a bit of work.

Since my previous posting on this thread I have discovered an interesting
utility for remote reporting of exceptions, although I couldn't try it:
http://egofile.com/cgi-bin/blosxom.cgi. My understanding is that it sends
an HTTP POST message. That should mean that it's much less powerful than
the Wing IDE stub but it should be also less intrusive.

And I also found win32traceutil
(http://www.python.org/windows/win32/#win32trace), which has quite a similar
use. Of course, it's only for Windows, and I'm not sure whether it works
with the listener and the debugged process on different nodes. But this is
something I may personally use.

Any experience with these tools? Especially with win32traceutil, because
that's something I may use.

Dan
 
D

David Bolen

Dan Perl said:
Interesting. So it's actively attached to the process, not just listening
to it. I would like to find out more about how this debug stub works. The
Wing IDE reference manual doesn't get into details like that, but I can see
that one can also set breakpoints remotely. That should mean that the stub
does quite a bit of work.

It's been a while, but if I recall the stub operates like a network
client, just connecting back to the IDE. It tries to connect when
imported. If it can't connect it sits around (in a background thread
I think) periodically trying - that shouldn't be much overhead. But
when connected, it invokes the normal wing debugging code (e.g., the
same stuff used when debugging an app locally) which I'm assuming ties
into sys.settrace or equivalent and uses the network link to exchange
the information with the IDE. I'm assuming sys.settrace or equivalent
since once connected you can explicitly interrupt the debuggee from
the IDE at any time.
Since my previous posting on this thread I have discovered an interesting
utility for remote reporting of exceptions, although I couldn't try it:
http://egofile.com/cgi-bin/blosxom.cgi. My understanding is that it sends
an HTTP POST message. That should mean that it's much less powerful than
the Wing IDE stub but it should be also less intrusive.

Depends on how it traps the exceptions. If it's using something like
sys.excepthook, then yes, it should definitely be less intrusive (but
also unable to catch anything but unhandled exceptions). But I would
expect any tool that is actively watching code at a finer granularity
is going to carry similar overhead to a debugger since they probably
all use the existing trace hooks in Python.
And I also found win32traceutil
(http://www.python.org/windows/win32/#win32trace), which has quite a similar
use. Of course, it's only for Windows, and I'm not sure whether it works
with the listener and the debugged process on different nodes. But this is
something I may personally use.

I'm pretty sure this is local only. You can get the source from the
PyWin32 package. Last time I looked it used a memory mapped file (in
swap space) to hold data pending someone reading from it. So it's
basically a simple inter-process pipe. Importing it routes all
stdout/stderr to the trace file so yes it would be a way to catch
unhandled exceptions at least for logging purposes.

It's useful (particularly for when you want to get output from
background stuff like COM servers or GUI applications that for one
reason or another can't have a console during debugging), but strictly
for providing an alternate stdout/stderr stream.

-- David
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Staff online

Members online

Forum statistics

Threads
473,755
Messages
2,569,536
Members
45,012
Latest member
RoxanneDzm

Latest Threads

Top