Dreaming of new generation IDE

[Vladimir Ignatov]

Hello,

I am sitting here for quite some time, but usually keep silent ;-) I
use Python since 2003 both "professionally" and for my hobby projects
and love it a much.
I notice however, that "maintaining" existing/older python code is may
be not so enjoyable task. It may be even harder than supporting old
code written in some type of "static" languages (like Java or C++).
Surely "dynamic" nature of python comes with price.

Finally I develop a feeling that strong instrumentation / tools can
bring us the best of two worlds. That I am dreaming on is an absolute
new type/class of IDE suitable for Python and potentially for other
dynamic-type languages. Instead of current text-oriented IDEs, it
should be a database-centric and resemble current CAD systems instead
of being just "fancy text editor". Source text should be an output
product of that CAD and not a "source material" itself.

Well. I understand that it is a very ambitious and experimental stuff.
Great efforts and motivation needed even to get something "runnable".
So I am looking for someone to get in kind of "virtual partnership".
If someone interesting it that soft of stuff, I would like to talk and
discuss this system.

Vladimir Ignatov

 

[Paul Rubin]

I notice however, that "maintaining" existing/older python code is may
be not so enjoyable task. It may be even harder than supporting old
code written in some type of "static" languages (like Java or C++).
Surely "dynamic" nature of python comes with price.

Yes, this is a well known drawback of dynamic typing. The usual remedy
is lots and lots of test automation (a full suite of unit and
integration tests that you run on every build, that check the properties
of basically every function in your program).

Instead of current text-oriented IDEs, it
should be a database-centric and resemble current CAD systems

I've never used a current CAD system, so I can't make any sense of this.
I don't see how databases would help.

 

[Stefan Behnel]

Paul Rubin, 03.02.2010 14:07:

I've never used a current CAD system, so I can't make any sense of this.
I don't see how databases would help.

Just like they help in current IDEs to index the source code.

Stefan

 

[Marco Salden]

Hello,

I am sitting here for quite some time, but usually keep silent ;-) I
use Python since 2003 both "professionally" and for my hobby projects
and love it a much.
I notice however, that "maintaining" existing/older python code is may
be not so enjoyable task. It may be even harder than supporting old
code written in some type of "static" languages (like Java or C++).
Surely "dynamic" nature of python comes with price.

Finally I develop a feeling that strong instrumentation / tools can
bring us the best of two worlds. That I am dreaming on is an absolute
new type/class of IDE suitable for Python and potentially for other
dynamic-type languages. Instead of current text-oriented IDEs, it
should be a database-centric and resemble current CAD systems instead
of being just "fancy text editor". Source text should be an output
product of that CAD and not a "source material" itself.

Well. I understand that it is a very ambitious and experimental stuff.
Great efforts and motivation needed even to get something "runnable".
So I am looking for someone to get in kind of "virtual partnership".
If someone interesting it that soft of stuff, I would like to talk and
discuss this system.

Vladimir Ignatov

The maintenance thing may be true but for me that doesn't outweigh the
clear benefits I get from using Python i.s.o. e.g. C++: the fact that
I have much less code that is more compact and for me more directly
readable is a clear advantage when doing maintance on code I didnt
touch for a while. Documenting the essential parameters used with some
examples and some logging helps (for me at least...).

The type of IDE you are looking for is more something like Rational
Rose (e.g. RRT?) type of tooling perhaps? There you "CAD" components
and their statebehavior and the system generates C or C++ code.

Regards,
Marco

 

[Vladimir Ignatov]

The maintenance thing may be true but for me that doesn't outweigh the

clear benefits I get from using Python i.s.o. e.g. C++: the fact that
I have much less code that is more compact and for me more directly
readable is a clear advantage when doing maintance on code I didnt
touch for a while. Documenting the essential parameters used with some
examples and some logging helps (for me at least...).

Python is very easy to write. But with code grow and time passes,
things get worse. Currently I tries to reanimate one of my old
freeware project and faced it again.

The type of IDE you are looking for is more something like Rational
Rose (e.g. RRT?) type of tooling perhaps? There you "CAD" components
and their statebehavior and the system generates C or C++ code.

Perhaps. I newer use Rational Rose but hear about it. But I use some
other Rational* products and they scare me a lot. So maybe Rose can be
used for "inspiration" but not much more. Actually I don't want to
put programmer in GUI-fashioned env. with a lot of buttons and
switches. I think about "classic" text-like view but that actually
"understands" that are you doing (typing) on. So your types goes in
"datastore" first and then renders back as a text representation.
Something like this.

Vladimir Ignatov

 

[Phlip]

Finally I develop a feeling that strong instrumentation / tools can
bring us the best of two worlds. That I am dreaming on is an absolute
new type/class of IDE suitable for Python and potentially for other
dynamic-type languages. Instead of current text-oriented IDEs, it
should be a database-centric and resemble current CAD systems instead
of being just "fancy text editor". Source text should be an output
product of that CAD and not a "source material" itself.

That's fine so long as I can also treat the source as source, at need.

You may have just reinvented Smalltalk's Squeak editor (reputedly the
testbed for all of modern GUIs...).

Current editors suck because they can't see into the code and browse
it - unless it's so statically typed it's painful.

That's why I wrote this:

http://www.oreillynet.com/onlamp/blog/2008/05/dynamic_languages_vs_editors.html

 

[John Bokma]

Finally I develop a feeling that strong instrumentation / tools can
bring us the best of two worlds. That I am dreaming on is an absolute
new type/class of IDE suitable for Python and potentially for other
dynamic-type languages. Instead of current text-oriented IDEs, it
should be a database-centric and resemble current CAD systems instead
of being just "fancy text editor". Source text should be an output
product of that CAD and not a "source material" itself.

Your idea is certainly not new. Moreover, I've been using such an IDE
and implemented support for a (small) language in it:

Programs are hierarchical compositions of formulae satisfying
structural and extra-structural relationships. A program editor can
use knowledge of such relationships to detect and provide immediate
feedback about violations of them. The Synthesizer Generator is a
tool for creating such editors from language descriptions. An editor
designer specifies the desired relationships and the feedback to be
given when they are violated, as well as a user interface; from the
specification, the Synthesizer Generator creates a full-screen editor
for manipulating programs in the language.

http://portal.acm.org/citation.cfm?id=390010.808247

It might be a good start to read as much as possible on the Synthesizer
Generator if you want to write such an IDE for Python. I am sure you
could write such an IDE in the Synthesizer Generator, but I have no idea
if it's still available. And back when I used it (at university) one had
to pay for it, and most likely was closed source as well.

See also: http://www.google.com/search?q=synthesizer generator

 

[Adam Tauno Williams]

That's fine so long as I can also treat the source as source, at need.
You may have just reinvented Smalltalk's Squeak editor (reputedly the
testbed for all of modern GUIs...).
Current editors suck because they can't see into the code and browse
it - unless it's so statically typed it's painful.

? I edit Python in MonoDevelop 2.2; and I can browse my file,
classes, etc... So I don't know what you mean by "can't see into the
code". It works pretty well.

Of course it can't tell that I've set x = {an integer}, as that only
happens at runtime.

 

[Phlip]

?  I edit Python in MonoDevelop  2.2;  and I can browse my file,
classes, etc...  So I don't know what you mean by "can't see into the
code".  It works pretty well.

Of course it can't tell that I've set x = {an integer}, as that only
happens at runtime.

You just said that your code browsing "works pretty well, except when
it doesn't".

Hence my blog entry. If your editor analyzed your code at runtime,
instead of just static analysis, then it could see that x = an
integer, or an object, no matter how dynamic your language.

 

[John Bokma]

You just said that your code browsing "works pretty well, except when
it doesn't".

Hence my blog entry. If your editor analyzed your code at runtime,
instead of just static analysis, then it could see that x = an
integer, or an object, no matter how dynamic your language.

In Perl:

my $x = ( 5, "hello", sub {}, [], {} )[ int rand 5 ];

what's $x? The answer is: it depends.

Moreover, even if your editor analyzes your code at runtime (which is
certainly not always desirable) it might not be able to find out what
the type is of x, simply because it would take too much time to find it
out. (It sounds like you want an editor that solves the halting problem
;-) )

I agree with you that to /some extent/ and editor can do analyses, if it
does compilation as well (and even runs the code, but the latter is not
always desirable). I mentioned the Synthesizer Generator before, which
can do compilation on the fly, if you implement it (or if it has been
implemented for the language you edit with it). I've written a very
simple assembler in it, ages ago, which did assembling on the fly.

 

[Robert]

dynamic-type languages. Instead of current text-oriented IDEs, it
should be a database-centric and resemble current CAD systems instead
of being just "fancy text editor". Source text should be an output
product of that CAD and not a "source material" itself.

can you sketch an example/use case more concretely?


Robert

 

[Phlip]

my $x = ( 5, "hello", sub {}, [], {} )[ int rand 5 ];

what's $x? The answer is: it depends.

That's why my blog post advocated (as usual for me) developer tests.
Then you either mock the rand, like all developers should, or you get
what you pay for, and Principle of Least Surprise still applies...

Over the past decade, teams discovered that developer tests more than
made up for the lack of rigor in dynamic languages. A dynamic language
with tests can be more productive than a static language, even with
its runtime type checks AND with its tests.

However, our editors must catch up to us. When I test, I am statically
declaring a set of types, even if the language would prefer to
dynamically fling them hither and yon. We should leverage that.

 

[John Bokma]

my $x = ( 5, "hello", sub {}, [], {} )[ int rand 5 ];

what's $x? The answer is: it depends.

That's why my blog post advocated (as usual for me) developer tests.
Then you either mock the rand, like all developers should, or you get
what you pay for, and Principle of Least Surprise still applies...

Yup, I agree with you that (to some extent) an IDE should be able to
determine types, especially if programmers don't reuse variables, like
(again Perl):

my $result = .... # int
:
:
if ( ... ) {

$result = .... # string
}

# $result can be still an int, or either a string, depending on the
# test.

Over the past decade, teams discovered that developer tests more than
made up for the lack of rigor in dynamic languages. A dynamic language
with tests can be more productive than a static language, even with
its runtime type checks AND with its tests.

Yup, this matches up with my experience. I can't recall that I ever
bumped into an issue in Perl (the dynamic language I've been using
the most for the past years). Not saying that it hasn't happened, but I
just can't recall. Probably also the reason why a "new" language I am
learning is also dynamic: Python ;-)

However, our editors must catch up to us. When I test, I am statically
declaring a set of types, even if the language would prefer to
dynamically fling them hither and yon. We should leverage that.

I am all for testing, but it should IMO not get into the way. I am quite
happy with Emacs as an editor (I "recently" switched), it satisfies most
(if not all) of the items on the check list.

 

[John Bokma]

can you sketch an example/use case more concretely?

I guess Vladimir means what's called a structure editor. The (by me)
aforementioned Synthesizer Generator is an example of such an editor
(environment).

http://en.wikipedia.org/wiki/Structure_editor
http://portal.acm.org/citation.cfm?doid=358746.358755

 

[Vladimir Ignatov]

can you sketch an example/use case more concretely?

Sorry, I don't have anything written down. I just have some rough idea
of implementation and some concrete features I would like to see in
such system. For example:

1) Instant refactoring. No more needs for manual
search/inspect/rename. Since system knows exactly that is going on,
the refactoring will be fully automatic.
2) Show "xref table" for each function. How often this function used?
Where is it used? (code snippets of calls) What functionality is
supported by this function?
3) Extended statistics. How many objects this object/function
interacts with? Popular functions, dead/unused functions.
4) Code smell detector - too long functions, too much interaction with
other objects, global objects, etc.
....

Vladimir Ignatov

 

[Robert Kern]

can you sketch an example/use case more concretely?

I believe that Smalltalk came pretty close to what Vladimir is asking for. You
wrote the methods as linear plain text, but you used the GUI three pane class
browser to define and navigate classes. You could export class definitions to a
text file and read them back in, but in Vladimir's terms, the text files were
not the "source material" themselves.

http://www.cosc.canterbury.ac.nz/wolfgang.kreutzer/cosc205/smalltalk1.html

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Vladimir Ignatov]

I guess Vladimir means what's called a structure editor. The (by me)

aforementioned Synthesizer Generator is an example of such an editor
(environment).

Maybe. Yes, it kind of "generator". It has (entered somehow) internal
representation of target program. Then it generates code out of this
internal data. Yes, it is imaginable system, I don't have any working
prototype yet. But about to start making it. For prototype I choose
python 2.x as implementation language, sqlite as internal database and
Django as UI.

Vladimir Ignatov

 

[John Bokma]

Maybe. Yes, it kind of "generator". It has (entered somehow) internal
representation of target program. Then it generates code out of this
internal data. Yes, it is imaginable system, I don't have any working
prototype yet. But about to start making it. For prototype I choose
python 2.x as implementation language, sqlite as internal database and
Django as UI.

You might check out articles that mention the Synthesizer Generator, etc.
http://citeseer.ist.psu.edu/cis?q=synthesizer+generator&cs=1
http://citeseer.ist.psu.edu/cis?q=structure+editor&cs=1

The idea is not new (at least 30 years old), and those articles might
help you.

 

[Steve Holden]

Sorry, I don't have anything written down. I just have some rough idea
of implementation and some concrete features I would like to see in
such system. For example:

1) Instant refactoring. No more needs for manual
search/inspect/rename. Since system knows exactly that is going on,
the refactoring will be fully automatic.

You'll probably want to take a look at "Bicycle Repair Man", which
offers Python refactoring functionality.

2) Show "xref table" for each function. How often this function used?
Where is it used? (code snippets of calls) What functionality is
supported by this function?

Bear in mind this information can only ever be partial, since functions
are first-class objects and aren't always referred to by their original
name.

3) Extended statistics. How many objects this object/function
interacts with? Popular functions, dead/unused functions.
4) Code smell detector - too long functions, too much interaction with
other objects, global objects, etc.
...

Also take a look at pylint and pychecker, which are already built to
perform that kind of check on Python code.

regards
Steve

 

[Fuzzyman]

You just said that your code browsing "works pretty well, except when
it doesn't".

Hence my blog entry. If your editor analyzed your code at runtime,
instead of just static analysis, then it could see that x = an
integer, or an object, no matter how dynamic your language.

The Wingware Wing IDE has an integrated debugger. The IDE does type
inferencing through static analysis to provide autocomplete and code
navigation features. If you run the code under the debugger it will
also use 'actual' type information to augment the static analysis.

Michael Foord