AutoComplete in C++ Editor for Python

[flamz3d]

Hello,
I am embedding python support in my C++ application and was looking at
adding "Intellisense" or "AutoComplete" support.

I found a way to do it using the "dir" function, but this creates a
problem. Here's why. Let's say I have the following code in my editor:

import sys
x = sys


Now, I would like to get all attributes of the object called 'x'. I
can "instrument" the code and add "print dir(x)" at the end,
temporarily redirect the python output to a string and execute the
code.

But this is not safe: I do NOT want to execute the code while the user
is typing!

Is there a way to "compile" the python code and get access to the
symbol table from that compiled block?

Did anybody ever implement AutoComplete in a editor for Python?

cheers.

 

[Dave Angel]

Hello,
I am embedding python support in my C++ application and was looking at
adding "Intellisense" or "AutoComplete" support.

I found a way to do it using the "dir" function, but this creates a
problem. Here's why. Let's say I have the following code in my editor:

import sys
x = sys


Now, I would like to get all attributes of the object called 'x'. I
can "instrument" the code and add "print dir(x)" at the end,
temporarily redirect the python output to a string and execute the
code.

But this is not safe: I do NOT want to execute the code while the user
is typing!

Is there a way to "compile" the python code and get access to the
symbol table from that compiled block?

Did anybody ever implement AutoComplete in a editor for Python?

cheers.

Several editors for Python support auto-complete, to one extent or
another. The only one I have experience with is Komodo. Komodo runs in
a separate process, so it doesn't suffer from the problems of having two
gui event-loops in the same process, and other similar problems. It
also won't be executing code that might have side effects in the child
process.

The downside is that in order to do auto-complete, it has to figure it
out from other clues. From the docs, and from reading, and from
experiementing, I believe that it uses two approaches. One approach is
a set of language files which try to describe all the symbols in the
standard language and library. They have one for each release (2.4,
2.5, ...) Theoretically, you could add your own for other libraries.
Second approach is to parse the code that's visible to it. That parsing
is well done, and surprisingly quick, but there are lots of tricks a
developer might use that can fool it. For example, wxPython lets you
import just one symbol, and lots more appear magically. It's no big
deal, they have code structured one way, but the interface is very
different. Catch is that code completion frequently gets fooled by these.

I'd point out that if you do walk the dictionaries at run time, you'll
get different results when you do it with nothing running than if you do
a strictly static analysis. So some things might only show up if you've
stepped into the function with the magic going on.

Simplest example I can come up with of something a static analysis won't
spot: An instance object obj may have some number of attributes
assigned the the __init__() method. But it could also have new fields
added by anyone with a referent into it.

There is a Python parser in module ast. Perhaps that'd be some help.
I've not used it, so can't give you any specifics.


DaveA

 

[flamz3d]

Several editors for Python support auto-complete, to one extent or
another.  The only one I have experience with is Komodo.  Komodo runs in
a separate process, so it doesn't suffer from the problems of having two
gui event-loops in the same process, and other similar problems.   It
also won't be executing code that might have side effects in the child
process.

The downside is that in order to do auto-complete, it has to figure it
out from other clues.  From the docs, and from reading, and from
experiementing, I believe that it uses two approaches.  One approach is
a set of language files which try to describe all the symbols in the
standard language and library.  They have one for each release (2.4,
2.5, ...)  Theoretically, you could add your own for other libraries.  
Second approach is to parse the code that's visible to it.  That parsing
is well done, and surprisingly quick, but there are lots of tricks a
developer might use that can fool it.  For example, wxPython lets you
import just one symbol, and lots more appear magically.  It's no big
deal, they have code structured one way, but the interface is very
different.  Catch is that code completion frequently gets fooled by these.

I'd point out that if you do walk the dictionaries at run time, you'll
get different results when you do it with nothing running than if you do
a strictly static analysis.  So some things might only show up if you've
stepped into the function with the magic going on.

Simplest example I can come up with of something a static analysis won't
spot:   An instance object    obj   may have some number of attributes
assigned the the __init__() method.  But it could also have new fields
added by anyone with a referent into it.

There is a Python parser in module ast.  Perhaps that'd be some help.  
I've not used it, so can't give you any specifics.

DaveA

I think it might be possible for me to figure out how to do
AutoCompletion using the symtable object. But I am having difficulty
making sense of it.

In my tests, I was use the following Python code:

import sys
x=sys

The idea is to get the type of the variable x

To attempt this, I use the following C code (my editor is written in
C)

symtable* st = Py_SymtableString ( "import sys
\nx=sys,"<string>",Py_single_input);

PyObject *name, *value;
Py_ssize_t pos = 0;
while (PyDict_Next(st->st_symbols, &pos, &name, &value))
{
....
}



Using this code, I can get information like the name of the symbol
(x), but I can't figure out how to get the type. If I knew how to get
this it would solve 99% of my problems

Any ideas?

 

[Dave Angel]

Several editors for Python support auto-complete, to one extent or
another. The only one I have experience with is Komodo. Komodo runs in
a separate process, so it doesn't suffer from the problems of having two
gui event-loops in the same process, and other similar problems. It
also won't be executing code that might have side effects in the child
process.

The downside is that in order to do auto-complete, it has to figure it
out from other clues. From the docs, and from reading, and from
experiementing, I believe that it uses two approaches. One approach is
a set of language files which try to describe all the symbols in the
standard language and library. They have one for each release (2.4,
2.5, ...) Theoretically, you could add your own for other libraries.
Second approach is to parse the code that's visible to it. That parsing
is well done, and surprisingly quick, but there are lots of tricks a
developer might use that can fool it. For example, wxPython lets you
import just one symbol, and lots more appear magically. It's no big
deal, they have code structured one way, but the interface is very
different. Catch is that code completion frequently gets fooled by these.

I'd point out that if you do walk the dictionaries at run time, you'll
get different results when you do it with nothing running than if you do
a strictly static analysis. So some things might only show up if you've
stepped into the function with the magic going on.

Simplest example I can come up with of something a static analysis won't
spot: An instance object obj may have some number of attributes
assigned the the __init__() method. But it could also have new fields
added by anyone with a referent into it.

There is a Python parser in module ast. Perhaps that'd be some help.
I've not used it, so can't give you any specifics.

DaveA

I think it might be possible for me to figure out how to do
AutoCompletion using the symtable object. But I am having difficulty
making sense of it.

In my tests, I was use the following Python code:

import sys
x=s

The idea is to get the type of the variable x

To attempt this, I use the following C code (my editor is written in
C)

symtable* st =y_SymtableString ( "import sys
\nx=s,"<string>",Py_single_input);

PyObject *name, *value;
Py_ssize_t pos =;
while (PyDict_Next(st->st_symbols, &pos, &name, &value))
{
...
}



Using this code, I can get information like the name of the symbol
(x), but I can't figure out how to get the type. If I knew how to get
this it would solve 99% of my problems

Any ideas?

There's a type() function as a builtin. It returns a string showing the
type of any object you pass it.

I have my doubts about the 99%; I'd say more like 1%. Objects have to
be instantiated to have a type, there's no declarations. And a function
has a type, but in order to see the result type through introspection,
you'd have to run it. And before you could do that, you'd have to guess
its parameter types. And hope it didn't have any serious side effects.

I can't help with the C++ code, since I've never written any that use
the Python runtime, but perhaps you can experiment with a Python
version, till you see what is and is-not possible. Look at the
following in Python:
import sys

dict = sys.__dict__
for sym in dict:
print "**", sym, type(dict[sym]), str(dict[sym])


Now, for many of these sym values, sym.__dict__ could similarly be
analyzed. So you might write a recursive generator that gave you a list
of symbols.

 

[justme]

I the rope project http://rope.sourceforge.net/ has an autocomplete
lib.
(I have not used it just remember reading about it)