parsing python code

[Ryan Krauss]

I need to parse a Python file by breaking it into blocks matching
indentation levels so that function definitions, for loops, and
classes are kept together as blocks. For example, if I have something
like

from scipy import*
from pylab import*

g = .6
Input_freq = 10.0

def load_data(path):
data = loadtxt(path, skiprows = 1)
t = data[:,0]
Input = data[:,1]
Output = data[:,2]
return t,Input,Output

def time_plot(x,y,n = 1):
figure(n)
clf()
for curx, cury in zip(x,y):
plot(curx,cury)
title('Time Plot')
xlabel('time')
ylabel('f(t)')
legend(['Input','Output'],1)
return figure(n)

t, Input, Output = load_data('system_data.txt')



I would like the blocks to be

block1 = ['from scipy import*', 'from pylab import*', 'g =
..6','Input_freq = 10.0']

block2 = ['def load_data(path):',
' data = loadtxt(path, skiprows = 1)',
' t = data[:,0]',
' Input = data[:,1]',
' Output = data[:,2]',
' return t,Input,Output']

and so on.

I think the parser module should enable me to do this, but I can't
seem to figure it out. Specifically, I think I need to use
parser.sequence2ast, but it doesn't work the way I think it should and
I can't find more documentation on it or an example that uses it.

I tried

f = open('Example.py','r')
mylines = f.readlines()
parser.sequence2ast(mylines)

but got a ParserError.

Is there an easy way to do what I need?

Thanks,

Ryan

 

[Kay Schluehr]

I need to parse a Python file by breaking it into blocks matching
indentation levels so that function definitions, for loops, and
classes are kept together as blocks. For example, if I have something
like

from scipy import*
from pylab import*

g = .6
Input_freq = 10.0

def load_data(path):
data = loadtxt(path, skiprows = 1)
t = data[:,0]
Input = data[:,1]
Output = data[:,2]
return t,Input,Output

def time_plot(x,y,n = 1):
figure(n)
clf()
for curx, cury in zip(x,y):
plot(curx,cury)
title('Time Plot')
xlabel('time')
ylabel('f(t)')
legend(['Input','Output'],1)
return figure(n)

t, Input, Output = load_data('system_data.txt')

I would like the blocks to be

block1 = ['from scipy import*', 'from pylab import*', 'g =
.6','Input_freq = 10.0']

block2 = ['def load_data(path):',
' data = loadtxt(path, skiprows = 1)',
' t = data[:,0]',
' Input = data[:,1]',
' Output = data[:,2]',
' return t,Input,Output']

and so on.

I think the parser module should enable me to do this, but I can't
seem to figure it out. Specifically, I think I need to use
parser.sequence2ast, but it doesn't work the way I think it should and
I can't find more documentation on it or an example that uses it.

I tried

f = open('Example.py','r')
mylines = f.readlines()
parser.sequence2ast(mylines)

but got a ParserError.

Is there an easy way to do what I need?

Thanks,

Ryan

You have to use parser.suite:

f_content = open('Example.py').read()
_ast = parser.suite(f_content)

You can display the _ast object as a nested list:

_lst = _ast.tolist()

This list can be converted back to an ast object using sequence2ast.

Another option is to use parser.expr instead of parser.suite but this
only applies for Python expressions but not statements. So
parser.expr / parser.suite are like eval / exec.