parsing string to a list of objects - help!

[Donn Ingle]

Hi, I really hope someone can help me -- I'm stuck.

I have written three versions of code over a week and still can't get past
this problem, it's blocking my path to getting other code written.

This might be a little hairy, but I'll try to keep it short.

Situation:
I want to pass a string to a function which will parse it and generate
objects in a list.
Each object is a "property" which keeps members like x,y,r,g,b etc.
Each element in the final list describes a frame in an animation.

I want to be able to pass only the properties of key moments (key frames) in
time to the function. Along with that is passed the *way* each moment
changes into the next.

The form I am using is like this:
t=Thing()
t.propLayout("#---#---#==_##_")
prop1 = Property(x=10,y=10,r=1,g=1,b=1,a=0)
prop2 = Property(x=20,y=10,r=0,g=0,b=1,a=1)
prop3 = Property(x=10,y=100,r=1,g=1,b=1,a=1)
prop4 = Property(x=100,y=200,r=1,g=1,b=1,a=1)
t.setProps(prop1,prop1,prop2,prop3,prop4)

So setProps is the parser and it creates a proplist within t.

My short-hand for this:
# is a property
- is a 'tween' (where the property before it influences it)
= is a property exactly the same as the one before it.
_ (underscore) is a blank frame

Things like this can be expressed:
1234567890123
#---#--#===_#
(4 keyframes, #'s. 13 frames, lifetime.)
In words that would say:
Frame 1, make a property.
Frame 2 to 4 tweening : copy previous property (from 1) and advance the
members by the average (calculated by getting the range between 1 and 5 and
going through each member variable (x,y,z,r,g,b) and dividing etc. This
means looking ahead.)
Frame 5, make a property
Frame 6 to 7 tweening.
Frame 8, make a property
Frame 9 to 11, copy property from 8
Frame 12, make a blank
Frame 13, make a property.

So, the animation will proceed: Start at 1, move/morph to 5, move/morph to
8, remain static to 11, go away for 12 and appear somewhere at 13.

It seems so simple, but I just cannot nail it down. Each time I end up with
a mess of string[index] and if else tests within while loops that seem to
work until I get to a test string that breaks it all.

I'm sure there must be a better way to formalize this little 'language' and
break it down into a list of property objects that exactly describes the
string -- catching user errors too.

The rules:
Tweens : must start and end in # signs: #----#
Static : must start with a # : #===
No other chars can be within a range: #==_= is bad. #--=--# is bad.

Odds:
Singles: things like ### are valid. This means each is a frame somewhere,
implying they will jump around (i.e. not be statics)
Blanks : are simple - they are properties for every underscore

I have glanced around at parsing and all the tech-speak confuses the heck
out of me. I am not too smart and would appreciate any help that steers
away from cold theory and simply hits at this problem.

Donn.

 

[Paul McGuire]

I have glanced around at parsing and all the tech-speak confuses the heck
out of me. I am not too smart and would appreciate any help that steers
away from cold theory and simply hits at this problem.

Donn -

Here is a pyparsing version that I hope is fairly easy-to-read and
tech-speak free:

from pyparsing import *

frame = Literal("#")
tween = Word("-") # that is, it is a "word" composed of 1 or more -'s
copy = Literal("=")
blank = Literal("_")

animation = OneOrMore((frame + Optional(
(tween + FollowedBy(frame)) |
OneOrMore(copy | blank) ) ) )

test = "#---#--#===_#"

print animation.parseString(test)

This prints the following:

['#', '---', '#', '--', '#', '=', '=', '=', '_', '#']

From here, the next step would be to define classes that these matched

tokens could be converted to. Pyparsing allows you to attach a method
to individual expressions using setParseAction - if you pass a class
instead of a method, then class instances are returned. For these
tokens, you could write:

class Prop(object):
def __init__(self,tokens):
pass

class Tween(object):
def __init__(self,tokens):
self.tween = tokens[0]
self.tweenLength = len(self.tween)

class CopyPrevious(object):
def __init__(self,tokens):
pass

class Blank(object):
def __init__(self,tokens):
pass

frame.setParseAction(Prop)
tween.setParseAction(Tween)
copy.setParseAction(CopyPrevious)
blank.setParseAction(Blank)

And now calling animation.parseString(test) returns a sequence of
objects representing the input string's frames, tweens, etc.:

[<__main__.Prop object at 0x00B9D8F0>, <__main__.Tween object at
0x00BA5390>, <__main__.Prop object at 0x00B9D9B0>, <__main__.Tween
object at 0x00BA55F0>, <__main__.Prop object at 0x00BA5230>,
<__main__.CopyPrevious object at 0x00BA58F0>, <__main__.CopyPrevious
object at 0x00BA59D0>, <__main__.CopyPrevious object at 0x00BA5AB0>,
<__main__.Blank object at 0x00BA5B70>, <__main__.Prop object at
0x00BA5510>]

Now you can implement behavior in these 4 classes to implement the
tweening, copying, etc. behavior that you want, and then walk through
this sequence invoking this animation logic.

-- Paul

 

[Donn Ingle]

Wow Paul, you have given me much to chew. I'll start testing it in my slow
way -- it looks so simple!

Thank you.
\d

 

[Donn Ingle]

Wow Paul, you have given me much to chew. I'll start testing it in my slow
way -- it looks so simple!

Thank you.
\d

 

[Donn Ingle]

Paul,
I quickly slapped your example into a py file for a first-glance test and
the first part works, but the second gives me the error below. Could I have
an old version of pyparser? I will dig into it anyway, just being lazy

code:
from pyparsing import *

frame = Literal("#")
tween = Word("-") # that is, it is a "word" composed of 1 or more -'s
copy = Literal("=")
blank = Literal("_")

animation = OneOrMore((frame + Optional(
(tween + FollowedBy(frame)) |
OneOrMore(copy | blank) ) ) )

test = "#---#--#===_#"

print animation.parseString(test)

class Prop(object):
def __init__(self,tokens):
pass

class Tween(object):
def __init__(self,tokens):
self.tween = tokens[0]
self.tweenLength = len(self.tween)

class CopyPrevious(object):
def __init__(self,tokens):
pass

class Blank(object):
def __init__(self,tokens):
pass

frame.setParseAction(Prop)
tween.setParseAction(Tween)
copy.setParseAction(CopyPrevious)
blank.setParseAction(Blank)


animation.parseString(test)

result:
containers:$ python parser.py
['#', '---', '#', '--', '#', '=', '=', '=', '_', '#']
Traceback (most recent call last):
File "parser.py", line 39, in ?
animation.parseString(test)
File "/usr/lib/python2.4/site-packages/pyparsing.py", line 620, in
parseString
loc, tokens = self.parse( instring.expandtabs(), 0 )
File "/usr/lib/python2.4/site-packages/pyparsing.py", line 562, in parse
loc,tokens = self.parseImpl( instring, loc, doActions )
File "/usr/lib/python2.4/site-packages/pyparsing.py", line 1768, in
parseImpl
loc, tokens = self.expr.parse( instring, loc, doActions )
File "/usr/lib/python2.4/site-packages/pyparsing.py", line 562, in parse
loc,tokens = self.parseImpl( instring, loc, doActions )
File "/usr/lib/python2.4/site-packages/pyparsing.py", line 1390, in
parseImpl
loc, resultlist = self.exprs[0].parse( instring, loc, doActions )
File "/usr/lib/python2.4/site-packages/pyparsing.py", line 588, in parse
tokens = self.parseAction( instring, tokensStart, retTokens )
TypeError: __init__() takes exactly 2 arguments (4 given)

 

[Donn Ingle]

Paul,

frame = Literal("#")
tween = Word("-") # that is, it is a "word" composed of 1 or more -'s
copy = Literal("=")
blank = Literal("_")

animation = OneOrMore((frame + Optional(
(tween + FollowedBy(frame)) |
OneOrMore(copy | blank) ) ) )

I found that this form insists on having a # in char 0. How could it be
changed to take a bunch of blanks before the first frame?
____# ?

I have tried:
animation = OneOrMore(
Optional ( blank | frame ) +
Optional( (tween + FollowedBy(frame) ) | OneOrMore(copy | blank) )
)

And permutations thereof, but it goes into a black hole.

\d

 

[Paul McGuire]

Donn -

The exception you posted is from using an old version of pyparsing.
You can get the latest from SourceForge - if you download the Windows
binary install, please get the docs package too. It has a full doc
directory (generated with epydoc), plus example scripts for a variety
of applications. There is also an e-book available from O'Reilly that
just came out in early October.

To add an optional lead-in of one or more blanks,just change animation
from:

animation = OneOrMore((frame + Optional(
(tween + FollowedBy(frame)) |
OneOrMore(copy | blank) ) ) )

to:

animation = Optional(OneOrMore(blank)) + \
OneOrMore((frame + Optional(
(tween + FollowedBy(frame)) |
OneOrMore(copy | blank) ) ) )

I modified your test, and there were no problems parsing strings with
and without the leading '_'s.

-- Paul

 

[Paul McGuire]

Here is a first cut at processing the parsed objects, given a list of
Property objects representing the frames at each '#':

class Property(object):
def __init__(self,**kwargs):
self.__dict__.update(kwargs)

def copy(self):
return Property(**self.__dict__)

@staticmethod
def tween(prev,next,n,i):
dt = 1.0/(n+1)
x = prev.x + (next.x-prev.x)*dt*(i+1)
y = prev.y + (next.y-prev.y)*dt*(i+1)
r = prev.r + (next.r-prev.r)*dt*(i+1)
g = prev.g + (next.g-prev.g)*dt*(i+1)
b = prev.b + (next.b-prev.b)*dt*(i+1)
a = prev.a + (next.a-prev.a)*dt*(i+1)
return Property(x=x,y=y,r=r,g=g,b=b,a=a)

@staticmethod
def makeBlank():
return Property(x=0,y=0,r=0,g=0,b=0,a=0)

def __repr__(self):
return "Property(x=%(x).3f, y=%(y).3f, r=%(r).3f, g=%(g).3f, b=
%(b).3f, a=%(a).3f)" % self.__dict__


def makeAllFrames(propFrameList, animationList):
ret = []
propFrameIndex = 0
for animIndex,animObj in enumerate(animationList):
if isinstance(animObj, Prop):
ret.append( propFrameList[propFrameIndex] )
propFrameIndex += 1
if isinstance(animObj, Blank):
ret.append( Property.makeBlank() )
if isinstance(animObj, CopyPrevious):
ret.append( ret[-1].copy() )
if isinstance(animObj, Tween):
# compute delta x,y,r,g,b,a between prev frame and next
frame
prev = propFrameList[propFrameIndex-1]
next = propFrameList[propFrameIndex]
numFrames = animObj.tweenLength
print `prev`, `next`
tweens = [ Property.tween(prev,next,numFrames,i) for i in
range(numFrames) ]
ret += tweens
return ret

prop1 = Property(x=10,y=10,r=1,g=1,b=1,a=0)
prop2 = Property(x=20,y=10,r=0,g=0,b=1,a=1)
prop3 = Property(x=10,y=100,r=1,g=1,b=1,a=1)
prop4 = Property(x=100,y=200,r=1,g=1,b=1,a=1)
propFrames = [ prop1, prop2, prop3, prop4 ]

allFramesList = makeAllFrames( propFrames, animation.parseString("#---
#--#===_#") )
from pprint import pprint
pprint( allFramesList )

Gives:
[Property(x=10.000, y=10.000, r=1.000, g=1.000, b=1.000, a=0.000),
Property(x=12.500, y=10.000, r=0.750, g=0.750, b=1.000, a=0.250),
Property(x=15.000, y=10.000, r=0.500, g=0.500, b=1.000, a=0.500),
Property(x=17.500, y=10.000, r=0.250, g=0.250, b=1.000, a=0.750),
Property(x=20.000, y=10.000, r=0.000, g=0.000, b=1.000, a=1.000),
Property(x=16.667, y=40.000, r=0.333, g=0.333, b=1.000, a=1.000),
Property(x=13.333, y=70.000, r=0.667, g=0.667, b=1.000, a=1.000),
Property(x=10.000, y=100.000, r=1.000, g=1.000, b=1.000, a=1.000),
Property(x=10.000, y=100.000, r=1.000, g=1.000, b=1.000, a=1.000),
Property(x=10.000, y=100.000, r=1.000, g=1.000, b=1.000, a=1.000),
Property(x=10.000, y=100.000, r=1.000, g=1.000, b=1.000, a=1.000),
Property(x=0.000, y=0.000, r=0.000, g=0.000, b=0.000, a=0.000),
Property(x=100.000, y=200.000, r=1.000, g=1.000, b=1.000, a=1.000)]


-- Paul