namespace/dictionary quandry

Discussion in 'Python' started by Jack Carter, Sep 18, 2004.

  1. Jack Carter

    Jack Carter Guest

    I have been delegated to produce a tool that has
    python embedded in it. The desire is to have a
    command line interface that inherits all the python
    scripting functionality so people can use the tool
    either with a flat commandline entry like you would
    a debugger or with a script using scoping, loops and
    conditional.

    I am going from knowing nothing about python to
    almost nothing so the learning curve is rather
    steep at this point.

    I have a dummy program that inherits code.InteractiveConsole
    functionality. I can grab each input line and scan it for
    commands that match my tool's syntax ("attach") and if it
    does, I change the line to reflect a python legal function
    call associated with that commond (DoAttach) and from there
    do what I want with the line. The actual input lines are
    executed with a self.push() call.

    The problem I have is in timely symbol resolution. I really
    want to run eval() on each of the arguments at the time
    the DoAttach() function is executed, but don't know how
    and where from does one get the dictionary with the proper
    name space. If I run eval at the time of initial reading
    of the input it works for flat first level input, but not
    for nested scoping:

    cli.py
    bosco=5
    if 1:
    attach bosco
    bosco=7
    attach bosco

    attach bosco

    This will result in:

    >>> bosco=5
    >>> if 1:

    ... attach bosco
    ... bosco=7
    ... attach bosco
    ...
    DoAttach: ['5']
    DoAttach: ['5'] <--- WRONG (at least, not what I want)
    >>> attach bosco

    DoAttach: ['7']

    How does one export the correct dictionary?

    Attached is the toy program that creates this output.

    Keep in mind that I am probably aproaching this all
    wrong. Any relevant suggestions would be most appreciated,

    Jack

    *****************
    File cli.py:
    *****************
    #!/usr/bin/env python

    import myparse

    cli = myparse.CLI(globals())
    cli.interact()


    *****************
    File myparse.py:
    *****************
    import code
    import re
    import string
    import sys

    def DoAttach(args):


    print "DoAttach:", args
    pass


    class CLI(code.InteractiveConsole):
    """Simple test of a Python interpreter augmented with custom
    commands."""

    commands = { \
    "attach" : "DoAttach"
    }

    def __init__(self, locals = None):

    # Call super-class initializer
    code.InteractiveConsole.__init__(self, locals, "<console>")

    # Compile regular expression for finding commmands
    self.regexp = re.compile('[a-z]*')


    def interact(self):

    my_dictionary = self.locals

    # Set the primary and secondary prompts
    sys.ps1 = ">>> "
    sys.ps2 = "... "

    # Input Loop
    is_more = 0
    bosco = 0
    while 1:
    try :
    # Display the appropriate prompt
    if not sys.stdin.isatty():
    prompt = ""
    elif is_more:
    prompt = sys.ps2
    else:
    prompt = sys.ps1

    # Read the next line of input
    #self.write("interact 1\n")
    line = self.raw_input(prompt)

    # TODO: add logging of input line here...

    # Process complete lines
    if 1:
    line = self.process(line)

    # Push incomplete lines onto input stack
    if line or is_more:
    is_more = self.push(line)

    # Handle CTRL-C
    except KeyboardInterrupt:
    self.write("\nKeyboardInterrupt\n")
    is_more = 0
    self.resetbuffer()

    # Handle CTRL-D
    except EOFError:
    self.write("\n")
    is_more = 0
    self.resetbuffer()
    raise SystemExit


    def process(parent, line):

    # Attempt to match line against our command regular expression

    temp_line = string.lstrip(line)
    len_1 = len(line)
    len_2 = len(temp_line)
    white_spaces = len_1-len_2
    if white_spaces is not 0:
    front_padding = line[0:white_spaces]

    match = parent.regexp.match(temp_line)
    if match is not None:

    # Extract the command and argument strings
    cmd_string = match.group()
    arg_string = string.lstrip(temp_line[match.end():])

    # Find the function for this command in the command
    dictionary
    function = parent.commands.get(cmd_string)

    if function is not None:

    # Split argument string into individual arguments
    args = string.split(arg_string)

    # Recursively evaluate all arguments
    i = 0
    while i < len(args):
    try:

    # Grab value and position of this argument
    unevaluated, pos, i = args, i, i + 1

    # Have Python attempt evaluation of the argument

    evaluated = eval(unevaluated, parent.locals)

    # Did the evaluation produce a "new" result?
    if str(evaluated) != str(unevaluated):

    # Place the evaluation in the argument list
    args = args[:pos] + \
    string.split(str(evaluated)) + \
    args[pos + 1:]

    # Go back to the beginning of the argument
    list
    i = 0

    except (SyntaxError, NameError):
    pass



    # Convert to Python function-call syntax for this
    command
    line = "myparse." + function + "(" + str(args) + ")"
    if white_spaces is not 0:
    line = front_padding + line

    # Return the line to be processed by Python
    return line
    Jack Carter, Sep 18, 2004
    #1
    1. Advertising

  2. Jack Carter

    Peter Otten Guest

    Jack Carter wrote:

    > This will result in:
    >
    > >>> bosco=5
    > >>> if 1:

    > ... attach bosco
    > ... bosco=7
    > ... attach bosco
    > ...
    > DoAttach: ['5']
    > DoAttach: ['5'] <--- WRONG (at least, not what I want)
    > >>> attach bosco

    > DoAttach: ['7']


    The suite

    attach bosco
    bosco = 7
    attach bosco

    is only executed after the final empty line. Therefore any access to the
    right dictionary in the process() method will give you the current value
    bosco==5 which is not what you want.
    You are in effect translating the suite into (simplified)

    DoAttach(5)
    bosco = 7
    DoAttach(5) # the previous line has not yet been executed

    I think the easiest way to get the desired effect ("late binding") is like
    so:

    def process(self, line):
    temp_line = line.lstrip()
    front_padding = line[:len(line)-len(temp_line)]

    match = self.regexp.match(temp_line)
    if match:
    line = front_padding + "myparse.DoAttach(bosco)"

    return line

    I. e. change the generated line to contain variable names instead of values
    and leave the resolution to python. (You hint you don't wont that either.
    Why?)

    By the way, I wasn't able to run your code - the indentation is seriously
    messed up. I recommend you ensure a 4-space indentation in all code you
    currently have before you move on. That will spare you a lot of trouble
    later on.

    Another minor issue: 'value is 0' may or may not work depending on the
    python implementation ('value is 1000' won't work even in current CPython).
    With 'value == 0' you are on the safe side.

    Peter
    Peter Otten, Sep 18, 2004
    #2
    1. Advertising

  3. Jack Carter

    Jack Carter Guest

    Peter,

    I guess I just don't understand.

    Basically this is to go into a tool that accepts
    commandline arguments, one at a time, but will also
    allow scripting for testing purposes. Thus the desire
    to leverage off python for the commandline interface,
    but not for the whole program.

    The input from the user will be either from the console or
    read from a command file. A simple example of what could
    be entered is:

    bosco=5
    if 1:
    print bosco
    attach bosco 9
    bosco=7
    print bosco
    attach bosco 9

    The result I would expect would be for my DoAttach()
    routine to receive the python evaluated value of each
    of the arguments leaving the ones that it doesn't understand
    alone. Whether this happens automagically or by hand I don't
    care as long as I get what right value.

    In the above trying to follow your advice, remembering that
    at this stage of the game I am probably missing the point, this
    is the result I expect:

    johmar % demo.py
    >>> bosco=5
    >>> if 1:

    .... print bosco
    .... attach bosco 9
    .... bosco=7
    .... print bosco
    .... attach bosco 9
    ....
    5
    DoAttach: [5, 9]
    7
    DoAttach: [7, 9]

    This is what I get:

    johmar % demo.py
    >>> bosco=5
    >>> if 1:

    .... print bosco
    .... attach bosco 9
    .... bosco=7
    .... print bosco
    .... attach bosco 9
    ....
    5
    DoAttach: ['bosco', '9']
    7
    DoAttach: ['bosco', '9']

    Now I realize that this is probably due to the fact that
    I have the lines:

    if white_spaces:
    line = front_padding + "myparse." + function + "(" +
    str(args) + ")"
    else :
    line = "myparse." + function + "(" + str(args) + ")"

    which put make the arguments strings, but that is because I don't
    know how to appropriately pack the "line" for later parsing. Maybe
    that is the crux of my problem. Remember, there will be many commands
    for my tool and the arguments will be variable length and this code
    will not know what variables the gentle use would use.

    Here is the simplified code with hopefully the tabs expanded
    base on your earlier input. Hopefully you'll see the obvious
    error of my way and point it out.

    Thanks ever so much,

    Jack

    **********************************
    demo.py
    **********************************
    #!/usr/bin/env python

    import myparse

    cli = myparse.CLI(globals())
    cli.interact()

    **********************************
    myparse.py
    **********************************
    import code
    import re
    import string
    import sys

    ################################################################################
    #
    # DoAttach
    #
    # Dummy function that I will eventually use to do
    # real stuff.
    #
    ################################################################################
    def DoAttach(args):

    print "DoAttach:", args
    pass


    class CLI(code.InteractiveConsole):
    """Simple test of a Python interpreter augmented with custom commands."""

    commands = { \
    "attach" : "DoAttach"
    }

    def __init__(self, locals = None):

    # Call super-class initializer
    code.InteractiveConsole.__init__(self, locals, "<console>")

    # Compile regular expression for finding commmands
    self.regexp = re.compile('[a-z]*')


    ##################################################################
    #
    # interact
    #
    # This will read and process input lines from within
    # my main application as though on a python commandline.
    #
    ##################################################################
    def interact(self):

    # Set the primary and secondary prompts
    sys.ps1 = ">>> "
    sys.ps2 = "... "

    # Input Loop
    is_more = 0
    bosco = 0
    while 1:
    try :
    # Display the appropriate prompt
    if not sys.stdin.isatty():
    prompt = ""
    elif is_more:
    prompt = sys.ps2
    else:
    prompt = sys.ps1

    # Read the next line of input
    #self.write("interact 1\n")
    line = self.raw_input(prompt)

    # TODO: add logging of input line here...

    # Process complete lines
    if 1:
    line = self.process(line)

    # Push incomplete lines onto input stack
    if line or is_more:
    is_more = self.push(line)

    # Handle CTRL-C
    except KeyboardInterrupt:
    self.write("\nKeyboardInterrupt\n")
    is_more = 0
    self.resetbuffer()

    # Handle CTRL-D
    except EOFError:
    self.write("\n")
    is_more = 0
    self.resetbuffer()
    raise SystemExit

    ##################################################################
    #
    # process
    #
    # This will determine if the input command is either
    # from my application's command language or a python
    # construct.
    #
    ##################################################################
    def process(parent, line):

    # Attempt to match line against our command regular expression

    temp_line = string.lstrip(line)
    len_1 = len(line)
    len_2 = len(temp_line)

    white_spaces = len_1-len_2
    if white_spaces:
    front_padding = line[0:white_spaces]

    match = parent.regexp.match(temp_line)
    if match is not None:

    #parent.write("process 1\n")
    # Extract the command and argument strings
    cmd_string = match.group()
    arg_string = string.lstrip(temp_line[match.end():])

    # Find the function for this command in the command dictionary
    function = parent.commands.get(cmd_string)

    if function is not None:

    # Split argument string into individual arguments
    args = string.split(arg_string)

    # Convert to Python function-call syntax for this command
    if white_spaces:
    line = front_padding + "myparse." + function + "(" +
    str(args) + ")"
    else :
    line = "myparse." + function + "(" + str(args) + ")"

    # Return the line to be processed by Python
    return line
    Jack Carter, Sep 20, 2004
    #3
  4. Jack Carter

    Peter Otten Guest

    Jack Carter wrote:

    > Basically this is to go into a tool that accepts
    > commandline arguments, one at a time, but will also
    > allow scripting for testing purposes. Thus the desire
    > to leverage off python for the commandline interface,
    > but not for the whole program.
    >
    > The input from the user will be either from the console or
    > read from a command file. A simple example of what could
    > be entered is:
    >
    > bosco=5
    > if 1:
    > print bosco
    > attach bosco 9
    > bosco=7
    > print bosco
    > attach bosco 9
    >
    > The result I would expect would be for my DoAttach()
    > routine to receive the python evaluated value of each
    > of the arguments leaving the ones that it doesn't understand
    > alone. Whether this happens automagically or by hand I don't
    > care as long as I get what right value.
    >
    > In the above trying to follow your advice, remembering that
    > at this stage of the game I am probably missing the point, this
    > is the result I expect:
    >
    > johmar % demo.py
    >>>> bosco=5
    >>>> if 1:

    > ... print bosco
    > ... attach bosco 9
    > ... bosco=7
    > ... print bosco
    > ... attach bosco 9
    > ...
    > 5
    > DoAttach: [5, 9]
    > 7
    > DoAttach: [7, 9]
    >
    > This is what I get:
    >
    > johmar % demo.py
    >>>> bosco=5
    >>>> if 1:

    > ... print bosco
    > ... attach bosco 9
    > ... bosco=7
    > ... print bosco
    > ... attach bosco 9
    > ...
    > 5
    > DoAttach: ['bosco', '9']
    > 7
    > DoAttach: ['bosco', '9']
    >
    > Now I realize that this is probably due to the fact that
    > I have the lines:
    >
    > if white_spaces:
    > line = front_padding + "myparse." + function + "(" +
    > str(args) + ")"
    > else :
    > line = "myparse." + function + "(" + str(args) + ")"
    >
    > which put make the arguments strings, but that is because I don't
    > know how to appropriately pack the "line" for later parsing. Maybe
    > that is the crux of my problem. Remember, there will be many commands
    > for my tool and the arguments will be variable length and this code
    > will not know what variables the gentle use would use.
    >
    > Here is the simplified code with hopefully the tabs expanded
    > base on your earlier input. Hopefully you'll see the obvious
    > error of my way and point it out.
    >
    > Thanks ever so much,
    >
    > Jack
    >
    > **********************************
    > demo.py
    > **********************************
    > #!/usr/bin/env python
    >
    > import myparse
    >
    > cli = myparse.CLI(globals())
    > cli.interact()
    >
    > **********************************
    > myparse.py
    > **********************************
    > import code
    > import re
    > import string
    > import sys
    >
    >

    ################################################################################
    > #
    > # DoAttach
    > #
    > # Dummy function that I will eventually use to do
    > # real stuff.
    > #
    >

    ################################################################################
    > def DoAttach(args):
    >
    > print "DoAttach:", args
    > pass
    >
    >
    > class CLI(code.InteractiveConsole):
    > """Simple test of a Python interpreter augmented with custom
    > commands."""
    >
    > commands = { \
    > "attach" : "DoAttach"
    > }
    >
    > def __init__(self, locals = None):
    >
    > # Call super-class initializer
    > code.InteractiveConsole.__init__(self, locals, "<console>")
    >
    > # Compile regular expression for finding commmands
    > self.regexp = re.compile('[a-z]*')
    >
    >
    > ##################################################################
    > #
    > # interact
    > #
    > # This will read and process input lines from within
    > # my main application as though on a python commandline.
    > #
    > ##################################################################
    > def interact(self):
    >
    > # Set the primary and secondary prompts
    > sys.ps1 = ">>> "
    > sys.ps2 = "... "
    >
    > # Input Loop
    > is_more = 0
    > bosco = 0
    > while 1:
    > try :
    > # Display the appropriate prompt
    > if not sys.stdin.isatty():
    > prompt = ""
    > elif is_more:
    > prompt = sys.ps2
    > else:
    > prompt = sys.ps1
    >
    > # Read the next line of input
    > #self.write("interact 1\n")
    > line = self.raw_input(prompt)
    >
    > # TODO: add logging of input line here...
    >
    > # Process complete lines
    > if 1:
    > line = self.process(line)
    >
    > # Push incomplete lines onto input stack
    > if line or is_more:
    > is_more = self.push(line)
    >
    > # Handle CTRL-C
    > except KeyboardInterrupt:
    > self.write("\nKeyboardInterrupt\n")
    > is_more = 0
    > self.resetbuffer()
    >
    > # Handle CTRL-D
    > except EOFError:
    > self.write("\n")
    > is_more = 0
    > self.resetbuffer()
    > raise SystemExit
    >
    > ##################################################################
    > #
    > # process
    > #
    > # This will determine if the input command is either
    > # from my application's command language or a python
    > # construct.
    > #
    > ##################################################################
    > def process(parent, line):
    >
    > # Attempt to match line against our command regular expression
    >
    > temp_line = string.lstrip(line)
    > len_1 = len(line)
    > len_2 = len(temp_line)
    >
    > white_spaces = len_1-len_2
    > if white_spaces:
    > front_padding = line[0:white_spaces]
    >
    > match = parent.regexp.match(temp_line)
    > if match is not None:
    >
    > #parent.write("process 1\n")
    > # Extract the command and argument strings
    > cmd_string = match.group()
    > arg_string = string.lstrip(temp_line[match.end():])
    >
    > # Find the function for this command in the command dictionary
    > function = parent.commands.get(cmd_string)
    >
    > if function is not None:
    >
    > # Split argument string into individual arguments
    > args = string.split(arg_string)
    >
    > # Convert to Python function-call syntax for this command
    > if white_spaces:
    > line = front_padding + "myparse." + function + "(" +
    > str(args) + ")"
    > else :
    > line = "myparse." + function + "(" + str(args) + ")"
    >

    # let's add some feedback
    print "fed to the snake:", line

    > # Return the line to be processed by Python
    > return line



    Now

    >>> bosco = 1

    fed to the snake: bosco = 1
    >>> attach bosco 2

    fed to the snake: myparse.DoAttach(['bosco', '2'])
    DoAttach: ['bosco', '2']
    >>>


    You build a function call

    myparse.DoAttach(['bosco', '2'])

    But what you need would rather be

    myparse.DoAttach([bosco, 2])

    To achieve that you have to somehow extract (Python-compatible) expressions
    for the arguments given in the line

    attach bosco 2

    which can be arbitrarily complex depending on how you defined your custom
    language. Assumming that you use the simplest possible spec, a
    space-separated list of already Python-compatible expressions that gives
    you

    def parseArgs(args):
    return args.split()

    def makePythonCall(func, args):
    return "%s([%s])" % (func, ", ".join(args))

    and the process() method will become:

    def process(parent, line):

    # Attempt to match line against our command regular expression

    temp_line = string.lstrip(line)
    len_1 = len(line)
    len_2 = len(temp_line)

    white_spaces = len_1-len_2
    if white_spaces:
    front_padding = line[0:white_spaces]

    match = parent.regexp.match(temp_line)
    if match is not None:

    #parent.write("process 1\n")
    # Extract the command and argument strings
    cmd_string = match.group()
    arg_string = string.lstrip(temp_line[match.end():])

    # Find the function for this command in the command dictionary
    function = parent.commands.get(cmd_string)

    if function is not None:

    args = parseArgs(arg_string)
    line = makePythonCall("myparse." + function, args)
    if white_spaces:
    line = front_padding + line
    print "fed to the snake:", line

    # Return the line to be processed by Python
    return line


    Testing it:

    >>> bosco = 1

    fed to the snake: bosco = 1
    >>> attach bosco 2

    fed to the snake: myparse.DoAttach([bosco, 2])
    DoAttach: [1, 2]
    >>> if 1:

    fed to the snake: if 1:
    .... attach bosco
    fed to the snake: myparse.DoAttach([bosco])
    .... bosco = 3
    fed to the snake: bosco = 3
    .... attach bosco
    fed to the snake: myparse.DoAttach([bosco])
    ....
    fed to the snake:
    DoAttach: [1]
    DoAttach: [3]

    Works here, but is not very robust:

    >>> attach = 99

    fed to the snake: myparse.DoAttach([=, 99])
    File "<console>", line 1
    myparse.DoAttach([=, 99])
    ^
    SyntaxError: invalid syntax
    >>> attach(bosco)

    fed to the snake: myparse.DoAttach([(bosco)])
    DoAttach: [3]
    >>> attach bosco "so what"

    fed to the snake: myparse.DoAttach([bosco, "so, what"])
    DoAttach: [3, 'so, what'] # note the comma

    Peter
    Peter Otten, Sep 21, 2004
    #4
  5. Jack Carter

    Jack Carter Guest

    Peter,

    Actually although your solution is a good one, it won't
    really help me because of the nature of our command language.

    We have commands in the form of:

    result = <command_name> <filename> -e <experiment_name>

    I currently scan ahead for <command_name> and could do so for
    the rest to prevent them from being evaluated by python by
    packing them with quotes and then unpacking them later. The
    problem lies in the case where the name being used in the
    commandline may or may not be a formal name such as a filename
    or may be a variable that will evaluate into a filename.

    This will be used in a list of filenames or a list of experiment
    names iterated through a for loop.

    >>> for name in (['file1','file2','file3']):

    .... expClose name
    ....
    expClose: ['file1']
    expClose: ['file2']
    expClose: ['file3']

    But when the user tries to use a formal name that is
    not through a python variable he/she hits an undefine
    name error:

    >>> expClose file4

    Traceback (most recent call last):
    File "<console>", line 1, in ?
    NameError: name 'file4' is not defined

    If I prescan the <filename> and pack it with quotes I
    lose the python evaluation of the name.

    The same problem will occur with all my other option arguments
    for which there are many.

    The solution it would seem would be to do the evaluation
    later within the called function. That way I could assume
    that all failed eval()'ed names are literals meant for my
    command and not a python variable/name.

    If this makes sense, the problem I need to solve is how to
    deliver the correct namespace dictionary to the called function
    so I can invoke eval with it.

    Does this make sense?

    Regards,

    Jack
    Jack Carter, Sep 22, 2004
    #5
  6. Jack Carter <> wrote:
    ...
    > The solution it would seem would be to do the evaluation
    > later within the called function. That way I could assume
    > that all failed eval()'ed names are literals meant for my
    > command and not a python variable/name.
    >
    > If this makes sense, the problem I need to solve is how to
    > deliver the correct namespace dictionary to the called function
    > so I can invoke eval with it.
    >
    > Does this make sense?


    Not very, but then I didn't follow the previous LONG posts on this
    thread, so I'll just answer this specific question and hope it helps.
    I'll assume the known variable-names are in some dictionary (such as a
    locals() or globals() or vars(something)):

    class WeirdNamespace:
    def __init__(self, d): self.d = d
    def __getitem__(self, n): return self.d.get(n,repr(n))

    voila: if n is a key in dict d, this returns the corresponding value,
    otherwise it returns n suitably quoted. Just pass to eval a
    WeirdNamespace(d) rather than the bare d.

    It appears to me your user interface is courting trouble: if I mispell
    'variablename' as 'varaiblename' I end up creating a file I didn't mean
    to rather than getting a clean error about unknown variable names. But
    I'll assume you know your users better than I do and that they _do_
    really desire with all their hearts this unholy confusion between
    variables and constants...


    Alex
    Alex Martelli, Sep 22, 2004
    #6
  7. Jack Carter

    Jack Carter Guest

    On Sep 22, 7:18pm, Alex Martelli wrote:
    > Subject: Re: namespace/dictionary quandry
    > Jack Carter <> wrote:
    > ...
    > > The solution it would seem would be to do the evaluation
    > > later within the called function. That way I could assume
    > > that all failed eval()'ed names are literals meant for my
    > > command and not a python variable/name.
    > >
    > > If this makes sense, the problem I need to solve is how to
    > > deliver the correct namespace dictionary to the called function
    > > so I can invoke eval with it.
    > >
    > > Does this make sense?

    >
    > Not very, but then I didn't follow the previous LONG posts on this


    Alex,

    The length was really taken up with a testcase to make
    the problem less hand wavey.

    > thread, so I'll just answer this specific question and hope it helps.
    > I'll assume the known variable-names are in some dictionary (such as a
    > locals() or globals() or vars(something)):
    >
    > class WeirdNamespace:
    > def __init__(self, d): self.d = d
    > def __getitem__(self, n): return self.d.get(n,repr(n))


    So, how and or where does this fit in with my example?
    Does both the call to the function where I want to do
    the eval() and self.push(line) command have to be in the
    same namespace and or file for this to work?

    >
    > voila: if n is a key in dict d, this returns the corresponding value,
    > otherwise it returns n suitably quoted. Just pass to eval a
    > WeirdNamespace(d) rather than the bare d.
    >
    > It appears to me your user interface is courting trouble: if I mispell
    > 'variablename' as 'varaiblename' I end up creating a file I didn't mean
    > to rather than getting a clean error about unknown variable names. But
    > I'll assume you know your users better than I do and that they _do_
    > really desire with all their hearts this unholy confusion between
    > variables and constants...


    This is driven by customers that who want both a type-in command
    language and a variant using all the power of python for test
    scripting purposes. If you misspell something and there is the
    possibility of it screwing something up that we can't tell either
    in the parser or downstream in the backend of the tool, well that's
    life in the big city.

    Thanks,

    Jack
    Jack Carter, Sep 22, 2004
    #7
  8. Jack Carter

    Peter Otten Guest

    Jack Carter wrote:

    >> class WeirdNamespace:
    >> def __init__(self, d): self.d = d
    >> def __getitem__(self, n): return self.d.get(n,repr(n))

    >
    > So, how and or where does this fit in with my example?
    > Does both the call to the function where I want to do
    > the eval() and self.push(line) command have to be in the
    > same namespace and or file for this to work?


    I use a slight variation of Alex' suggestion:

    class Locals(dict):
    def __getitem__(self, name):
    try:
    return dict.__getitem__(self, name)
    except KeyError:
    return name

    class CLI(code.InteractiveConsole):
    """Simple test of a Python interpreter augmented with custom
    commands."""

    commands = { \
    "attach" : "DoAttach"
    }

    def __init__(self, locals = None):

    # Call super-class initializer
    code.InteractiveConsole.__init__(self, locals, "<console>")
    self.locals = Locals(self.locals)

    # Compile regular expression for finding commmands
    self.regexp = re.compile('[a-z]*')

    [The rest of the code is the same as in my previous post]

    Now try it:

    >>> for name in "abc":

    fed to the snake: for name in "abc":
    .... attach name
    fed to the snake: myparse.DoAttach([name])
    .... attach noname
    fed to the snake: myparse.DoAttach([noname])
    ....
    fed to the snake:
    DoAttach: ['a']
    DoAttach: ['noname']
    DoAttach: ['b']
    DoAttach: ['noname']
    DoAttach: ['c']
    DoAttach: ['noname']

    Unfortunately this will only work with Python 2.4.

    Here's a solution that might work for 2.3:

    [Does not require the above modifications]

    def process(self, line):
    indent = line.lstrip()
    # Attempt to match line against our command regular expression

    temp_line = string.lstrip(line)
    len_1 = len(line)
    len_2 = len(temp_line)

    white_spaces = len_1-len_2
    if white_spaces:
    front_padding = line[0:white_spaces]

    match = self.regexp.match(temp_line)
    if match is not None:

    #self.write("process 1\n")
    # Extract the command and argument strings
    cmd_string = match.group()
    arg_string = string.lstrip(temp_line[match.end():])

    # Find the function for this command in the command dictionary
    function = self.commands.get(cmd_string)

    if function is not None:

    args = parseArgs(arg_string)
    for arg in args:
    if arg not in self.locals:
    self.locals[arg] = arg
    line = makePythonCall("myparse." + function, args)
    if white_spaces:
    line = front_padding + line
    print "fed to the snake:", line

    # Return the line to be processed by Python
    return line

    I just ensure that all arguments not already in self.locals are added with
    their name as their value. (I also renamed 'parent' to 'self' - I could not
    stand it any longer :)

    Note that I don't particularly like both hacks and would rather use plain
    old python functions with standard python syntax instead of your custom
    language.


    Peter
    Peter Otten, Sep 22, 2004
    #8
  9. Jack Carter <> wrote:
    ...
    > > > later within the called function. That way I could assume
    > > > that all failed eval()'ed names are literals meant for my
    > > > command and not a python variable/name.

    ...
    > The length was really taken up with a testcase to make
    > the problem less hand wavey.


    If you can't make a shorter example, I'm not gonna make time to study
    that one, sorry.


    > > I'll assume the known variable-names are in some dictionary (such as a
    > > locals() or globals() or vars(something)):
    > >
    > > class WeirdNamespace:
    > > def __init__(self, d): self.d = d
    > > def __getitem__(self, n): return self.d.get(n,repr(n))

    >
    > So, how and or where does this fit in with my example?
    > Does both the call to the function where I want to do
    > the eval() and self.push(line) command have to be in the
    > same namespace and or file for this to work?


    All it does is implement what you asked: make a mapping that assumes all
    unknown variable names are to be taken as literals, starting from a
    mapping of all 'known variable names' to their values.


    > > I'll assume you know your users better than I do and that they _do_
    > > really desire with all their hearts this unholy confusion between
    > > variables and constants...

    >
    > This is driven by customers that who want both a type-in command
    > language and a variant using all the power of python for test
    > scripting purposes. If you misspell something and there is the
    > possibility of it screwing something up that we can't tell either
    > in the parser or downstream in the backend of the tool, well that's
    > life in the big city.


    I suspect the first time your users want to use a file named 'for'
    they'll scream bloody murder, but hopefully that won't happen before
    they've paid you enough for you to retire in comfort to a nice
    Carribbean island without extradition treaties, so, cheer up.


    Alex
    Alex Martelli, Sep 22, 2004
    #9
  10. Jack Carter

    Jack Carter Guest

    On Sep 22, 8:48pm, Peter Otten wrote:
    > Subject: Re: namespace/dictionary quandry


    > I just ensure that all arguments not already in self.locals are added with
    > their name as their value. (I also renamed 'parent' to 'self' - I could not
    > stand it any longer :)


    Novice error on my part. Thanks for the correction.

    >
    > Note that I don't particularly like both hacks and would rather use plain
    > old python functions with standard python syntax instead of your custom
    > language.


    It makes the programmers job easier, but not the customer
    who may just want a non-gui debugger commandline session tool.
    That customer doesn't want to know anything about python or
    function calls even though underneath the covers that's what
    they are getting. At the same time, the testers want the full
    scripting power of python.

    To make life even more fun, there is a C++ front end and C++
    back end that deal with many processes over many nodes. Some
    of the commands are syncronous and return a value that feed
    into the python name space and others are asyncronous that
    may implode somewhere in the future and need to bring the
    whole mess to some sane state to handle the problem

    I guess it is my own private hell to work on.

    Your change seems to work great!

    Thanks,

    Jack
    >
    >
    > Peter
    >
    Jack Carter, Sep 22, 2004
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Fred
    Replies:
    3
    Views:
    1,082
    Bobby Ryzhy
    Jul 12, 2004
  2. Ilias Lazaridis
    Replies:
    6
    Views:
    437
    Ilias Lazaridis
    Feb 21, 2006
  3. BMarsh
    Replies:
    7
    Views:
    274
    infobahn
    Jan 14, 2005
  4. design quandry ..

    , Oct 26, 2005, in forum: C++
    Replies:
    2
    Views:
    274
  5. james_027
    Replies:
    1
    Views:
    322
    Marc 'BlackJack' Rintsch
    Aug 22, 2007
Loading...

Share This Page