how to convert string to list or tuple

Discussion in 'Python' started by flyaflya, May 26, 2005.

  1. flyaflya

    flyaflya Guest

    a = "(1,2,3)"
    I want convert a to tuple:(1,2,3),but tuple(a) return ('(', '1', ',',
    '2', ',', '3', ')') not (1,2,3)
     
    flyaflya, May 26, 2005
    #1
    1. Advertising

  2. "flyaflya" <> wrote:
    >a = "(1,2,3)"
    > I want convert a to tuple:(1,2,3),but tuple(a) return ('(', '1', ',',
    > '2', ',', '3', ')') not (1,2,3)


    if you trust the source, use

    eval(a)

    if you don't trust it, you can use, say

    tuple(int(x) for x in re.findall("\d+", a))

    or, perhaps

    tuple(int(x) for x in a[1:-1].split(","))

    or some variation thereof.

    (if you're using a version older than 2.4, add brackets inside
    the tuple() call:

    tuple([int(x) for x in a[1:-1].split(",")])

    etc.

    </F>
     
    Fredrik Lundh, May 26, 2005
    #2
    1. Advertising

  3. On 5/26/05, flyaflya <> wrote:
    > a = "(1,2,3)"
    > I want convert a to tuple:(1,2,3),but tuple(a) return ('(', '1', ',',
    > '2', ',', '3', ')') not (1,2,3)


    Short answer - use eval().

    Long answer - *don't* use eval unless you are in control of the source
    of the string that you are evaluating.

    --
    Cheers,
    Simon B,
    ,
    http://www.brunningonline.net/simon/blog/
     
    Simon Brunning, May 26, 2005
    #3
  4. On Thu, 26 May 2005 19:53:38 +0800, flyaflya wrote:

    > a = "(1,2,3)"
    > I want convert a to tuple:(1,2,3),but tuple(a) return ('(', '1', ',',
    > '2', ',', '3', ')') not (1,2,3)


    Others have already given some suggestions. Here are some others.

    You didn't say where the input string a came from. Do you control
    it? Instead of using:

    String_Tuple_To_Real_Tuple("(1,2,3)")

    can you just create the tuple in the first place?

    a = (1, 2, 3)

    Second suggestion: if you know that the input string will ALWAYS be in the
    form "(1,2,3)" then you can do this:

    a = "(1,2,3)"
    a = a[1:-1] # deletes leading and trailing parentheses
    a = a.split(",") # creates a list ["1", "2", "3"] (items are strings)
    a = [int(x) for x in a] # creates a list [1, 2, 3] (items are integers)
    a = tuple(a) # coverts to a tuple

    or as a one-liner:

    a = "(1,2,3)"
    a = tuple([int(x) for x in a[1:-1].split(",")])

    Best of all, wrap your logic in a function definition with some
    error-checking:

    def String_Tuple_To_Real_Tuple(s):
    """Return a tuple of ints from a string that looks like a tuple."""
    if not s:
    return ()
    if (s[0] == "(") and s[-1] == ")"):
    s = s[1:-1]
    else:
    raise ValueError("Missing bracket(s) in string.")
    return tuple([int(x) for x in s.split(",")])


    Hope this helps,


    --
    Steven.
     
    Steven D'Aprano, May 29, 2005
    #4
  5. flyaflya

    Dan Bishop Guest

    Simon Brunning wrote:
    > On 5/26/05, flyaflya <> wrote:
    > > a = "(1,2,3)"
    > > I want convert a to tuple:(1,2,3),but tuple(a) return ('(', '1', ',',
    > > '2', ',', '3', ')') not (1,2,3)

    >
    > Short answer - use eval().
    >
    > Long answer - *don't* use eval unless you are in control of the source
    > of the string that you are evaluating.


    Or if you do use eval, don't give it access to any names.

    >>> import os
    >>> eval(raw_input(), {})

    os.system("rm -rf *")
    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    File "<string>", line 0, in ?
    NameError: name 'os' is not defined
     
    Dan Bishop, May 29, 2005
    #5
  6. flyaflya

    Duncan Booth Guest

    Dan Bishop wrote:

    > Simon Brunning wrote:
    >> [...]

    >
    > Or if you do use eval, don't give it access to any names.
    >
    >> [...]

    > os.system("rm -rf *")
    > Traceback (most recent call last):
    > File "<stdin>", line 1, in ?
    > File "<string>", line 0, in ?
    > NameError: name 'os' is not defined
    >

    Have you tried giving it the string '__import__("os").system("rm -rf *")'?
    [Don't try that at home children!]

    Even if you take steps to avoid that working by hiding the builtins, there
    are still too many ways to do nasty things with eval for it ever to be
    safe.
     
    Duncan Booth, May 29, 2005
    #6
  7. flyaflya

    John Roth Guest

    "Duncan Booth" <> wrote in message
    news:Xns9665B54CBAA38duncanbooth@127.0.0.1...
    > Dan Bishop wrote:
    >
    >> Simon Brunning wrote:
    >>> [...]

    >>
    >> Or if you do use eval, don't give it access to any names.
    >>
    >>> [...]

    >> os.system("rm -rf *")
    >> Traceback (most recent call last):
    >> File "<stdin>", line 1, in ?
    >> File "<string>", line 0, in ?
    >> NameError: name 'os' is not defined
    >>

    > Have you tried giving it the string '__import__("os").system("rm -rf *")'?
    > [Don't try that at home children!]
    >
    > Even if you take steps to avoid that working by hiding the builtins, there
    > are still too many ways to do nasty things with eval for it ever to be
    > safe.


    There was a posting here Nov 5, 2003 by Huaiyu Zhu at IBM Almaden
    that shows how to do eval type stuff safely. The basic notion is to use the
    compiler and then check the ast to see if the result fits the straitjacket
    you
    want to put it into. Pass / Fail; trying to fix it up if it's "close" is
    usually a
    real bad idea.

    He gives an example, and there's a much more extensive set of working
    code in the taBase.py module of PyFit that handles lists, tuples and
    dicts which contain arbitrary literals including complex and arbitrarily
    nested
    lists, tuples and dicts.

    ------- code snippet starts here --------

    def _safeEval(self, s):
    """
    Evaluate strings that only contain the following structures:
    const, tuple, list, dict
    Taken from c.l.py newsgroup posting Nov 5, 2003 by Huaiyu Zhu at IBM
    Almaden
    """
    #print "in _safeEval. input: '%s'" % s
    node1 = compiler.parse(s)

    # !!! special case of attempting to compile a lone string
    if node1.doc is not None and len(node1.node.nodes) == 0:
    #print "in _safeEval. string: '%s' found as docstring" %
    node1.doc
    return node1.doc

    #print "in _safeEval. nodes: '%s'" % (node1,)
    stmts = node1.node.nodes
    assert len(stmts) == 1
    node = compiler.parse(s).node.nodes[0]
    assert node.__class__ == compiler.ast.Discard
    nodes = node.getChildNodes()
    assert len(nodes) == 1
    result = self._safeAssemble(nodes[0])
    #print "in _safeEval result: '%s'" % (result,)
    return result

    seq_types = {
    compiler.ast.Tuple: tuple,
    compiler.ast.List: list,
    }
    map_types = {
    compiler.ast.Dict: dict,
    }

    oper_types = {
    compiler.ast.Add: operator.add,
    compiler.ast.Sub: operator.sub,
    }

    builtin_consts = {
    "True": True,
    "False": False,
    "None": None,
    }

    def _safeAssemble(self, node):
    """ Recursively assemble parsed ast node """
    cls = node.__class__
    if cls == compiler.ast.Const:
    return node.value
    elif cls in self.seq_types:
    nodes = node.nodes
    args = map(self._safeAssemble, nodes)
    return self.seq_types[cls](args)
    elif cls in self.map_types:
    keys, values = zip(*node.items)
    keys = map(self._safeAssemble, keys)
    values = map(self._safeAssemble, values)
    return self.map_types[cls](zip(keys, values))
    elif cls in self.oper_types:
    left = self._safeAssemble(node.left)
    right = self._safeAssemble(node.right)
    if type(left) == type(1.0j) or type(right) == type(1.0j):
    return self.oper_types[cls](left, right)
    else:
    raise FitException, ("Parse001",)
    elif cls == compiler.ast.Name:
    result = self.builtin_consts.get(node.name, "?")
    if result != "?":
    return result
    else:
    raise FitException, ("Parse002", node.name)
    else:
    raise FitException, ("Parse003", cls)

    ------- end of code snippet -----------

    John Roth


    >
     
    John Roth, May 29, 2005
    #7
  8. Duncan Booth wrote:
    > Dan Bishop wrote:
    >> Or if you do use eval, don't give it access to any names.

    [snip]
    >> os.system("rm -rf *")
    >> Traceback (most recent call last):
    >> File "<stdin>", line 1, in ?
    >> File "<string>", line 0, in ?
    >> NameError: name 'os' is not defined

    >
    > Have you tried giving it the string '__import__("os").system("rm -rf *")'?
    > [Don't try that at home children!]


    But you can try it at home if you set __builtins__ to something other
    than the default:

    py> eval("""__import__("os").system('echo "hello"')""",
    dict(__builtins__=None))
    Traceback (most recent call last):
    File "<interactive input>", line 1, in ?
    File "<string>", line 0, in ?
    NameError: name '__import__' is not defined

    If you're just doing work with constants, the lack of access to any
    builtins is ok:

    py> eval("(1,2,3)", dict(__builtins__=None))
    (1, 2, 3)

    I know there have been security holes in this technique before, but I
    looked at the archives, and all the old ones I found have been patched.
    (Or at least I wasn't able to reproduce them.)

    STeVe
     
    Steven Bethard, May 29, 2005
    #8
  9. flyaflya

    Duncan Booth Guest

    Steven Bethard wrote:

    >> Have you tried giving it the string '__import__("os").system("rm -rf
    >> *")'? [Don't try that at home children!]

    >
    > But you can try it at home if you set __builtins__ to something other
    > than the default:
    >
    > py> eval("""__import__("os").system('echo "hello"')""",
    > dict(__builtins__=None))
    > Traceback (most recent call last):
    > File "<interactive input>", line 1, in ?
    > File "<string>", line 0, in ?
    > NameError: name '__import__' is not defined
    >
    > If you're just doing work with constants, the lack of access to any
    > builtins is ok:
    >
    > py> eval("(1,2,3)", dict(__builtins__=None))
    > (1, 2, 3)
    >
    > I know there have been security holes in this technique before, but I
    > looked at the archives, and all the old ones I found have been
    > patched.
    > (Or at least I wasn't able to reproduce them.)
    >

    I guess you are referring to things like this not working when you use eval
    with an empty __builtins__:

    eval('''[ cls for cls in {}.__class__.__bases__[0].__subclasses__()
    if '_Printer' in `cls`
    ][0]._Printer__setup.func_globals['__builtins__']['__import__']''',
    dict(__builtins__=None))

    That gets blocked because func_globals is a 'restricted attribute', so I
    can't get directly at __import__ that way, but what I can do is to access
    any new style class you have defined and call any of its methods with
    whatever arguments I wish.

    Even with the big holes patched you are going to find it pretty hard to
    write a safe program that uses eval on untrusted strings. The only way to
    go is to filter the AST (or possibly the bytecode).
     
    Duncan Booth, May 30, 2005
    #9
  10. Duncan Booth wrote:
    > Steven Bethard wrote:
    >
    >>But you can try it at home if you set __builtins__ to something other
    >>than the default:
    >>
    >>py> eval("""__import__("os").system('echo "hello"')""",
    >>dict(__builtins__=None))
    >>Traceback (most recent call last):
    >> File "<interactive input>", line 1, in ?
    >> File "<string>", line 0, in ?
    >>NameError: name '__import__' is not defined
    >>

    [snip]
    >>
    >>I know there have been security holes in this technique before, but I
    >>looked at the archives, and all the old ones I found have been
    >>patched.
    >> (Or at least I wasn't able to reproduce them.)

    >
    > I guess you are referring to things like this not working when you use eval
    > with an empty __builtins__:
    >
    > eval('''[ cls for cls in {}.__class__.__bases__[0].__subclasses__()
    > if '_Printer' in `cls`
    > ][0]._Printer__setup.func_globals['__builtins__']['__import__']''',
    > dict(__builtins__=None))
    >
    > That gets blocked because func_globals is a 'restricted attribute', so I
    > can't get directly at __import__ that way


    Among other things, yes, that's one of the big ones. func_globals is
    inaccessible. Also, IIRC the file constructor is inaccessible.

    > but what I can do is to access
    > any new style class you have defined and call any of its methods with
    > whatever arguments I wish.


    Any new style class that I've defined? Or just any one I pass in as
    part of dict(__builtins__=None, ...)? If the former, could you
    elaborate? If the latter, then yes, I can see the problem. However for
    the case where all you pass in is dict(__builtins__=None), is there
    still a risk? Note that in the OP's case, all that is necessary is
    constant parsing, so no names need to be available.

    STeVe
     
    Steven Bethard, May 30, 2005
    #10
  11. flyaflya

    Duncan Booth Guest

    Steven Bethard wrote:

    > Duncan Booth wrote:
    >> any new style class you have defined and call any of its methods with
    >> whatever arguments I wish.

    >
    > Any new style class that I've defined? Or just any one I pass in as
    > part of dict(__builtins__=None, ...)? If the former, could you
    > elaborate? If the latter, then yes, I can see the problem. However
    > for the case where all you pass in is dict(__builtins__=None), is
    > there still a risk? Note that in the OP's case, all that is necessary
    > is constant parsing, so no names need to be available.
    >

    Any new style class you have defined is accessible through
    object.__subclasses__(), and as I showed object itself is always accessible
    through {}.__class__.__bases__[0].

    I'm assuming that the source code for your program is available. That means
    I can find the name of an interesting class which has a method that does
    something destructive, and call it.

    e.g. Assuming that the MyDatabase class does something nasty to a file:

    >>> class MyDatabase(object):

    def __init__(self, filename):
    self.filename = filename
    def initialise(self):
    print "Splat %s" % self.filename


    >>> eval('''[ cls for cls in {}.__class__.__bases__[0].__subclasses__()

    if 'MyDatabase' in `cls`
    ][0]('importantfile').initialise()''', dict(__builtins__=None))
    Splat importantfile
     
    Duncan Booth, May 31, 2005
    #11
  12. Duncan Booth wrote:
    > e.g. Assuming that the MyDatabase class does something nasty to a file:
    >
    >>>>class MyDatabase(object):

    >
    > def __init__(self, filename):
    > self.filename = filename
    > def initialise(self):
    > print "Splat %s" % self.filename
    >
    >>>>eval('''[ cls for cls in {}.__class__.__bases__[0].__subclasses__()

    >
    > if 'MyDatabase' in `cls`
    > ][0]('importantfile').initialise()''', dict(__builtins__=None))
    > Splat importantfile


    Interestingly, I don't seem to be able to create a file object as a
    class attribute in restricted mode:

    py> class C(object):
    .... def __init__(self):
    .... self.f = file('temp.txt', 'w')
    ....
    py> eval('''[ cls for cls in {}.__class__.__bases__[0].__subclasses__()
    if cls.__name__ == 'C'][0]().f.write("stuff")''', dict(__builtins__=None))
    Traceback (most recent call last):
    File "<interactive input>", line 1, in ?
    File "<string>", line 0, in ?
    AttributeError: 'C' object has no attribute 'f'
    py> eval('''[ cls for cls in {}.__class__.__bases__[0].__subclasses__()
    if cls.__name__ == 'C'][0]().__dict__''', dict(__builtins__=None))
    {}

    I don't get an error for calling the file constructor, but the f
    attribute is never set AFAICT.

    STeVe
     
    Steven Bethard, May 31, 2005
    #12
  13. flyaflya

    Duncan Booth Guest

    Steven Bethard wrote:

    > Interestingly, I don't seem to be able to create a file object as a
    > class attribute in restricted mode:
    >
    > py> class C(object):
    > ... def __init__(self):
    > ... self.f = file('temp.txt', 'w')
    > ...
    > py> eval('''[ cls for cls in
    > {}.__class__.__bases__[0].__subclasses__() if cls.__name__ ==
    > 'C'][0]().f.write("stuff")''', dict(__builtins__=None)) Traceback
    > (most recent call last):
    > File "<interactive input>", line 1, in ?
    > File "<string>", line 0, in ?
    > AttributeError: 'C' object has no attribute 'f'
    > py> eval('''[ cls for cls in
    > {}.__class__.__bases__[0].__subclasses__() if cls.__name__ ==
    > 'C'][0]().__dict__''', dict(__builtins__=None)) {}
    >


    Weird. I copied and paste your class and eval exactly (apart from deleting
    the ... prompts) and it worked exactly as expected: writing 'stuff' to
    temp.txt. (Python 2.4)
     
    Duncan Booth, Jun 1, 2005
    #13
  14. flyaflya

    Fuzzyman Guest

    Fuzzyman, Jun 1, 2005
    #14
  15. Duncan Booth wrote:
    > Steven Bethard wrote:
    >
    >
    >>Interestingly, I don't seem to be able to create a file object as a
    >>class attribute in restricted mode:
    >>
    >>py> class C(object):
    >>... def __init__(self):
    >>... self.f = file('temp.txt', 'w')
    >>...
    >>py> eval('''[ cls for cls in
    >>{}.__class__.__bases__[0].__subclasses__() if cls.__name__ ==
    >>'C'][0]().f.write("stuff")''', dict(__builtins__=None)) Traceback
    >>(most recent call last):
    >> File "<interactive input>", line 1, in ?
    >> File "<string>", line 0, in ?
    >>AttributeError: 'C' object has no attribute 'f'
    >>py> eval('''[ cls for cls in
    >>{}.__class__.__bases__[0].__subclasses__() if cls.__name__ ==
    >>'C'][0]().__dict__''', dict(__builtins__=None)) {}

    >
    > Weird. I copied and paste your class and eval exactly (apart from deleting
    > the ... prompts) and it worked exactly as expected: writing 'stuff' to
    > temp.txt. (Python 2.4)


    So, I played around with this a little bit. If I start up a new
    interpreter and type it in like above, I get the behavior you do. What
    I had actually done (abbreviated) was:

    py> class C(object):
    .... pass
    ....
    py> class C(object):
    .... def __init__(self):
    .... self.f = file('temp.txt', 'w')
    ....
    py> eval('''[ cls for cls in {}.__class__.__bases__[0].__subclasses__()
    if cls.__name__ == 'C'][0]().f.write("stuff")''', dict(__builtins__=None))
    Traceback (most recent call last):
    File "<interactive input>", line 1, in ?
    File "<string>", line 0, in ?
    AttributeError: 'C' object has no attribute 'f'

    And the problem with this is that both __main__.C objects are now
    subclasses of object:

    py> eval('''[ cls for cls in {}.__class__.__bases__[0].__subclasses__()
    if cls.__name__ == 'C']''', dict(__builtins__=None))
    [<class '__main__.C'>, <class '__main__.C'>]

    So I was getting the wrong __main__.C object. Sorry for the confusion!

    Now, even using this technique, *your* code can't call the file constructor:

    py> class C(object):
    .... def __init__(self):
    .... self.file = file
    ....
    py> eval('''[ cls for cls in {}.__class__.__bases__[0].__subclasses__()
    if cls.__name__ == 'C'][-1]().file("temp.txt", "w")''',
    dict(__builtins__=None))
    Traceback (most recent call last):
    File "<interactive input>", line 1, in ?
    File "<string>", line 0, in ?
    IOError: file() constructor not accessible in restricted mode

    But unless the person eval-ing your code *only* writes immaculate code I
    can see that you can probably screw them. ;) I wonder why
    __subclasses__ isn't a restricted attribute... Is it ever used for
    something that isn't evil? ;)

    STeVe
     
    Steven Bethard, Jun 1, 2005
    #15
  16. flyaflya

    Ruud de Jong Guest

    Steven Bethard schreef:
    > But unless the person eval-ing your code *only* writes immaculate code I
    > can see that you can probably screw them. ;) I wonder why
    > __subclasses__ isn't a restricted attribute... Is it ever used for
    > something that isn't evil? ;)
    >
    > STeVe


    Completely off topic, but I just cannot resist showing off.
    Some time ago I used __subclasses__ in a way that is not evil. I think.

    The details are described in the following thread:
    http://groups.google.nl/group/comp.lang.python/browse_thread/thread/5c1ccb986c66cdc1/

    A summary: I used __subclasses__ to apply the Chain-of-Responsibility
    pattern to object creation. The code would appear to instantiate
    an object of the root of a class hierarchy, but the actual object
    that was created would be an instance of a subclass.

    So to get back to your question: yes, there are non-evil
    uses for __subclasses__. Weird perhaps, but non-evil.
    Non-standard, sure . Too clever for my own good, very likely.

    Regards,

    Ruud

    --
    Ruud de Jong

    '@'.join('.'.join(s) for s in (['ruud','de','jong'],['tiscali','nl']))
     
    Ruud de Jong, Jun 1, 2005
    #16
  17. flyaflya

    Duncan Booth Guest

    Ruud de Jong wrote:

    > Steven Bethard schreef:
    >> But unless the person eval-ing your code *only* writes immaculate
    >> code I can see that you can probably screw them. ;) I wonder why
    >> __subclasses__ isn't a restricted attribute... Is it ever used for
    >> something that isn't evil? ;)
    >>
    >> STeVe

    >
    > Completely off topic, but I just cannot resist showing off.
    > Some time ago I used __subclasses__ in a way that is not evil. I
    > think.
    >
    > The details are described in the following thread:
    > http://groups.google.nl/group/comp.lang.python/browse_thread/thread/5c1
    > ccb986c66cdc1/
    >
    > A summary: I used __subclasses__ to apply the Chain-of-Responsibility
    > pattern to object creation. The code would appear to instantiate
    > an object of the root of a class hierarchy, but the actual object
    > that was created would be an instance of a subclass.
    >
    > So to get back to your question: yes, there are non-evil
    > uses for __subclasses__. Weird perhaps, but non-evil.
    > Non-standard, sure . Too clever for my own good, very likely.


    I've done almost exactly the same thing. The base class uses __subclasses__
    to find the best matching subclass based on the factory parameters. In my
    case I was retrieving files from the web, so I had a base Handler class and
    created HtmlHandler, ImageHandler &c.

    class Handler(object):
    '''Class to process files'''
    __map = {}

    @classmethod
    def _resolveClass(klass, isdir, name):
    map = Handler.__map
    if not map:
    for c in klass.__subclasses__():
    for ext in c.Extensions:
    map['.'+ext.lower()] = c

    if isdir:
    klass = FolderHandler
    else:
    ext = os.path.splitext(name)[1].lower()
    if ext not in map:
    map[ext] = DefaultHandler

    klass = map[ext]
    return klass(name)

    @classmethod
    def fromPathname(klass, name, path, uri, db):
    isdir = os.path.isdir(os.path.join(path, name))
    obj = klass._resolveClass(isdir, name)
    obj._initialize(name, path, uri, db)
    return obj

    @classmethod
    def fromUrl(klass, uri, text, db=None):
    ... and so on ...

    and then subclasses such as:

    class ImageHandler(Handler):
    Extensions = ('jpg', 'jpeg', 'gif', 'png')
    type = 'Image'

    class DefaultHandler(Handler):
    Extensions = ('',)
    type = 'Ignored'

    This also contains the only code I think I've written with a class
    definition in a for loop:

    # General categories
    EXTENSIONS = {
    'js': 'javascript',
    'php': 'php',
    'doc': 'Word Document',
    'xls': 'Spreadsheet',
    'ppt': 'Powerpoint',
    'css': 'Stylesheet',
    'swf': 'Flash',
    'pdf': 'File',
    'rtf': 'File',
    'zip': 'File',
    }

    Classes = []
    for ext in EXTENSIONS:
    class GeneralHandler(Handler):
    Extensions = (ext,)
    type = EXTENSIONS[ext]

    Classes.append(GeneralHandler)
     
    Duncan Booth, Jun 2, 2005
    #17
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Michal Mikolajczyk
    Replies:
    1
    Views:
    832
    Larry Bates
    Apr 20, 2004
  2. Jeff Epler
    Replies:
    0
    Views:
    1,026
    Jeff Epler
    Apr 20, 2004
  3. Bill Scherer
    Replies:
    0
    Views:
    631
    Bill Scherer
    Apr 20, 2004
  4. Davy
    Replies:
    3
    Views:
    1,908
    Wildemar Wildenburger
    Nov 7, 2007
  5. Poppy
    Replies:
    1
    Views:
    413
    Poppy
    May 30, 2008
Loading...

Share This Page