Shell quoting as part of the standard library?

Discussion in 'Python' started by David M. Wilson, Dec 13, 2003.

  1. Hello fellow users!

    I've been using Python in a couple of different environments for a few
    years now. I have quite often found that I have needed to deal with
    filenames which may contain characters that confuse the UNIX shell
    (space and double quote in particular).

    Quite a while ago I came up with two simple functions which allowed me
    to safely build a shell command line quickly. I've used the functions
    hundreds of times by now, and find myself constantly cut/pasting them
    into new projects. It is very likely that someone other than myself
    would find these functions to be useful, and I was wondering if such a
    thing might find a home in the standard library.

    http://botanicus.net/dw/tmp/shellquote.py

    Does this sort of thing have a home in the standard library? It
    doesn't really apply across platforms, which is my only real concern.
    Where should it be added to? shutil?

    If you are thinking of commenting on the way shell_quote_string always
    quotes, regardless of whether it is needed, please don't. If it acted
    in a more human-friendly manner then it would be called
    shell_friendly_quote_string, or something even more insane than that.
    :)

    Thanks,


    David.
     
    David M. Wilson, Dec 13, 2003
    #1
    1. Advertising

  2. David M. Wilson

    Peter Otten Guest

    David M. Wilson wrote:

    > I've been using Python in a couple of different environments for a few
    > years now. I have quite often found that I have needed to deal with
    > filenames which may contain characters that confuse the UNIX shell
    > (space and double quote in particular).
    >
    > Quite a while ago I came up with two simple functions which allowed me
    > to safely build a shell command line quickly. I've used the functions
    > hundreds of times by now, and find myself constantly cut/pasting them
    > into new projects. It is very likely that someone other than myself
    > would find these functions to be useful, and I was wondering if such a
    > thing might find a home in the standard library.
    >
    > http://botanicus.net/dw/tmp/shellquote.py
    >
    > Does this sort of thing have a home in the standard library? It
    > doesn't really apply across platforms, which is my only real concern.
    > Where should it be added to? shutil?


    >>> from commands import mkarg
    >>> mkarg("one silver $")

    " 'one silver $'"
    >>> mkarg("one's own $")

    ' "one\'s own \\$"'

    So something very similar for argument preparation is already there. The
    joining, as far as I know, currently has to be done manually:

    >>> "".join([mkarg(a) for a in ["a a", "$'"]])

    ' \'a a\' "\\$\'"'

    Peter
     
    Peter Otten, Dec 13, 2003
    #2
    1. Advertising

  3. David M. Wilson wrote in message
    <>...
    >Hello fellow users!
    >
    >I've been using Python in a couple of different environments for a few
    >years now. I have quite often found that I have needed to deal with
    >filenames which may contain characters that confuse the UNIX shell
    >(space and double quote in particular).


    Isn't this the more general problem of escaping shell metachars in
    user-supplied input?

    Why don't you just let the shell deal with it?

    $ cat > do_st_evil
    #! /bin/sh
    echo "Gotcha!"
    ^D
    $ chmod u+x do_st_evil
    $ python
    Python 2.2.1 (#1, Apr 21 2002, 08:38:44)
    [GCC 2.95.4 20011002 (Debian prerelease)] on linux2
    Type "help", "copyright", "credits" or "license" for more information
    >>> import os
    >>> badness = '$(./do_st_evil)'
    >>> os.popen('echo '+badness).read()

    'Gotcha!\n'
    >>> os.environ['shellvar'] = badness
    >>> os.popen('echo $shellvar').read()

    '$(./do_st_evil)\n'

    --
    Francis Avila
     
    Francis Avila, Dec 14, 2003
    #3
  4. Peter Otten <> wrote...

    > >>> from commands import mkarg
    > >>> mkarg("one silver $")

    > " 'one silver $'"
    > >>> mkarg("one's own $")

    > ' "one\'s own \\$"'


    I don't get it. I just don't get it. I'm really starting to doubt I
    have any initiative whatsoever. :)

    Ok, yet another mystery "why hasn't python got that" solved. Thanks!


    David.
     
    David M. Wilson, Dec 14, 2003
    #4
  5. David M. Wilson

    Oren Tirosh Guest

    On Sat, Dec 13, 2003 at 02:39:52PM -0800, David M. Wilson wrote:
    > Hello fellow users!
    >
    > I've been using Python in a couple of different environments for a few
    > years now. I have quite often found that I have needed to deal with
    > filenames which may contain characters that confuse the UNIX shell
    > (space and double quote in particular).


    The best solution is often to bypass the shell (os.system) and execute
    the external command you want directly with os.spawnv. In this case the
    arguments are passed as a list of strings rather than a space-separated
    string and therefore need no quoting.

    Oren
     
    Oren Tirosh, Dec 14, 2003
    #5
  6. "Francis Avila" <> wrote...

    > Why don't you just let the shell deal with it?
    >
    > ...
    >


    That is a very simple solution I have never seen used before. However,
    it does not solve my original problem. The solution as I have now been
    educated, is mkarg() from the commands module.

    Thanks,


    David.
     
    David M. Wilson, Dec 14, 2003
    #6
  7. Oren Tirosh <> wrote...

    > The best solution is often to bypass the shell (os.system) and execute
    > the external command you want directly with os.spawnv. In this case the
    > arguments are passed as a list of strings rather than a space-separated
    > string and therefore need no quoting.


    That would be a fine solution, except again it does not solve my
    problem. In the latest application of shell quoting, I needed to
    insert filenames into a shell excerpt provided by a user who is
    knowledgeable in shell.

    They could write, for instance:

    process_app %(input_filename) || something %(blah)
    [ -s %(output_filename) ] && ...


    But it would be above them to express that in Python, so bypassing the
    shell is not an option. Thanks,


    David.
     
    David M. Wilson, Dec 14, 2003
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Fredrik Lundh
    Replies:
    0
    Views:
    456
    Fredrik Lundh
    Feb 4, 2005
  2. steve.leach

    How standard is the standard library?

    steve.leach, Apr 18, 2005, in forum: Python
    Replies:
    1
    Views:
    418
    Christos TZOTZIOY Georgiou
    Apr 18, 2005
  3. funkyj
    Replies:
    5
    Views:
    1,184
    funkyj
    Jan 20, 2006
  4. Replies:
    5
    Views:
    600
    Pierre Asselin
    Feb 10, 2007
  5. rduke15
    Replies:
    7
    Views:
    232
    Stephane CHAZELAS
    May 24, 2004
Loading...

Share This Page