building strings with variable input

Discussion in 'Python' started by Olaf Meyer, Jan 12, 2004.

  1. Olaf Meyer

    Olaf Meyer Guest

    Sometimes if find it clumsy unsing the following approach building strings:

    cmd = "%s -start %s -end %s -dir %s" % (executable, startTime, endTime,
    directory)

    Especially if you have a lot of variable input it makes it hard to match
    the variables to the proper fields. From other scripting languanges I'm
    used to something like:

    $cmd = "$executable -start $startTime -end $endTime -dir $directory"

    This makes it very easy to see how the string is actually built. You
    dont't have to worry where which variables go.

    Is there a similar way to do this in python?

    Thanks,
    Olaf
     
    Olaf Meyer, Jan 12, 2004
    #1
    1. Advertising

  2. Olaf Meyer wrote:

    > Especially if you have a lot of variable input it makes it hard to
    > match
    > the variables to the proper fields. From other scripting languanges
    > I'm
    > used to something like:
    >
    > $cmd = "$executable -start $startTime -end $endTime -dir $directory"
    >
    > This makes it very easy to see how the string is actually built. You
    > dont't have to worry where which variables go.
    >
    > Is there a similar way to do this in python?


    Sure:

    cmd = "%(executable)s -start %(startTime)s -end %(endTime)s -dir
    %(directory)s" % locals()

    There are also more expansive solutions such as YAPTU or EmPy.

    Note, however, that what you are trying to do (presuming you're passing
    this to os.system or something similar) is potentially a serious
    security risk. If the values of the strings you are constructing the
    command line are not fully trustworthy, they can be easily manipulated
    to make your program execute arbitrary shell commands.

    --
    __ Erik Max Francis && && http://www.alcyone.com/max/
    / \ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
    \__/ In the fight between you and the world, back the world.
    -- Frank Zappa
     
    Erik Max Francis, Jan 12, 2004
    #2
    1. Advertising

  3. Olaf Meyer

    Peter Otten Guest

    Olaf Meyer wrote:

    > Sometimes if find it clumsy unsing the following approach building
    > strings:
    >
    > cmd = "%s -start %s -end %s -dir %s" % (executable, startTime, endTime,
    > directory)
    >
    > Especially if you have a lot of variable input it makes it hard to match
    > the variables to the proper fields. From other scripting languanges I'm
    > used to something like:
    >
    > $cmd = "$executable -start $startTime -end $endTime -dir $directory"
    >
    > This makes it very easy to see how the string is actually built. You
    > dont't have to worry where which variables go.
    >
    > Is there a similar way to do this in python?


    >>> "from %(org)s to %(dest)s" % dict(org="X", dest="Y")

    'from X to Y'

    or even

    >>> org = "A"
    >>> dest = "B"
    >>> "from %(org)s to %(dest)s" % locals()

    'from A to B'

    Peter
     
    Peter Otten, Jan 12, 2004
    #3
  4. Olaf Meyer

    Olaf Meyer Guest

    Erik Max Francis wrote:

    > Olaf Meyer wrote:
    >
    >
    >>Especially if you have a lot of variable input it makes it hard to
    >>match
    >>the variables to the proper fields. From other scripting languanges
    >>I'm
    >>used to something like:
    >>
    >> $cmd = "$executable -start $startTime -end $endTime -dir $directory"
    >>
    >>This makes it very easy to see how the string is actually built. You
    >>dont't have to worry where which variables go.
    >>
    >>Is there a similar way to do this in python?

    >
    >
    > Sure:
    >
    > cmd = "%(executable)s -start %(startTime)s -end %(endTime)s -dir
    > %(directory)s" % locals()
    >
    > There are also more expansive solutions such as YAPTU or EmPy.
    >
    > Note, however, that what you are trying to do (presuming you're passing
    > this to os.system or something similar) is potentially a serious
    > security risk. If the values of the strings you are constructing the
    > command line are not fully trustworthy, they can be easily manipulated
    > to make your program execute arbitrary shell commands.
    >


    Erik,

    thanks for your solution suggestion and pointing out the security risks.
    However security is not an issue in my case ;-)

    Olaf
     
    Olaf Meyer, Jan 12, 2004
    #4
  5. At some point, Erik Max Francis <> wrote:

    > Olaf Meyer wrote:
    >
    >> Especially if you have a lot of variable input it makes it hard to
    >> match
    >> the variables to the proper fields. From other scripting languanges
    >> I'm
    >> used to something like:
    >>
    >> $cmd = "$executable -start $startTime -end $endTime -dir $directory"
    >>
    >> This makes it very easy to see how the string is actually built. You
    >> dont't have to worry where which variables go.
    >>
    >> Is there a similar way to do this in python?

    >
    > Sure:
    >
    > cmd = "%(executable)s -start %(startTime)s -end %(endTime)s -dir
    > %(directory)s" % locals()
    >
    > There are also more expansive solutions such as YAPTU or EmPy.
    >
    > Note, however, that what you are trying to do (presuming you're passing
    > this to os.system or something similar) is potentially a serious
    > security risk. If the values of the strings you are constructing the
    > command line are not fully trustworthy, they can be easily manipulated
    > to make your program execute arbitrary shell commands.


    In which case he's probably better off with his original format (almost):

    cmd = '"$executable" -start "$startTime" -end "$endTime" -dir "$directory"'
    os.environ['executable'] = 'blah'
    os.environ['startTime'] = '12'
    os.environ['endTime'] = '18'
    os.environ['directory'] = './'
    os.system(cmd)

    This way, the shell handles all the quoting. You can do
    del os.environ['executable']
    afterwards to clean up. I got this technique from
    http://freshmeat.net/articles/view/337/

    For the quoting, compare:
    >>> os.environ['string'] = "`uname` $TERM"
    >>> os.system('echo "$string"')

    `uname` $PATH
    (this is what we want: don't run arbitrary commands or expand
    environment variables given in a user string)

    with
    >>> string = "`uname` $TERM"
    >>> os.system('echo "%s"' % string)

    Linux xterm
    (whoops, security leak)

    --
    |>|\/|<
    /--------------------------------------------------------------------------\
    |David M. Cooke
    |cookedm(at)physics(dot)mcmaster(dot)ca
     
    David M. Cooke, Jan 12, 2004
    #5
  6. Olaf Meyer

    Olaf Meyer Guest

    Erik Max Francis wrote:

    > Olaf Meyer wrote:
    >
    >
    >>Especially if you have a lot of variable input it makes it hard to
    >>match
    >>the variables to the proper fields. From other scripting languanges
    >>I'm
    >>used to something like:
    >>
    >> $cmd = "$executable -start $startTime -end $endTime -dir $directory"
    >>
    >>This makes it very easy to see how the string is actually built. You
    >>dont't have to worry where which variables go.
    >>
    >>Is there a similar way to do this in python?

    >
    >
    > Sure:
    >
    > cmd = "%(executable)s -start %(startTime)s -end %(endTime)s -dir
    > %(directory)s" % locals()
    >
    > There are also more expansive solutions such as YAPTU or EmPy.
    >
    > Note, however, that what you are trying to do (presuming you're passing
    > this to os.system or something similar) is potentially a serious
    > security risk. If the values of the strings you are constructing the
    > command line are not fully trustworthy, they can be easily manipulated
    > to make your program execute arbitrary shell commands.
    >


    I just found out another way ;-) Using the locals() has the disadvantage
    that I cannot use more complex variable parameters (e.g. certain values
    of a dictionary). The following works well:

    cmd = (executable + " -start " + startTime + " -end " + endTime +
    " -dir " + options.dir)

    Olaf
     
    Olaf Meyer, Jan 12, 2004
    #6
  7. "David M. Cooke" wrote:

    > In which case he's probably better off with his original format
    > (almost):
    >
    > cmd = '"$executable" -start "$startTime" -end "$endTime" -dir \
    > "$directory"'
    > os.environ['executable'] = 'blah'
    > os.environ['startTime'] = '12'
    > os.environ['endTime'] = '18'
    > os.environ['directory'] = './'
    > os.system(cmd)


    This doesn't resolve the underlying possibility for mailicious people in
    control of the contents of those variables to get it to execute
    arbitrary shell code. (In his case he says it isn't an issue, but
    still.)

    --
    __ Erik Max Francis && && http://www.alcyone.com/max/
    / \ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
    \__/ It was involuntary. They sank my boat.
    -- John F. Kennedy (on how he became a war hero)
     
    Erik Max Francis, Jan 12, 2004
    #7
  8. At some point, Erik Max Francis <> wrote:

    > "David M. Cooke" wrote:
    >
    >> In which case he's probably better off with his original format
    >> (almost):
    >>
    >> cmd = '"$executable" -start "$startTime" -end "$endTime" -dir \
    >> "$directory"'
    >> os.environ['executable'] = 'blah'
    >> os.environ['startTime'] = '12'
    >> os.environ['endTime'] = '18'
    >> os.environ['directory'] = './'
    >> os.system(cmd)

    >
    > This doesn't resolve the underlying possibility for mailicious people in
    > control of the contents of those variables to get it to execute
    > arbitrary shell code. (In his case he says it isn't an issue, but
    > still.)


    Do you mean something like
    os.environ['startTime'] = '`rm -rf /`'
    ?
    That 'rm -rf /' *won't* be executed: the shell will expand
    "$startTime" to "`rm -rf /`", and that's it. Of course, if the
    executable you're calling is a shell script that doesn't handle it's
    arguments correctly, then you're in trouble. That means $executable is
    bad practice -- you're allowing arbitrary commands to be called.

    --
    |>|\/|<
    /--------------------------------------------------------------------------\
    |David M. Cooke
    |cookedm(at)physics(dot)mcmaster(dot)ca
     
    David M. Cooke, Jan 13, 2004
    #8
  9. "David M. Cooke" wrote:

    > Do you mean something like
    > os.environ['startTime'] = '`rm -rf /`'
    > ?


    No, I mean something like

    os.environ['startTime'] = '"; rm -rf /; : "'

    The lesson to be learned here is: Do not build shell commands from
    untrusted inputs. Ever.

    --
    __ Erik Max Francis && && http://www.alcyone.com/max/
    / \ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
    \__/ You are free and that is why you are lost.
    -- Franz Kafka
     
    Erik Max Francis, Jan 13, 2004
    #9
  10. At some point, Erik Max Francis <> wrote:

    > "David M. Cooke" wrote:
    >
    >> Do you mean something like
    >> os.environ['startTime'] = '`rm -rf /`'
    >> ?

    >
    > No, I mean something like
    >
    > os.environ['startTime'] = '"; rm -rf /; : "'
    >
    > The lesson to be learned here is: Do not build shell commands from
    > untrusted inputs. Ever.


    Doesn't work:
    >>> os.environ['string'] = '"; uname; : "'
    >>> os.system('echo "$string"')

    "; uname; : "

    Although the advice of not building shell commands is still prudent;
    just because none of mine or your methods to defeat haven't worked,
    doesn't mean there isn't a technique that will.

    It's also dependent on having a good shell -- I'm using bash 2.05b.0.

    --
    |>|\/|<
    /--------------------------------------------------------------------------\
    |David M. Cooke
    |cookedm(at)physics(dot)mcmaster(dot)ca
     
    David M. Cooke, Jan 13, 2004
    #10
  11. Olaf Meyer

    Tim Roberts Guest

    Olaf Meyer <> wrote:
    >
    >I just found out another way ;-) Using the locals() has the disadvantage
    >that I cannot use more complex variable parameters (e.g. certain values
    >of a dictionary). The following works well:
    >
    >cmd = (executable + " -start " + startTime + " -end " + endTime +
    > " -dir " + options.dir)


    Yes, that works, but you should bear in mind that it is slower than the %s
    option. The "+" operations are all separate interpreter steps, while the
    "%" operation is done in C.

    At least, it was that way when I asked this same question several years
    ago. If it has changed, I'm sure someone will point out my error.

    Sometimes, it can make sense to write it this way:

    cmd = ' '.join((
    executable,
    "-start", startTime,
    "-end", endTime,
    "-dir", options.dir
    ))
    --
    - Tim Roberts,
    Providenza & Boekelheide, Inc.
     
    Tim Roberts, Jan 14, 2004
    #11
  12. Olaf Meyer

    Paul McGuire Guest

    "Tim Roberts" <> wrote in message
    news:...
    > Olaf Meyer <> wrote:
    > >
    > >I just found out another way ;-) Using the locals() has the disadvantage
    > >that I cannot use more complex variable parameters (e.g. certain values
    > >of a dictionary). The following works well:
    > >
    > >cmd = (executable + " -start " + startTime + " -end " + endTime +
    > > " -dir " + options.dir)

    >
    > Yes, that works, but you should bear in mind that it is slower than the %s
    > option. The "+" operations are all separate interpreter steps, while the
    > "%" operation is done in C.
    >


    On the relative time scales of concatenating 7 strings compared to forking
    off a separate process (which I presume is what is to be done with cmd), I'd
    go for the more readable representation, to aid in long term
    maintainability.

    If I have some string concatenation being done in a highly repetitive part
    of code, then by all means, replace it with one of the half dozen documented
    optimized alternatives. But if I build a string in order to create a
    sub-process, or invoke a database query, or make a remote CORBA invocation,
    etc., then these "optimizations" don't really save much time, and instead
    distract me/reviewers/testers/maintainers from the important program logic.

    -- Paul
     
    Paul McGuire, Jan 14, 2004
    #12
  13. In article <pduMb.6321$>, Olaf Meyer wrote:
    > Sometimes if find it clumsy unsing the following approach building strings:
    >
    > cmd = "%s -start %s -end %s -dir %s" % (executable, startTime, endTime,
    > directory)
    >
    > Especially if you have a lot of variable input it makes it hard to match
    > the variables to the proper fields. From other scripting languanges I'm
    > used to something like:
    >
    > $cmd = "$executable -start $startTime -end $endTime -dir $directory"
    >
    > This makes it very easy to see how the string is actually built. You
    > dont't have to worry where which variables go.
    >
    > Is there a similar way to do this in python?


    Go here:
    http://lfw.org/python/

    Look under "string interpolation for Python".

    Examples supported:

    "Here is a $string."
    "Here is a $module.member."
    "Here is an $object.member."
    "Here is a $functioncall(with, arguments)."
    "Here is an ${arbitrary + expression}."
    "Here is an $array[3] member."
    "Here is a $dictionary['member']."

    Thanks to Ka-Ping Yee! I've succesfully used this to build a homebrew
    templating language. It's nice and lightweight.

    --
    ..:[ dave benjamin (ramenboy) -:- www.ramenfest.com -:- www.3dex.com ]:.
    : d r i n k i n g l i f e o u t o f t h e c o n t a i n e r :
     
    Dave Benjamin, Jan 17, 2004
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Gal Diskin

    building strings from variables

    Gal Diskin, Oct 5, 2006, in forum: Python
    Replies:
    3
    Views:
    336
    Gal Diskin
    Oct 7, 2006
  2. wesley chun

    Re: building strings from variables

    wesley chun, Oct 6, 2006, in forum: Python
    Replies:
    4
    Views:
    262
    Gal Diskin
    Oct 7, 2006
  3. Ben

    Strings, Strings and Damned Strings

    Ben, Jun 22, 2006, in forum: C Programming
    Replies:
    14
    Views:
    814
    Malcolm
    Jun 24, 2006
  4. Midex
    Replies:
    24
    Views:
    974
  5. Chris Carlen
    Replies:
    1
    Views:
    641
    Gabriel Genellina
    Sep 18, 2007
Loading...

Share This Page