Easy "here documents" ??

Discussion in 'Python' started by Jim Hill, Dec 19, 2004.

  1. Jim Hill

    Jim Hill Guest

    I've done some Googling around on this and it seems like creating a here
    document is a bit tricky with Python. Trivial via triple-quoted strings
    if there's no need for variable interpolation but requiring a long, long
    formatted arglist via (%s,%s,%s,ad infinitum) if there is. So my
    question is:

    Is there a way to produce a very long multiline string of output with
    variables' values inserted without having to resort to this wacky

    """v = %s"""%(variable)

    business?

    Thanks,


    Jim, Python no0b
    --
    "I regard NASCAR the same way I regard gay porn: I know it exists and I
    know some guys like it; I just don't want to see it." -- Z. B. Goode
     
    Jim Hill, Dec 19, 2004
    #1
    1. Advertising

  2. Jim Hill

    Peter Hansen Guest

    Jim Hill wrote:
    > I've done some Googling around on this and it seems like creating a here
    > document is a bit tricky with Python. Trivial via triple-quoted strings
    > if there's no need for variable interpolation but requiring a long, long
    > formatted arglist via (%s,%s,%s,ad infinitum) if there is. So my
    > question is:
    >
    > Is there a way to produce a very long multiline string of output with
    > variables' values inserted without having to resort to this wacky
    >
    > """v = %s"""%(variable)
    >
    > business?


    I have no idea what a "here document" is, but there are several
    alternatives to the "wacky" basic substitution with a tuple of
    values.

    The simplest uses a mapping type:

    mydict = {'namedVal': 666}
    '''v = %(namedVal)s''' % mydict

    Does that let you build whatever a "here document" is?
     
    Peter Hansen, Dec 19, 2004
    #2
    1. Advertising

  3. Jim Hill

    M.E.Farmer Guest

    I was curious so I googled , looks like a unix thing :)

    http://www.faqs.org/docs/abs/HTML/here-docs.html

    Ok I am with Peter on this , still clueless.
    What about string replacement.

    py> x = """ Hello me name is ~NAME~. \n I am ~AGE~ years old.\n
    .... I live in ~CITY~.\n The city of ~CITY~ is nice.\n
    .... I have live here for ~AGE~ years.\n
    .... """
    py> x = x.replace('~AGE~', '12')
    py> x = x.replace('~NAME~', 'Jimmy')
    py> x = x.replace('~CITY~', 'Oma')

    It makes your template cleaner cause you can use what you want
    for the tokens, but takes more lines to do the replacements.
    still clueless,
    M.E.Farmer


    Peter Hansen wrote:
    > Jim Hill wrote:
    > > I've done some Googling around on this and it seems like creating a

    here
    > > document is a bit tricky with Python. Trivial via triple-quoted

    strings
    > > if there's no need for variable interpolation but requiring a long,

    long
    > > formatted arglist via (%s,%s,%s,ad infinitum) if there is. So my
    > > question is:
    > >
    > > Is there a way to produce a very long multiline string of output

    with
    > > variables' values inserted without having to resort to this wacky
    > >
    > > """v = %s"""%(variable)
    > >
    > > business?

    >
    > I have no idea what a "here document" is, but there are several
    > alternatives to the "wacky" basic substitution with a tuple of
    > values.
    >
    > The simplest uses a mapping type:
    >
    > mydict = {'namedVal': 666}
    > '''v = %(namedVal)s''' % mydict
    >
    > Does that let you build whatever a "here document" is?
     
    M.E.Farmer, Dec 19, 2004
    #3
  4. Peter Hansen wrote:
    > Jim Hill wrote:
    >
    >> I've done some Googling around on this and it seems like creating a here
    >> document is a bit tricky with Python. Trivial via triple-quoted strings
    >> if there's no need for variable interpolation but requiring a long, long
    >> formatted arglist via (%s,%s,%s,ad infinitum) if there is. So my
    >> question is:
    >>
    >> Is there a way to produce a very long multiline string of output with
    >> variables' values inserted without having to resort to this wacky
    >>
    >> """v = %s"""%(variable)
    >>
    >> business?

    >
    >
    > I have no idea what a "here document" is, but there are several
    > alternatives to the "wacky" basic substitution with a tuple of
    > values.


    OP is looking for "heredoc" syntax; in, let's say, PHP
    this lets you do something like:

    $foo = new foo();
    $name = 'MyName';

    echo <<<EOT
    My name is "$name". I am printing some $foo->foo.
    Now, I am printing some {$foo->bar[1]}.
    This should print a capital 'A': \x41
    EOT;

    AFAIK, there is no direct Python equivalent for this kind of syntax.
    Using a mapping like you suggested or the string.Template class in
    Python 2.4 still maybe improvements over what OP calls that "wacky"
    business.

    --
    Vincent Wehren

    >
    > The simplest uses a mapping type:
    >
    > mydict = {'namedVal': 666}
    > '''v = %(namedVal)s''' % mydict
    >
    > Does that let you build whatever a "here document" is?
     
    vincent wehren, Dec 19, 2004
    #4
  5. Jim Hill

    Nick Coghlan Guest

    Jim Hill wrote:
    > Is there a way to produce a very long multiline string of output with
    > variables' values inserted without having to resort to this wacky
    >
    > """v = %s"""%(variable)
    >
    > business?


    Try combining Python 2.4's subprocess module with its string Templates.

    Cheers,
    Nick.

    --
    Nick Coghlan | | Brisbane, Australia
    ---------------------------------------------------------------
    http://boredomandlaziness.skystorm.net
     
    Nick Coghlan, Dec 19, 2004
    #5
  6. (Jim Hill) writes:

    > I've done some Googling around on this and it seems like creating a here
    > document is a bit tricky with Python. Trivial via triple-quoted strings
    > if there's no need for variable interpolation but requiring a long, long
    > formatted arglist via (%s,%s,%s,ad infinitum) if there is. So my
    > question is:
    >
    > Is there a way to produce a very long multiline string of output with
    > variables' values inserted without having to resort to this wacky
    >
    > """v = %s"""%(variable)
    > business?


    Hmmmmmm, by using the %(varname)[dsf...] with vars(), locals(),
    globals(), someDict, et al, a little messy but not terribly difficult.

    It gets uglier though if you want to do this from inside a function
    and have variables from more than one scope interpolated. For that
    you need something that can treat a series of dicts as one.If there's
    built in functionality in Python for this, I haven't discovered it
    yet.

    To wit;

    # feed this thing with one or more dicts, in order of decreasing
    #search priority.
    class multiDict:
    def __init__(self, *dicts):
    self.dicts = dicts
    def __getitem__(self, key):
    for dict in self.dicts:
    if dict.has_key(key):
    return dict[key]
    raise(KeyError)

    globalvar = 100

    def foo():
    localvar = 200

    print """
    %(globalvar)d
    %(localvar)d
    """ % multiDict(globals(), locals())

    foo()

    ------------------

    Now all else that we need to make this pretty would be macros, a la
    cpp m4 or similar

    define(`VARS', `multiDict(locals(), globals())')

    print "..." % VARS

    You get the idea.


    --
    -------------------------------------------------------------------------------
    Jerry Sievers 305 854-3001 (home) WWW ECommerce Consultant
    305 321-1144 (mobile http://www.JerrySievers.com/
     
    Jerry Sievers, Dec 19, 2004
    #6
  7. Jerry Sievers wrote:

    > It gets uglier though if you want to do this from inside a function
    > and have variables from more than one scope interpolated. For that
    > you need something that can treat a series of dicts as one.If there's
    > built in functionality in Python for this, I haven't discovered it
    > yet.


    you use them all the time: plain old instance objects...

    here's a rather horrid piece of code that turns a list of dictionaries
    into a single object the automatically searches the dictionary chain
    when you ask for attributes (use getattr for non-standard keys).

    def flatten_dicts(*dicts):
    root = None
    for dict in dicts:
    class wrapper: pass
    if root:
    wrapper.__bases__ = (root,)
    wrapper.__dict__ = dict
    root = wrapper
    return wrapper()

    obj = flatten_dicts(d1, d2, d3)

    </F>
     
    Fredrik Lundh, Dec 19, 2004
    #7
  8. Jim Hill <> wrote:
    > I've done some Googling around on this and it seems like creating a here
    > document is a bit tricky with Python. Trivial via triple-quoted strings
    > if there's no need for variable interpolation but requiring a long, long
    > formatted arglist via (%s,%s,%s,ad infinitum) if there is. So my
    > question is:
    >
    > Is there a way to produce a very long multiline string of output with
    > variables' values inserted without having to resort to this wacky
    >
    > """v = %s"""%(variable)


    I prefer this

    >>> amount = 1
    >>> cost = 2.0
    >>> what = 'potato'
    >>> print """\

    ... I'll have %(amount)s %(what)s
    ... for $%(cost)s please""" % locals()
    I'll have 1 potato
    for $2.0 please
    >>>


    Its almost as neat as perl / shell here documents and emacs parses """
    strings properly too ;-)

    Note the \ after the triple quote so the first line is flush on the
    left, and the locals() statement. You can use globals() or a
    dictionary you might have lying around instead which is much more
    flexible than perl. You can even pass self.__dict__ if you are in a
    class method so you can access attributes.

    >>> class A: pass

    ...
    >>> a=A()
    >>> a.amount=10
    >>> a.what="rutabaga"
    >>> a.cost=17.5
    >>> print """\

    ... I'll have %(amount)s %(what)s
    ... for $%(cost)s please""" % a.__dict__
    I'll have 10 rutabaga
    for $17.5 please
    >>>


    --
    Nick Craig-Wood <> -- http://www.craig-wood.com/nick
     
    Nick Craig-Wood, Dec 19, 2004
    #8
  9. Jim Hill

    Keith Dart Guest

    Jim Hill wrote:
    > I've done some Googling around on this and it seems like creating a here
    > document is a bit tricky with Python. Trivial via triple-quoted strings
    > if there's no need for variable interpolation but requiring a long, long
    > formatted arglist via (%s,%s,%s,ad infinitum) if there is. So my
    > question is:
    >
    > Is there a way to produce a very long multiline string of output with
    > variables' values inserted without having to resort to this wacky


    I was thinking about this. But I can't think of any reason why you would
    want to do this in Python. What's wrong with a regular parameterized
    function?


    --
    -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Keith Dart <>
    public key: ID: F3D288E4
    =====================================================================
     
    Keith Dart, Dec 19, 2004
    #9
  10. Jim Hill

    Steve Holden Guest

    Fredrik Lundh wrote:

    > Jerry Sievers wrote:
    >
    >
    >>It gets uglier though if you want to do this from inside a function
    >>and have variables from more than one scope interpolated. For that
    >>you need something that can treat a series of dicts as one.If there's
    >>built in functionality in Python for this, I haven't discovered it
    >>yet.

    >
    >
    > you use them all the time: plain old instance objects...
    >
    > here's a rather horrid piece of code that turns a list of dictionaries
    > into a single object the automatically searches the dictionary chain
    > when you ask for attributes (use getattr for non-standard keys).
    >
    > def flatten_dicts(*dicts):
    > root = None
    > for dict in dicts:
    > class wrapper: pass
    > if root:
    > wrapper.__bases__ = (root,)
    > wrapper.__dict__ = dict
    > root = wrapper
    > return wrapper()
    >
    > obj = flatten_dicts(d1, d2, d3)
    >

    Iterative subclassing, yet. Yerch :) It's sometimes amazing just how
    down and dirty Python will let you get.

    Of course, if the mappings were all dictionaries then it would be rather
    simpler to just update an empty dict with the outermost through to the
    innermost scopes.

    though-efficiency-would-be-determined-by-usage-ly y'rs - steve
    --
    Steve Holden http://www.holdenweb.com/
    Python Web Programming http://pydish.holdenweb.com/
    Holden Web LLC +1 703 861 4237 +1 800 494 3119
     
    Steve Holden, Dec 19, 2004
    #10
  11. Steve Holden wrote:

    >> here's a rather horrid piece of code that turns a list of dictionaries
    >> into a single object [that] automatically searches the dictionary chain
    >> when you ask for attributes (use getattr for non-standard keys).
    >>
    >> def flatten_dicts(*dicts):
    >> root = None
    >> for dict in dicts:
    >> class wrapper: pass
    >> if root:
    >> wrapper.__bases__ = (root,)
    >> wrapper.__dict__ = dict
    >> root = wrapper
    >> return wrapper()
    >>
    >> obj = flatten_dicts(d1, d2, d3)
    >>

    > Iterative subclassing, yet. Yerch :) It's sometimes amazing just how down and dirty Python will
    > let you get.


    you can have a lot more fun with "class" than with "def"...

    > Of course, if the mappings were all dictionaries then it would be rather simpler to just update an
    > empty dict with the outermost through to the innermost scopes.


    except that if you do that, changes to the individual dictionaries won't
    be visible in the "flattened" view.

    </F>
     
    Fredrik Lundh, Dec 19, 2004
    #11
  12. Nick Craig-Wood wrote:
    > I prefer this
    >
    > >>> amount = 1
    > >>> cost = 2.0
    > >>> what = 'potato'
    > >>> print """\

    > ... I'll have %(amount)s %(what)s
    > ... for $%(cost)s please""" % locals()
    > I'll have 1 potato
    > for $2.0 please
    > >>>


    And if you enjoy building insecure stuff, try:

    def fix(text, globals_=None, locals=None, quote='"'):
    d = (globals_ or locals or globals()).copy()
    source = text.split(quote)
    source[1::2] = (str(eval(expr, d, locals or d))
    for expr in source[1::2])
    return ''.join(source)


    amount = 1
    cost = 2.0
    what = 'potato'
    print fix("""I'll have "amount" "what"s
    for "'$%.2f' % cost"s please""", locals())


    --Scott David Daniels
     
    Scott David Daniels, Dec 19, 2004
    #12
  13. Scott David Daniels wrote:

    > And if you enjoy building insecure stuff, try:
    >
    > def fix(text, globals_=None, locals=None, quote='"'):
    > d = (globals_ or locals or globals()).copy()
    > source = text.split(quote)
    > source[1::2] = (str(eval(expr, d, locals or d))
    > for expr in source[1::2])
    > return ''.join(source)
    >
    >
    > amount = 1
    > cost = 2.0
    > what = 'potato'
    > print fix("""I'll have "amount" "what"s
    > for "'$%.2f' % cost"s please""", locals())


    And if you prefer not to type so much:

    def I(*args): return "".join(map(str, args))
    def F(v, fmt): return ("%" + fmt) % v

    print I("I'll have ", amount, " ", what, "s for $", F(cost, ".2f"), "s please")

    </F>
     
    Fredrik Lundh, Dec 19, 2004
    #13
  14. Jim Hill

    Doug Holton Guest

    Jim Hill wrote:

    > Is there a way to produce a very long multiline string of output with
    > variables' values inserted without having to resort to this wacky
    >
    > """v = %s"""%(variable)
    >


    No, not without the god-awful hacks you've already seen.

    But it is possible in boo: : http://boo.codehaus.org/
    See http://boo.codehaus.org/String Interpolation

    variable1 = 1
    variable2 = 2

    s = """
    v = ${variable1}
    v2's value is: ${variable2}
    """

    print s
     
    Doug Holton, Dec 20, 2004
    #14
  15. On Sun, 19 Dec 2004 16:46:34 +0100, "Fredrik Lundh" <> wrote:

    >Jerry Sievers wrote:
    >
    >> It gets uglier though if you want to do this from inside a function
    >> and have variables from more than one scope interpolated. For that
    >> you need something that can treat a series of dicts as one.If there's
    >> built in functionality in Python for this, I haven't discovered it
    >> yet.

    >
    >you use them all the time: plain old instance objects...
    >
    >here's a rather horrid piece of code that turns a list of dictionaries
    >into a single object the automatically searches the dictionary chain
    >when you ask for attributes (use getattr for non-standard keys).
    >
    >def flatten_dicts(*dicts):
    > root = None
    > for dict in dicts:
    > class wrapper: pass
    > if root:
    > wrapper.__bases__ = (root,)
    > wrapper.__dict__ = dict
    > root = wrapper
    > return wrapper()
    >
    >obj = flatten_dicts(d1, d2, d3)
    >

    That's one I hadn't thought of in any form, but I'm glad to have been
    shown the opening (although I am bit surprised at such dark side stuff
    from you, even with the "horrid piece of code" qualification ;-)

    Seriously, has anyone done a definitive documentation of
    all the namespaces (incl name-search paths and r/w rules) of Python?
    ISTM the name access games are a bit irregular. The difficulty of creating
    a mapping usable like

    txt = 'bim bam %(boo)s' % mapping

    that will do _exactly_ the same thing as

    txt = 'bim bam %s' % (boo,)

    no matter whether boo is local, global, or a closure cell is symptomatic, ISTM.
    flattendicts(locals(), globals()) is close, but not quite there due to locals()'s
    being only a snapshot and leaving out closure cells -- is the latter intentional BTW?
    ISTM closure cell variables belong somewhere, and it can't be in globals().

    If every local context had a magic __localnamespace__ object so that

    assert __localnamespace__.x is x and __localnamespace__['x'] is x

    never failed in any local context except with NameError, you could do lots
    of things easily. Might not be so easy to implement though. But with __getattr__
    and __getitem__ doing the same access,

    txt = 'bim bam %(boo)s' % __localnamespace__

    would do what one might like. A sys._getframe(depth).f_localnamespace
    frame attribute as a way of peeking into various local namespaces via their
    __localnamespace__ objects might make name-chasing easier and more regular
    there too. Just a thought, not thought out ;-)

    Regards,
    Bengt Richter
     
    Bengt Richter, Dec 20, 2004
    #15
  16. Jim Hill

    John Roth Guest

    "Jim Hill" <> wrote in message
    news:cq2pgr$irj$...
    > I've done some Googling around on this and it seems like creating a here
    > document is a bit tricky with Python. Trivial via triple-quoted strings
    > if there's no need for variable interpolation but requiring a long, long
    > formatted arglist via (%s,%s,%s,ad infinitum) if there is. So my
    > question is:
    >
    > Is there a way to produce a very long multiline string of output with
    > variables' values inserted without having to resort to this wacky
    >
    > """v = %s"""%(variable)
    >
    > business?


    The major issue here is that both ways of text substitution
    in Python are based on the C and C++ printf statement.
    Here documents are based on a much less disciplined
    use of variables; they use the local variables on the
    calling stack.

    It would certainly be possible to write a module that
    substitutes the values of variables in the calling stack.
    Look at the inspect module in the Python Runtime
    Services section of the Python Library for the basic
    facilities you'd need to use to write such a module.

    I'm not sure why you'd want to do this, though.
    It seems like it would be mostly useful in a style
    of programming that's quite foreign to the way
    Python wants to be programmed.

    John Roth


    >
    > Thanks,
    >
    >
    > Jim, Python no0b
     
    John Roth, Dec 20, 2004
    #16
  17. Jim Hill

    Jim Hill Guest

    Nick Craig-Wood wrote:

    >Jim Hill <> wrote:
    >> Is there a way to produce a very long multiline string of output with
    >> variables' values inserted without having to resort to this wacky
    >> """v = %s"""%(variable)


    >I prefer this
    >
    > ... I'll have %(amount)s %(what)s
    > ... for $%(cost)s please""" % locals()


    Looks pretty slick. This might just be what I need.

    >Its almost as neat as perl / shell here documents and emacs parses """
    >strings properly too ;-)


    Mmm...emacs...

    Thanks for the tip.


    Jim
    --
    "I regard NASCAR the same way I regard gay porn: I know it exists and I
    know some guys like it; I just don't want to see it." -- Z. B. Goode
     
    Jim Hill, Dec 20, 2004
    #17
  18. Jim Hill

    Jim Hill Guest

    Fredrik Lundh wrote:
    >Scott David Daniels wrote:
    >
    >> And if you enjoy building insecure stuff, try:
    >>
    >> def fix(text, globals_=None, locals=None, quote='"'):
    >> d = (globals_ or locals or globals()).copy()
    >> source = text.split(quote)
    >> source[1::2] = (str(eval(expr, d, locals or d))
    >> for expr in source[1::2])
    >> return ''.join(source)
    >>

    >And if you prefer not to type so much:
    >
    >def I(*args): return "".join(map(str, args))
    >def F(v, fmt): return ("%" + fmt) % v


    Not that I don't appreciate the suggestions from masters of the Python
    universe, but the reason I'm switching to Python from Perl is for the
    readability. What you fells are suggesting might as well be riddled
    with dollar signs and semicolons... <emoticon>.


    Jim
    --
    "I regard NASCAR the same way I regard gay porn: I know it exists and I
    know some guys like it; I just don't want to see it." -- Z. B. Goode
     
    Jim Hill, Dec 20, 2004
    #18
  19. Jim Hill

    Jim Hill Guest

    Keith Dart wrote:
    >Jim Hill wrote:
    >> Is there a way to produce a very long multiline string of output with
    >> variables' values inserted without having to resort to this wacky

    >
    >I was thinking about this. But I can't think of any reason why you would
    >want to do this in Python. What's wrong with a regular parameterized
    >function?


    I'm trying to write a script that writes a script for a rather specialized
    task. I know that seems weird, but the original version was written in
    Korn shell and most of my team are familiar with the way it does things
    even though they don't read Korn. (The original author has since
    decamped for greener pastures.) Since we're trying to standardize all
    our scripts on Python I got the task of rewriting. Once I have the
    simple port done I'll see if I can't figure out A Better Way.


    Jim
    --
    "I regard NASCAR the same way I regard gay porn: I know it exists and I
    know some guys like it; I just don't want to see it." -- Z. B. Goode
     
    Jim Hill, Dec 20, 2004
    #19
  20. Jim Hill

    Jim Hill Guest

    John Roth wrote:

    [Here docs]
    >I'm not sure why you'd want to do this, though.
    >It seems like it would be mostly useful in a style
    >of programming that's quite foreign to the way
    >Python wants to be programmed.


    I'm going to try some of the suggestions that others have floated but I
    think you've really tumbled to the core of my situation: Trying to take
    a Korn shell script and convert it as closely to line-for-line into
    Python as possible is Just Dumb.


    Jim
    --
    "I regard NASCAR the same way I regard gay porn: I know it exists and I
    know some guys like it; I just don't want to see it." -- Z. B. Goode
     
    Jim Hill, Dec 20, 2004
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. calmar
    Replies:
    6
    Views:
    479
    Nick Craig-Wood
    Nov 15, 2004
  2. Douglas Wells
    Replies:
    8
    Views:
    164
    Nobuyoshi Nakada
    Jan 27, 2007
  3. Roger Pack

    array of here documents

    Roger Pack, Feb 8, 2008, in forum: Ruby
    Replies:
    6
    Views:
    116
    Xavier Noria
    Feb 9, 2008
  4. equinox

    while loop in a here documents

    equinox, Dec 2, 2008, in forum: Ruby
    Replies:
    4
    Views:
    97
    Robert Klemme
    Dec 2, 2008
  5. Replies:
    8
    Views:
    160
    Uri Guttman
    Jan 28, 2007
Loading...

Share This Page