Easy "here documents" ??

[Jim Hill]

I've done some Googling around on this and it seems like creating a here
document is a bit tricky with Python. Trivial via triple-quoted strings
if there's no need for variable interpolation but requiring a long, long
formatted arglist via (%s,%s,%s,ad infinitum) if there is. So my
question is:

Is there a way to produce a very long multiline string of output with
variables' values inserted without having to resort to this wacky

"""v = %s"""%(variable)

business?

Thanks,


Jim, Python no0b

 

[Peter Hansen]

I've done some Googling around on this and it seems like creating a here
document is a bit tricky with Python. Trivial via triple-quoted strings
if there's no need for variable interpolation but requiring a long, long
formatted arglist via (%s,%s,%s,ad infinitum) if there is. So my
question is:

Is there a way to produce a very long multiline string of output with
variables' values inserted without having to resort to this wacky

"""v = %s"""%(variable)

business?

I have no idea what a "here document" is, but there are several
alternatives to the "wacky" basic substitution with a tuple of
values.

The simplest uses a mapping type:

mydict = {'namedVal': 666}
'''v = %(namedVal)s''' % mydict

Does that let you build whatever a "here document" is?

 

[M.E.Farmer]

I was curious so I googled , looks like a unix thing

http://www.faqs.org/docs/abs/HTML/here-docs.html

Ok I am with Peter on this , still clueless.
What about string replacement.

py> x = """ Hello me name is ~NAME~. \n I am ~AGE~ years old.\n
.... I live in ~CITY~.\n The city of ~CITY~ is nice.\n
.... I have live here for ~AGE~ years.\n
.... """
py> x = x.replace('~AGE~', '12')
py> x = x.replace('~NAME~', 'Jimmy')
py> x = x.replace('~CITY~', 'Oma')

It makes your template cleaner cause you can use what you want
for the tokens, but takes more lines to do the replacements.
still clueless,
M.E.Farmer

 

[vincent wehren]

I have no idea what a "here document" is, but there are several
alternatives to the "wacky" basic substitution with a tuple of
values.

OP is looking for "heredoc" syntax; in, let's say, PHP
this lets you do something like:

$foo = new foo();
$name = 'MyName';

echo <<<EOT
My name is "$name". I am printing some $foo->foo.
Now, I am printing some {$foo->bar[1]}.
This should print a capital 'A': \x41
EOT;

AFAIK, there is no direct Python equivalent for this kind of syntax.
Using a mapping like you suggested or the string.Template class in
Python 2.4 still maybe improvements over what OP calls that "wacky"
business.

 

[Nick Coghlan]

Is there a way to produce a very long multiline string of output with
variables' values inserted without having to resort to this wacky

"""v = %s"""%(variable)

business?

Try combining Python 2.4's subprocess module with its string Templates.

Cheers,
Nick.

 

[Jerry Sievers]

I've done some Googling around on this and it seems like creating a here
document is a bit tricky with Python. Trivial via triple-quoted strings
if there's no need for variable interpolation but requiring a long, long
formatted arglist via (%s,%s,%s,ad infinitum) if there is. So my
question is:

Is there a way to produce a very long multiline string of output with
variables' values inserted without having to resort to this wacky

"""v = %s"""%(variable)
business?

Hmmmmmm, by using the %(varname)[dsf...] with vars(), locals(),
globals(), someDict, et al, a little messy but not terribly difficult.

It gets uglier though if you want to do this from inside a function
and have variables from more than one scope interpolated. For that
you need something that can treat a series of dicts as one.If there's
built in functionality in Python for this, I haven't discovered it
yet.

To wit;

# feed this thing with one or more dicts, in order of decreasing
#search priority.
class multiDict:
def __init__(self, *dicts):
self.dicts = dicts
def __getitem__(self, key):
for dict in self.dicts:
if dict.has_key(key):
return dict[key]
raise(KeyError)

globalvar = 100

def foo():
localvar = 200

print """
%(globalvar)d
%(localvar)d
""" % multiDict(globals(), locals())

foo()

------------------

Now all else that we need to make this pretty would be macros, a la
cpp m4 or similar

define(`VARS', `multiDict(locals(), globals())')

print "..." % VARS

You get the idea.

 

[Fredrik Lundh]

It gets uglier though if you want to do this from inside a function
and have variables from more than one scope interpolated. For that
you need something that can treat a series of dicts as one.If there's
built in functionality in Python for this, I haven't discovered it
yet.

you use them all the time: plain old instance objects...

here's a rather horrid piece of code that turns a list of dictionaries
into a single object the automatically searches the dictionary chain
when you ask for attributes (use getattr for non-standard keys).

def flatten_dicts(*dicts):
root = None
for dict in dicts:
class wrapper: pass
if root:
wrapper.__bases__ = (root,)
wrapper.__dict__ = dict
root = wrapper
return wrapper()

obj = flatten_dicts(d1, d2, d3)

</F>

 

[Nick Craig-Wood]

I've done some Googling around on this and it seems like creating a here
document is a bit tricky with Python. Trivial via triple-quoted strings
if there's no need for variable interpolation but requiring a long, long
formatted arglist via (%s,%s,%s,ad infinitum) if there is. So my
question is:

Is there a way to produce a very long multiline string of output with
variables' values inserted without having to resort to this wacky

"""v = %s"""%(variable)

I prefer this
... I'll have %(amount)s %(what)s
... for $%(cost)s please""" % locals()
I'll have 1 potato
for $2.0 please
Its almost as neat as perl / shell here documents and emacs parses """
strings properly too ;-)

Note the \ after the triple quote so the first line is flush on the
left, and the locals() statement. You can use globals() or a
dictionary you might have lying around instead which is much more
flexible than perl. You can even pass self.__dict__ if you are in a
class method so you can access attributes.
... I'll have %(amount)s %(what)s
... for $%(cost)s please""" % a.__dict__
I'll have 10 rutabaga
for $17.5 please

 

[Keith Dart]

I've done some Googling around on this and it seems like creating a here
document is a bit tricky with Python. Trivial via triple-quoted strings
if there's no need for variable interpolation but requiring a long, long
formatted arglist via (%s,%s,%s,ad infinitum) if there is. So my
question is:

Is there a way to produce a very long multiline string of output with
variables' values inserted without having to resort to this wacky

I was thinking about this. But I can't think of any reason why you would
want to do this in Python. What's wrong with a regular parameterized
function?

 

[Steve Holden]

Jerry Sievers wrote:




you use them all the time: plain old instance objects...

here's a rather horrid piece of code that turns a list of dictionaries
into a single object the automatically searches the dictionary chain
when you ask for attributes (use getattr for non-standard keys).

def flatten_dicts(*dicts):
root = None
for dict in dicts:
class wrapper: pass
if root:
wrapper.__bases__ = (root,)
wrapper.__dict__ = dict
root = wrapper
return wrapper()

obj = flatten_dicts(d1, d2, d3)

Iterative subclassing, yet. Yerch It's sometimes amazing just how
down and dirty Python will let you get.

Of course, if the mappings were all dictionaries then it would be rather
simpler to just update an empty dict with the outermost through to the
innermost scopes.

though-efficiency-would-be-determined-by-usage-ly y'rs - steve

 

[Fredrik Lundh]

here's a rather horrid piece of code that turns a list of dictionaries
into a single object [that] automatically searches the dictionary chain
when you ask for attributes (use getattr for non-standard keys).

def flatten_dicts(*dicts):
root = None
for dict in dicts:
class wrapper: pass
if root:
wrapper.__bases__ = (root,)
wrapper.__dict__ = dict
root = wrapper
return wrapper()

obj = flatten_dicts(d1, d2, d3)

Iterative subclassing, yet. Yerch It's sometimes amazing just how down and dirty Python will
let you get.

you can have a lot more fun with "class" than with "def"...

Of course, if the mappings were all dictionaries then it would be rather simpler to just update an
empty dict with the outermost through to the innermost scopes.

except that if you do that, changes to the individual dictionaries won't
be visible in the "flattened" view.

</F>

 

[Scott David Daniels]

I prefer this

... I'll have %(amount)s %(what)s
... for $%(cost)s please""" % locals()
I'll have 1 potato
for $2.0 please

And if you enjoy building insecure stuff, try:

def fix(text, globals_=None, locals=None, quote='"'):
d = (globals_ or locals or globals()).copy()
source = text.split(quote)
source[1::2] = (str(eval(expr, d, locals or d))
for expr in source[1::2])
return ''.join(source)


amount = 1
cost = 2.0
what = 'potato'
print fix("""I'll have "amount" "what"s
for "'$%.2f' % cost"s please""", locals())


--Scott David Daniels
(e-mail address removed)

 

[Fredrik Lundh]

And if you enjoy building insecure stuff, try:

def fix(text, globals_=None, locals=None, quote='"'):
d = (globals_ or locals or globals()).copy()
source = text.split(quote)
source[1::2] = (str(eval(expr, d, locals or d))
for expr in source[1::2])
return ''.join(source)


amount = 1
cost = 2.0
what = 'potato'
print fix("""I'll have "amount" "what"s
for "'$%.2f' % cost"s please""", locals())

And if you prefer not to type so much:

def I(*args): return "".join(map(str, args))
def F(v, fmt): return ("%" + fmt) % v

print I("I'll have ", amount, " ", what, "s for $", F(cost, ".2f"), "s please")

</F>

 

[Doug Holton]

Is there a way to produce a very long multiline string of output with
variables' values inserted without having to resort to this wacky

"""v = %s"""%(variable)

No, not without the god-awful hacks you've already seen.

But it is possible in boo: : http://boo.codehaus.org/
See http://boo.codehaus.org/String+Interpolation

variable1 = 1
variable2 = 2

s = """
v = ${variable1}
v2's value is: ${variable2}
"""

print s

 

[Bengt Richter]

you use them all the time: plain old instance objects...

here's a rather horrid piece of code that turns a list of dictionaries
into a single object the automatically searches the dictionary chain
when you ask for attributes (use getattr for non-standard keys).

def flatten_dicts(*dicts):
root = None
for dict in dicts:
class wrapper: pass
if root:
wrapper.__bases__ = (root,)
wrapper.__dict__ = dict
root = wrapper
return wrapper()

obj = flatten_dicts(d1, d2, d3)

That's one I hadn't thought of in any form, but I'm glad to have been
shown the opening (although I am bit surprised at such dark side stuff
from you, even with the "horrid piece of code" qualification ;-)

Seriously, has anyone done a definitive documentation of
all the namespaces (incl name-search paths and r/w rules) of Python?
ISTM the name access games are a bit irregular. The difficulty of creating
a mapping usable like

txt = 'bim bam %(boo)s' % mapping

that will do _exactly_ the same thing as

txt = 'bim bam %s' % (boo,)

no matter whether boo is local, global, or a closure cell is symptomatic, ISTM.
flattendicts(locals(), globals()) is close, but not quite there due to locals()'s
being only a snapshot and leaving out closure cells -- is the latter intentional BTW?
ISTM closure cell variables belong somewhere, and it can't be in globals().

If every local context had a magic __localnamespace__ object so that

assert __localnamespace__.x is x and __localnamespace__['x'] is x

never failed in any local context except with NameError, you could do lots
of things easily. Might not be so easy to implement though. But with __getattr__
and __getitem__ doing the same access,

txt = 'bim bam %(boo)s' % __localnamespace__

would do what one might like. A sys._getframe(depth).f_localnamespace
frame attribute as a way of peeking into various local namespaces via their
__localnamespace__ objects might make name-chasing easier and more regular
there too. Just a thought, not thought out ;-)

Regards,
Bengt Richter

 

[John Roth]

I've done some Googling around on this and it seems like creating a here
document is a bit tricky with Python. Trivial via triple-quoted strings
if there's no need for variable interpolation but requiring a long, long
formatted arglist via (%s,%s,%s,ad infinitum) if there is. So my
question is:

Is there a way to produce a very long multiline string of output with
variables' values inserted without having to resort to this wacky

"""v = %s"""%(variable)

business?

The major issue here is that both ways of text substitution
in Python are based on the C and C++ printf statement.
Here documents are based on a much less disciplined
use of variables; they use the local variables on the
calling stack.

It would certainly be possible to write a module that
substitutes the values of variables in the calling stack.
Look at the inspect module in the Python Runtime
Services section of the Python Library for the basic
facilities you'd need to use to write such a module.

I'm not sure why you'd want to do this, though.
It seems like it would be mostly useful in a style
of programming that's quite foreign to the way
Python wants to be programmed.

John Roth

 

[Jim Hill]

I prefer this

... I'll have %(amount)s %(what)s
... for $%(cost)s please""" % locals()

Looks pretty slick. This might just be what I need.

Its almost as neat as perl / shell here documents and emacs parses """
strings properly too ;-)

Mmm...emacs...

Thanks for the tip.


Jim

 

[Jim Hill]

And if you enjoy building insecure stuff, try:

def fix(text, globals_=None, locals=None, quote='"'):
d = (globals_ or locals or globals()).copy()
source = text.split(quote)
source[1::2] = (str(eval(expr, d, locals or d))
for expr in source[1::2])
return ''.join(source)

And if you prefer not to type so much:

def I(*args): return "".join(map(str, args))
def F(v, fmt): return ("%" + fmt) % v

Not that I don't appreciate the suggestions from masters of the Python
universe, but the reason I'm switching to Python from Perl is for the
readability. What you fells are suggesting might as well be riddled
with dollar signs and semicolons... <emoticon>.


Jim

 

[Jim Hill]

I was thinking about this. But I can't think of any reason why you would
want to do this in Python. What's wrong with a regular parameterized
function?

I'm trying to write a script that writes a script for a rather specialized
task. I know that seems weird, but the original version was written in
Korn shell and most of my team are familiar with the way it does things
even though they don't read Korn. (The original author has since
decamped for greener pastures.) Since we're trying to standardize all
our scripts on Python I got the task of rewriting. Once I have the
simple port done I'll see if I can't figure out A Better Way.


Jim

 

[Jim Hill]

John Roth wrote:

[Here docs]

I'm not sure why you'd want to do this, though.
It seems like it would be mostly useful in a style
of programming that's quite foreign to the way
Python wants to be programmed.

I'm going to try some of the suggestions that others have floated but I
think you've really tumbled to the core of my situation: Trying to take
a Korn shell script and convert it as closely to line-for-line into
Python as possible is Just Dumb.


Jim