what difference does redirection make?

[Nikola Skoric]

When I execute
nick@rilmir:~/code/simplepyged/docs/examples$ python latex.py
I get expected output (bunch of latex markup).

But, when I add a redirection, I get:
nick@rilmir:~/code/simplepyged/docs/examples$ python latex.py > foo.tex
File "latex.py", line 87, in <module>
print mytemplate.render_unicode(stack=stack, index=latex_index(stack), pages=pages)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 625: ordinal not in range(128)

Now, how does python even know I added a redirection?! And then... why
would it care?

Code of latex.py can be seen here:
http://github.com/dijxtra/simplepyg...20b502e5af71d78ed55bf0/docs/examples/latex.py

 

[Benjamin Kaplan]

When I execute
nick@rilmir:~/code/simplepyged/docs/examples$ python latex.py
I get expected output (bunch of latex markup).

But, when I add a redirection, I get:
nick@rilmir:~/code/simplepyged/docs/examples$ python latex.py > foo.tex
 File "latex.py", line 87, in <module>
   print  mytemplate.render_unicode(stack=stack, index=latex_index(stack), pages=pages)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 625: ordinal not in range(128)

Now, how does python even know I added a redirection?! And then... why
would it care?

Code of latex.py can be seen here:
http://github.com/dijxtra/simplepyg...20b502e5af71d78ed55bf0/docs/examples/latex.py

You're trying to write Unicode to a file. But there's no such thing as
a Unicode file- everything has to be stored on disk as a sequence of
bytes. So Python needs to encode the unicode string using a sequence
of bytes. When stdout points to a terminal (you can check the source
code to see how it figures that out), it can use some environment
variables to figure out what your terminal's encoding is. Your file
however does not have anything stating what encoding you expect it to
be in. So Python refuses to make a guess and just defaults to the
lowest common denominator- ASCII. Since you have a byte that can't be
encoded as ASCII, Python gives up and throws that error.

To avoid this, you'll need to explicitly set the encoding of what you
want to print out. For instance, add .encode("utf-8") or
..encode("cp1252") or whatever encoding you want to use to the end of
the mytemplate.render_unicode(...) call on the last line.

 

[Nikola Skoric]

Dana Sun, 17 Oct 2010 15:36:13 -0400,

You're trying to write Unicode to a file. But there's no such thing as /snip
.encode("cp1252") or whatever encoding you want to use to the end of
the mytemplate.render_unicode(...) call on the last line.

Thank you, thank you, thank you! I got it now...