Documentation for "str()" could use some adjustment.

[John Nagle]

The Python documentation for "str" says
"str([object]) :
Return a string containing a nicely printable representation of an object."

However, there's no mention of the fact that "str" of a Unicode string
with non-ASCII characters will raise a conversion exception. The
documentation (and several Python books) seem to indicate that "str" will
produce some "printable representation" for anything, not raise an exception.

I know, it was proposed in PEP 349 to change "str" to return Unicode
strings, and that's coming someday, along with all-Unicode Python,
but meanwhile, the documentation should be clear about this.

John Nagle

 

[Daniel Nogradi]

The Python documentation for "str" says

"str([object]) :
Return a string containing a nicely printable representation of an
object."

However, there's no mention of the fact that "str" of a Unicode string
with non-ASCII characters will raise a conversion exception. The
documentation (and several Python books) seem to indicate that "str" will
produce some "printable representation" for anything, not raise an
exception.

I know, it was proposed in PEP 349 to change "str" to return Unicode
strings, and that's coming someday, along with all-Unicode Python,
but meanwhile, the documentation should be clear about this.

Hi John, I'm not at all an expert around here but my understanding of
the workflow is that bug reports should be submitted to the tracker at
sourceforge. There is no guarantee that they will be noticed there but
the chances there are much higher than for a message on the mailing
list. I just mention this because I've seen your mysqld, urllib and
other potentially important bug reports here with no action taken but
maybe submitting them to sourceforge would attract some developers.

 

[John Nagle]

The Python documentation for "str" says
"str([object]) :
Return a string containing a nicely printable representation of an
object."

However, there's no mention of the fact that "str" of a Unicode string
with non-ASCII characters will raise a conversion exception. The
documentation (and several Python books) seem to indicate that "str" will
produce some "printable representation" for anything, not raise an
exception.

I know, it was proposed in PEP 349 to change "str" to return Unicode
strings, and that's coming someday, along with all-Unicode Python,
but meanwhile, the documentation should be clear about this.

Hi John, I'm not at all an expert around here but my understanding of
the workflow is that bug reports should be submitted to the tracker at
sourceforge. There is no guarantee that they will be noticed there but
the chances there are much higher than for a message on the mailing
list. I just mention this because I've seen your mysqld, urllib and
other potentially important bug reports here with no action taken but
maybe submitting them to sourceforge would attract some developers.

More people read the mailing list than the bug reports, and
Google indexes it, so it puts the problem on record. If this problem
were reported as a bug, there'd be denial that it was a problem, on
the grounds that it will be fixed in a future version of Python.

This was addressed in PEP 349 (status "Deferred"), so it's
a known problem that's being ignored.

Python strings are in transition; in ASCII-only Python, "str"
could never raise a conversion exception, and when we get to
Unicode-only strings, "str" will never raise a conversion exception.
But right now, "str" can definitely raise an exception.

The real problem is the published books on Python:

"Learning Python", by Lutz and Ascher:
"str(string) -- returns the string representation of any object."

"Python in a Nutshell", by Martelli
Doesn't really address the issue, but says that "print" calls
"str" for conversions.

Neither of these mentions that "str" is ASCII-only.

I think that you can use "unicode()" on any object on
which you can use "str()", but I'm not sure that's official.

Incidentally, is "repr" ASCII-only, or does it understand
Unicode? And what happens if the __str__() method of an object
returns Unicode?

John Nagle

 

[Terry Reedy]

| The Python documentation for "str" says
| "str([object]) :
| Return a string containing a nicely printable representation of an
object."
|
| However, there's no mention of the fact that "str" of a Unicode string
| with non-ASCII characters will raise a conversion exception. The
| documentation (and several Python books) seem to indicate that "str" will
| produce some "printable representation" for anything, not raise an
exception.
|
| I know, it was proposed in PEP 349 to change "str" to return Unicode
| strings, and that's coming someday, along with all-Unicode Python,
| but meanwhile, the documentation should be clear about this.

I agree. Formulate a concise sentence and submit it as a doc patch on SF.
No need for a separate file -- just put it in the report box along with the
section where the change goes and an explanation and example of the
exception raised. If accepted, a doc maintainer will put it in the Latex
source.

tjr

 

[greg]

and when we get to
Unicode-only strings, "str" will never raise a conversion exception.

On *strings*, maybe, but objects can implement __str__ in
arbitrary ways, so you can't say anything in general about
what str() will do.

 

[Steven D'Aprano]

More people read the mailing list than the bug reports, and
Google indexes it, so it puts the problem on record.

"More people" is irrelevant. What matters is "the right people".

I believe Google also indexes bug reports. It depends whether the site
allows Google's spiders or not.

If this problem
were reported as a bug, there'd be denial that it was a problem, on
the grounds that it will be fixed in a future version of Python.

I don't think so much it's denied as that the developers have decided
that fixing it isn't a valuable use of their time because there are more
important (or more interesting) problems that they could be fixing
instead, and it will go away on its own with Python 3.

This was addressed in PEP 349 (status "Deferred"), so it's
a known problem that's being ignored.

Then stop ignoring it, and submit a fix!

If you personally don't have the time or expertise to fix the problem
yourself, pay somebody to do so. Who says Open Source isn't compatible
with capitalism?

 

[Alex Martelli]

The real problem is the published books on Python:

"Learning Python", by Lutz and Ascher:
"str(string) -- returns the string representation of any object."

"Python in a Nutshell", by Martelli
Doesn't really address the issue, but says that "print" calls
"str" for conversions.

Neither of these mentions that "str" is ASCII-only.

Are you looking at p. 109 of the 2nd edition of the Nutshell? That's
where looking for str will eventually lead you. The word "ASCII" is
nowhere in sight in this page (because it's only true if you don't
change the default encoding, so saying "str is ASCII-only" would be a
lie -- now THAT kind of disinformation WOULD be a "real problem" in a
"published book on Python") but right after the documentation for
__str__ you see that for __unicode__ which explains their respective
roles (returning plain-string types and unicode strings respectively).

Unicode is documented on page 198 ff, where you will clearly see that:
"""
Python uses a default encoding, normally 'ascii'.
"""
followed by pointers to modules and functions connected with changing
that default, and a recommendation to avoid doing that and why.

I therefore dispute your contention that "Python in a Nutshell"
``doesn't really address the issue", as well as your other contention
that "the real problem is" my book (I won't speak for Lutz and Ascher).

If you disagree, feel free to open an errata on O'Reilly's page devoted
to my book (and/or to the one by Lutz and Ascher); I will of course do
my best to strongly rebut your provably-false contentions, if it's my
book you choose to attack (I use the word advisedly, because it's easy
to see that you're not interested in offering constructive criticism and
suggestions -- it would have been obvious to open an errata, had you
been thus interested -- but only in picking a public fight about this).


Alex

 

[John Nagle]

Are you looking at p. 109 of the 2nd edition of the Nutshell?

No, I'm looking at the original edition from 2003. That
was what Borders had in stock late last year.

John Nagle

 

[Alex Martelli]

No, I'm looking at the original edition from 2003. That
was what Borders had in stock late last year.

Eeek -- the 2nd edition came out in July 2006, and several months later
Borders were STILL stocking the older, long-out-of-print one?!

I occasionally checked this summer and the Palo Alto and Sunnyvale
locations of Borders did seem to upgrade reasonably fast -- but maybe
that's just a Silicon Valley thing...?


Alex

 

[Dennis Lee Bieber]

I occasionally checked this summer and the Palo Alto and Sunnyvale
locations of Borders did seem to upgrade reasonably fast -- but maybe
that's just a Silicon Valley thing...?

I'm not so sure -- I've seen both new and previous editions of books
on the shelves. I suspect they have computer reorder flag and may not
put up some new stuff until the computer says the old one has dropped
below a certain number <G>

--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/