u'a' in string.letters fails: a Python 2.3 bug?

[Edward K. Ream]

From the documentation for the string module at:

C:\Python23\Doc\Python-Docs-2.3.1\lib\module-string.html

letters: The concatenation of the strings lowercase and uppercase described
below. The specific value is locale-dependent, and will be updated when
locale.setlocale() is called.
[end quote]

If uch is a unicode character, the operation

uch in string.letters

may (will?) fail in Python 2.3. I've never seen it fail in previous
versions. Examples:

Python 2.3.1 (#47, Sep 23 2003, 23:47:32) [MSC v.1200 32 bit (Intel)] on
win32
[snip]
IDLE 1.0
Traceback (most recent call last):
File "<pyshell#3>", line 1, in -toplevel-
u'\xa6' in string.letters
UnicodeDecodeError: 'ascii' codec can't decode byte 0x83 in position 52:
ordinal not in range(128)
Traceback (most recent call last):
File "<pyshell#1>", line 1, in -toplevel-
u'a' in string.letters
UnicodeDecodeError: 'ascii' codec can't decode byte 0x83 in position 52:
ordinal not in range(128)

Questions:

1. Is this a bug, or am I missing something?

2. Is this an issue only with Idle? I think not completely: this kind of
code seems to work for my app on XP, and not for some of my app's users on
Linux.

3. Is replacing string.letters by string.ascii_letters the recommended
workaround?

Edward

P.S.
'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz\x83\x8a\x8c\x8e\x9a\x9
c\x9e\x9f\xaa\xb5\xba\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xc
d\xce\xcf\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe
1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf
4\xf5\xf6\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff'

EKR

 

[Michael Hudson]

From the documentation for the string module at:

C:\Python23\Doc\Python-Docs-2.3.1\lib\module-string.html

letters: The concatenation of the strings lowercase and uppercase described
below. The specific value is locale-dependent, and will be updated when
locale.setlocale() is called.
[end quote]

If uch is a unicode character, the operation

uch in string.letters

may (will?) fail in Python 2.3. I've never seen it fail in previous
versions.

Must be because you weren't looking <wink>:

Python 2.2.1 (#1, Apr 9 2002, 13:10:27)
[GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-98)] on linux2
Type "help", "copyright", "credits" or "license" for more information.Traceback (most recent call last):
File "<stdin>", line 1, in ?
UnicodeError: ASCII decoding error: ordinal not in range(128)

(python from Sean's 2.2.1 RPM on redhat 7.2-ish).

1. Is this a bug, or am I missing something?

What you may be missing is that factors including but not limited to
readline, the way python was invoked, orders of imports, locals
settings and the phase of the moon may have an effect on whether
"ordinals not in range(128)" get into string.letters.

I think the interaction of readline and locale settings got a going
over for 2.3 which *might* explain any differences you're seeing.

2. Is this an issue only with Idle? I think not completely: this
kind of code seems to work for my app on XP, and not for some of my
app's users on Linux.

See above

3. Is replacing string.letters by string.ascii_letters the recommended
workaround?

Err, probably. Depends what you're testing for, I guess. Wouldn't
uch.isalpha() or one of the unicodedata thingies be more appropriate
most of the time?

Cheers,
mwh

PS: on typing control-D into the 2.2.1 session above, I get a
segfault. Now *that's* got to be a bug!

 

[Edward K. Ream]

1. Is this a bug, or am I missing something?

What you may be missing is that factors including but not limited to
readline, the way python was invoked, orders of imports, locals
settings and the phase of the moon may have an effect on whether
"ordinals not in range(128)" get into string.letters.

Thanks for this info. I wonder why string.letters remains. Shouldn't it be
deprecated?

I've substituted string.ascii_letters for string.letters as a temporary
expedient, and will consider ch.isalpha() for future work. Thanks again.

Edward