2.3: test test_normalization failed

[Hans-Joachim Widmaier]

When I compiled Python 2.3 at home, everything went well. make test
reported no problems. At work, though, I got:

test test_normalization failed -- 03F9;03F9;03F9;03A3;03A3;

At first I thought it might be a problem with a somewhat inconsistently
upgraded system, but a few looks showed me the real difference.
At home I had updated Python quite frequently and downloaded
NormalizationTest.txt some months ago. At work, I downloaded it last week.
With revision 3.2.0 of said file the test passes, with revision 4.0.0 it
fails.

I have almost zip knowledge of the whole unicode business 'til now, so
I'm not overly concerned. Was 3.2.0 buggy? Is 4.0.0 buggy? Are there
different, incompatible specs? (Now _that_ I can't think of!)

Whatever, maybe someone can give a little insight.

Thanks for a great 2.3!

Hans-Joachim

 

[Michael Hudson]

When I compiled Python 2.3 at home, everything went well. make test
reported no problems. At work, though, I got:

test test_normalization failed -- 03F9;03F9;03F9;03A3;03A3;

At first I thought it might be a problem with a somewhat inconsistently
upgraded system, but a few looks showed me the real difference.
At home I had updated Python quite frequently and downloaded
NormalizationTest.txt some months ago. At work, I downloaded it last week.
With revision 3.2.0 of said file the test passes, with revision 4.0.0 it
fails.

This is known: Python from CVS explicitly requests the 3.2 file.

I have almost zip knowledge of the whole unicode business 'til now, so
I'm not overly concerned. Was 3.2.0 buggy? Is 4.0.0 buggy? Are there
different, incompatible specs? (Now _that_ I can't think of!)

I would imagine it being something like unicode 4 adding more
characters, but Python 2.3 -- being built on 3.2 normalization data --
doesn't know how these new character compose or decompose. Just a
guess.

Cheers,
mwh

 

[Hans-Joachim Widmaier]

[test failure with 4.0.0]

This is known: Python from CVS explicitly requests the 3.2 file.

Maybe test_normalization.py should look at the file version itself and
tell the unsuspicious that it really wants the 3.2 version.

I would imagine it being something like unicode 4 adding more
characters, but Python 2.3 -- being built on 3.2 normalization data --
doesn't know how these new character compose or decompose. Just a
guess.

Sounds reasonable. So the 2.3 normalization is just incomplete
regarding the newest standard.

Dismissed.

Thanks,
Hans-Joachim

 

[Michael Hudson]

[test failure with 4.0.0]

This is known: Python from CVS explicitly requests the 3.2 file.

Maybe test_normalization.py should look at the file version itself and
tell the unsuspicious that it really wants the 3.2 version.

It's kind of hard to do that for the already released 2.3

Sounds reasonable. So the 2.3 normalization is just incomplete
regarding the newest standard.

Yep. I would guess it likely that 2.4 will be up to date again, but
it's not really my field.

Cheers,
mwh

 

[Martin v. =?iso-8859-15?q?L=F6wis?=]

Yep. I would guess it likely that 2.4 will be up to date again, but
it's not really my field.

It will be tricky to update. Python now also supports IDNA
(encodings/idna.py), which implements RFC 3490/91/92, which requires
the Unicode 3.2 database; updating to 4.0 would be a violation of that
RFC. So if we want to update the database, we need to find a way still
to keep the old one.

Regards,
Martin