utf8 encoding problem

[Ad Ad]

Hi,
I am retrieving a string from a txt file.
The file contains some utf8 characters.

I am comparing these characters against a default string.

The problem is that some of the characters are not stored in a default
format.

For example:
A is stored as Ａ

Naturally when I compare the character it fails.
Strangely when I unpacked the character it appears as 65313 which is the
correct utf8 number for A.

Any way around this?

thanks.

 

[Eric Hodel]

Hi,
I am retrieving a string from a txt file.
The file contains some utf8 characters.

I am comparing these characters against a default string.

The problem is that some of the characters are not stored in a default
format.

For example:
A is stored as =EF=BC=A1

Naturally when I compare the character it fails.
Strangely when I unpacked the character it appears as 65313 which is =20=

the
correct utf8 number for A.

Any way around this?

Well, =EF=BC=A1 is "Fullwidth Latin Capital Letter A" from the "Hiragana =
and =20
Katakana" category (Unicode FF21) whereas A is "Latin Capital Letter =20
A" from the "Latin" category (Unicode 0041).

I don't know of a way to translate between the two categories, but =20
maybe that will help.=

 

[James Rubingh]

Although I haven't tried it myself, I did a search for $BA43QH>3QJQ49(B and
found this page.
It appears people use jcode and tr to solve this problem.

http://www.eml.ele.cst.nihon-u.ac.jp/~momma/wiki/wiki.cgi/Ruby/全角半角変換.html
http://blog.grayproductions.net/articles/the_kcode_variable_and_jcode_library

 

[Ad Ad]

Although I haven't tried it myself, I did a search for
$BA43QH>3QJQ49(B and
found this page.
It appears people use jcode and tr to solve this problem.

http://www.eml.ele.cst.nihon-u.ac.jp/~momma/wiki/wiki.cgi/Ruby/全角半角変換.html
http://blog.grayproductions.net/articles/the_kcode_variable_and_jcode_library


2009/6/25 Eric Hodel <[email protected]>:

brilliant!
str.tr!('ï½-ｚＡ-Ｚ','a-zA-z') worked like a charm.