string conversion latin2 to ascii

[Martin Landa]

Hi all,

sorry for a newbie question. I have unicode string (or better say
latin2 encoding) containing non-ascii characters, e.g.

s = "Ukázka_možnosti_využití_programu_OpenJUMP_v_SOA"

I would like to convert this string to plain ascii (using some lookup
table for latin2)

to get

-> Ukazka_moznosti_vyuziti_programu_OpenJUMP_v_SOA

Thanks for any hits! Regards, Martin Landa

 

[kyosohma]

Hi all,

sorry for a newbie question. I have unicode string (or better say
latin2 encoding) containing non-ascii characters, e.g.

s = "Ukázka_možnosti_využití_programu_OpenJUMP_v_SOA"

I would like to convert this string to plain ascii (using some lookup
table for latin2)

to get

-> Ukazka_moznosti_vyuziti_programu_OpenJUMP_v_SOA

Thanks for any hits! Regards, Martin Landa

With a little googling, I found this:

http://www.peterbe.com/plog/unicode-to-ascii

You might also find this article useful:

http://www.reportlab.com/i18n/python_unicode_tutorial.html

Mike

 

[Martin v. Löwis]

sorry for a newbie question. I have unicode string (or better say

latin2 encoding) containing non-ascii characters, e.g.

s = "Ukázka_možnosti_využití_programu_OpenJUMP_v_SOA"

That's not a Unicode string (at least in Python 2); it is
a latin-2 encoded byte string; it has nothing to do with Unicode.

I would like to convert this string to plain ascii (using some lookup
table for latin2)

to get

-> Ukazka_moznosti_vyuziti_programu_OpenJUMP_v_SOA

I recommend to use string.translate. You need a translation
table there, which is best generated with string.maketrans.

table=string.maketrans("áží","azi")
print s.translate(table)

HTH,
Martin

 

[John Machin]

With a little googling, I found this:

http://www.peterbe.com/plog/unicode-to-ascii

and if the OP has the patience to read *ALL* the comments on that blog
entry, he will find that comment[-2] points to

http://effbot.python-hosting.com/file/stuff/sandbox/text/unaccent.py

and comment[-1] (from the blog owner) is "Brilliant! Thank you."

The bottom line is that there is no universal easy solution; you need
to handcraft a translation table suited to your particular purpose
(e.g. do you want u-with-umlaut to become u or ue?). The
unicodedata.normalize function is useful for off-line preparation of a
set of candidate mappings for that table; it should not be applied
either on-line or blindly.

Cheers,
John

 

[Jakub Wilk]

I have unicode string (or better say latin2 encoding) containing
non-ascii characters, e.g.

s = "Ukázka_možnosti_využití_programu_OpenJUMP_v_SOA"

I would like to convert this string to plain ascii (using some lookup
table for latin2)

to get

-> Ukazka_moznosti_vyuziti_programu_OpenJUMP_v_SOA

You may try python-elinks
Ukazka_moznosti_vyuziti_programu_OpenJUMP_v_SOA

 

[kyosohma]

With a little googling, I found this:

http://www.peterbe.com/plog/unicode-to-ascii

and if the OP has the patience to read *ALL* the comments on that blog
entry, he will find that comment[-2] points to

http://effbot.python-hosting.com/file/stuff/sandbox/text/unaccent.py

and comment[-1] (from the blog owner) is "Brilliant! Thank you."

The bottom line is that there is no universal easy solution; you need
to handcraft a translation table suited to your particular purpose
(e.g. do you want u-with-umlaut to become u or ue?). The
unicodedata.normalize function is useful for off-line preparation of a
set of candidate mappings for that table; it should not be applied
either on-line or blindly.

Cheers,
John

Sorry...I didn't know about translation tables or I would have
mentioned that instead. My bad.

Mike