Arabic text displayed as question marks

[Swetha]

Hello all

I am developing a desktop Swing application which needs to support
multiple languages. The problem I face is when I am trying to display
Arabic text in my application. The Menu items and Label texts that are
assigned to Arabic text end up showing as ?????.

I store the Arabic text in a Hashtable from where I retrieve it and
then assign it to the appropriate JComponent.

I think the problem occurs when I retrieve the values from the
Hashtable. The value is returned as ????.

From what I've been seeing in the various groups, it seems like a

problem with the encoding scheme. But how do I go about making the
application know that it has to use UTF-8 encoding?

Can someone please suggest a solution?

Thanks!

 

[Thomas Fritsch]

I am developing a desktop Swing application which needs to support
multiple languages. The problem I face is when I am trying to display
Arabic text in my application. The Menu items and Label texts that are
assigned to Arabic text end up showing as ?????.

I store the Arabic text in a Hashtable from where I retrieve it and
then assign it to the appropriate JComponent.

I think the problem occurs when I retrieve the values from the
Hashtable. The value is returned as ????.

problem with the encoding scheme. But how do I go about making the
application know that it has to use UTF-8 encoding?

Can someone please suggest a solution?

Before coming to a solution further analysis is needed. Your problem can
have 2 different causes:
(1) The text values (probably Strings) could be corrupted somehow,
may be by using a wrong encoding.
(2) The font (of menuItem, label) might not support arabic characters.

To check for (2) there are some tools in the net:
http://segal.org/java/Arabic/index.html
http://www.myjavaserver.com/~thomasfritsch/unicode.html
(scroll down to 0600)
To check for (1) we would need to see some of your java source,
especially how you construct your Strings and put them into the Hashtable.

 

[Swetha]

Hi Thomas

(1)

This is how I put the strings into the Hashtable:

Hashtable table = new Hashtable();
table.add("javax.swing.JMenu-0:ar", "هعثق");
table.add("javax.swing.JMenu-0:ar", "نثت");
table.add("javax.swing.JMenu-0:ar", "مضصث");
table.add("javax.swing.JMenu-0:ar", "بصمث");
table.add("javax.swing.JMenu-0:ar", "صنثق");

(2)

http://segal.org/java/Arabic/index.html
I'm able to see the Arabic script in the applet that is created above.

http://www.myjavaserver.com/~thomasfritsch/unicode.html
When I scroll down to 0600, I can see some of the Arabic characters
while others appear as boxes.

 

[Swetha]

Actually I have a problem in re-opening files with Arabic text created
from the IDE. When I create the file with Arabic text, it shows up
fine. But on saving, closing and re-opening the file, all the
characters are displayed as question marks. This happens only with
files I create in the IDE.

However, files I create with worpad are stored and reopen with the
Arabic text properly. However, when I open these files (created with
Arbic text on wordpad) with the IDE, I get weird characters such as
"ÕËÞÕËÞ" and the like....

 

[Thomas Weidenfeller]

This is how I put the strings into the Hashtable:

Hashtable table = new Hashtable();
table.add("javax.swing.JMenu-0:ar", "هعثق");
table.add("javax.swing.JMenu-0:ar", "نثت");
table.add("javax.swing.JMenu-0:ar", "مضصث");
table.add("javax.swing.JMenu-0:ar", "بصمث");
table.add("javax.swing.JMenu-0:ar", "صنثق");

This is a rather strange approach.

a) You always use the same key. Only your last value will survive. The
others will be lost.

b) Usually one puts such strings in a resource bundle. Not the least,
because resource bundles are supposed to hold locale-specific
information like GUI messages and text, so you get them out of the
source code.

c) Placing these characters in the source code, while apparently not
knowing which encoding is in use in the IDE is a receipt for disaster.
If you really feel the need to do it, you have to get each bit carefully
in place. Your editor, the IDE and the compiler need all be configured
so that they agree on the encoding of your source code.

Which ...

d) ... generates a maintenance nightmare, in case a maintainer isn't
aware of all the details, and has to guess all this.


Place the text in a resource bundle and try to stick with ASCII in the
source code. If you still need non-ASCII characters in the code, use the
\u syntax (or native2ascii). You will anyhow need to get used to it,
since resource bundles need to be written with ISO Latin one only, and
\u escapes - which are simple to generate via native2ascii.

/Thomas

 

[Thomas Fritsch]

(1)

This is how I put the strings into the Hashtable:

Hashtable table = new Hashtable();
table.add("javax.swing.JMenu-0:ar", "هعثق");

Putting non-ASCII-characters directly between "..." is a risky thing.
The compiler javac might use another encoding for reading your source,
than you expect. Therefore I would recommend to use the \uxxxx unicode
escape syntax, for example:
table.add("javax.swing.JMenu-0:ar", "\u0630\u0631\u0632\u0633");
That makes your java-source totally independent of encodings.

 

[Roedy Green]

problem with the encoding scheme. But how do I go about making the
application know that it has to use UTF-8 encoding?

see http://mindprod.com/applets/fontshower.html
http://mindprod.com/applets/fontshowerawt.html

You need first of all to make sure the font you are using supports
Arabic.

 

[Roedy Green]

table.add("javax.swing.JMenu-0:ar", "????");
table.add("javax.swing.JMenu-0:ar", "???");
table.add("javax.swing.JMenu-0:ar", "????");
table.add("javax.swing.JMenu-0:ar", "????");
table.add("javax.swing.JMenu-0:ar", "????");

for a HashMap or Hashtable the keys must be unique.

 

[Roedy Green]

However, files I create with worpad are stored and reopen with the
Arabic text properly. However, when I open these files (created with
Arbic text on wordpad) with the IDE, I get weird characters such as
"ÕËÞÕËÞ" and the like....

code them with \uxxxx to get the program going.

You need to tell Javac which encoding you are using in your source
files.

See http://mindprod.com/jgloss/javacexe.html

 

[Swetha]

Thanks everyone for your inputs!!

The

table.add("javax.swing.JMenu-0:ar", "????");
table.add("javax.swing.JMenu-0:ar", "????");
table.add("javax.swing.JMenu-0:ar", "????");

was a copy-paste error, I copy-pasted the same line and forgot to
change the keys before posting the message!!

And I will take your suggestions of using \uxxxx unicode escape syntax.
It seems to work fine this way!!

Thanks again.

 

[Roedy Green]

And I will take your suggestions of using \uxxxx unicode escape syntax.
It seems to work fine this way!!

Unless this code will never be translated to any other language,
normally you would encode your Arabic Strings using a ResourceBundle.
See http://mindprod.com/jgloss/internationalisation.html
http://mindprod.com/jgloss/resourcebundle.html

 

[Swetha]

Unless this code will never be translated to any other language,

normally you would encode your Arabic Strings using a ResourceBundle.

I am using ResourceBundle with properties files for storing the data,
since I am using more than one language and might have to add more
languages later on.

The Hashtable method was only an initial try.

Thanks for that!!

 

[Roedy Green]

I am using ResourceBundle with properties files for storing the data,
since I am using more than one language and might have to add more
languages later on.

Since the resource file itself has to be ISO-8959-1, you can either
code the \uxxx chars manually, or do them with an Arabic aware editor
and convert the file with nativetoascii.

 

[Swetha]

Where can I get the nativetoascii tool and how do I use it?

I checked the http://mindprod.com/jgloss/encoding.html#NATIVETOASCII
site, but am not able to figure out what to do.

 

[Swetha]

I think I got it.

Found the native2ascii tool in Java's bin directory and ran it with
-encoding Cp1256 and it seemed to give the expected output.

Thanks!!
Swetha

 

[Roedy Green]

Where can I get the nativetoascii tool and how do I use it?

It comes with the JDK. See http://mindprod.com/jgloss/encoding.html

 

[Roedy Green]

Found the native2ascii tool in Java's bin directory and ran it with
-encoding Cp1256 and it seemed to give the expected output.

Great. You can do the reverse to make your property files editable .