How do you deal with wchar_t and internationalization?

[DeMarcus]

Hi,

Let's say you have an application and suddenly you want to support
Japanese! How do you deal with that? Do you have to swap out all char
for wchar_t in the whole application?

I tried to read the section about internationalization in Josuttis' book
but he just skims on top of it.

Do you have experience with converting an application or can you
recommend links or books where they go into detail about it? As
mentioned, I'm especially interested in how to deal with the
char/wchar_t conversion.


Thanks,
Daniel

 

[thomas]

Hi,

Let's say you have an application and suddenly you want to support
Japanese! How do you deal with that? Do you have to swap out all char
for wchar_t in the whole application?

I tried to read the section about internationalization in Josuttis' book
but he just skims on top of it.

Do you have experience with converting an application or can you
recommend links or books where they go into detail about it? As
mentioned, I'm especially interested in how to deal with the
char/wchar_t conversion.

Thanks,
Daniel

It reminds me of the internationalization of the Linux system. You can
change the locale setting to display menus in different languages. I'm
sure it will give you some hint on how to do such conversion. So check
the related code of Linux, I would suggest.

BTW, it seems that people are getting more and more interested in such
kind of areas. Maybe it's because economy is recovering and companies
are expanding their business to asian countries.

Tom

 

[Öö Tiib]

Let's say you have an application and suddenly you want to support
Japanese! How do you deal with that? Do you have to swap out all char
for wchar_t in the whole application?

Easier is to assume that char is from now on a byte in UTF-8 sequence
(but you need editor that supports you writing UTF-8 string literals
and user interface layer that supports UTF-8). I have gone the way
that char is dirty byte for me and wchar_t is for holding trusted text
data. But there again my programming style is often more paranoid than
average.

I tried to read the section about internationalization in Josuttis' book
but he just skims on top of it.

Do you have experience with converting an application or can you
recommend links or books where they go into detail about it? As
mentioned, I'm especially interested in how to deal with the
char/wchar_t conversion.

C++ does not support internationalization of texts in any useful way.
Operating systems support it, but in non-portable way. Most GUI
frameworks support it, portable ones in more-or less portable way.
Also there are independent portable or ported internationalization
libs like GNU gettext. Finally ... portability and
internationalization makes sense only on lots of cases, on case of
tiny ARM processor in little box you just ironed together to automate
something in your house for example ... it more hurts than helps.

 

[DeMarcus]

My Windows IRC client performs UTF-8 (char and std::string) to UTF-16
(wchar_t and std::wstring) conversion to allow Unicode rendering of GUI
components, Japanese works fine. Operating systems other than windows
may use either UTF-16 or UTF-32 as their "native" Unicode encoding.

/Leigh

So if I just change everything from std::string to std::wstring,
std:stream to std::wostream, and all strings from "blah" to L"blah",
then have I done the conversion?

My point is; shall I convert everything in the system, or shall I mix
and just convert where I need to output Japanese? You may say; "it
depends", but what's your experience giving the cleanest solution?

 

[Paul N]

Let's say you have an application and suddenly you want to support
Japanese! How do you deal with that? Do you have to swap out all char
for wchar_t in the whole application?

I tried to read the section about internationalization in Josuttis' book
but he just skims on top of it.

Do you have experience with converting an application or can you
recommend links or books where they go into detail about it? As
mentioned, I'm especially interested in how to deal with the
char/wchar_t conversion.

The Microsoft approach is to use "TCHAR" to mean either char or
wchar_t, as appropriate, together with further typedefs such as
LPCTSTR to mean a pointer to constant char/wchar_t. This allows you to
to switch a whole program from chars to wchar_ts, just by setting a
single option which alters the typedefs in the headers. Some people
think this is neat, others (particularly those interested in porting
the code to other systems) think it is dreadful...

 

[DeMarcus]

I use std::string (and UTF-8) in my lower level "model" classes, I only
convert to/from std::wstring in higher level GUI component I/O code.

/Leigh

Ok, thanks!

 

[DeMarcus]

The Microsoft approach is to use "TCHAR" to mean either char or
wchar_t, as appropriate, together with further typedefs such as
LPCTSTR to mean a pointer to constant char/wchar_t. This allows you to
to switch a whole program from chars to wchar_ts, just by setting a
single option which alters the typedefs in the headers. Some people
think this is neat, others (particularly those interested in porting
the code to other systems) think it is dreadful...

Thanks. I'll take a look at that.

 

[DeMarcus]

Easier is to assume that char is from now on a byte in UTF-8 sequence
(but you need editor that supports you writing UTF-8 string literals
and user interface layer that supports UTF-8). I have gone the way
that char is dirty byte for me and wchar_t is for holding trusted text
data. But there again my programming style is often more paranoid than
average.

I'm quite paranoid too.

C++ does not support internationalization of texts in any useful way.
Operating systems support it, but in non-portable way. Most GUI
frameworks support it, portable ones in more-or less portable way.
Also there are independent portable or ported internationalization
libs like GNU gettext. Finally ... portability and
internationalization makes sense only on lots of cases, on case of
tiny ARM processor in little box you just ironed together to automate
something in your house for example ... it more hurts than helps.

Ok, thanks!