ncurses ruby and utf-8

[Brian Schröder]

hello group,

I'm writing a small application using ncurses and I wonder how to make
it output and input utf-8 characters. I also have a readline interface
to the program, and there I can input things like û ö ñ etc. quite
fine, using the ncurses routines it chokes.
I'm using the getstr(str) call for text input.

Thanks,

Brian

 

[Nikolai Weibull]

* Brian Schröder (Mar 07, 2005 13:40):

I'm writing a small application using ncurses and I wonder how to make
it output and input utf-8 characters. I also have a readline interface
to the program, and there I can input things like û ö ñ etc. quite
fine, using the ncurses routines it chokes. I'm using the getstr(str)
call for text input.

You need ncursesw (configure --enable-widec).

Check the archive (http://www.ruby-talk.org/) for previous discussions
(Simon Strandgaard has done some investigation into using it from Ruby).

The ruby-ncurses package currently doesn't support wide character
ncurses,
nikolai

 

[Lee Braiden]

The ruby-ncurses package currently doesn't support wide character
ncurses,
nikolai

What is the general state of unicode support in ruby?

 

[Nikolai Weibull]

* Lee Braiden (Mar 07, 2005 14:40):

What is the general state of unicode support in ruby?

There is none, beyond a very hackish one for regular expression
matching via the $KCODE variable (and -K command-line option). Ruby 2.0
will fix this, hopefully in a wonderful way (I'm talking total coverage,
i.e, strings, io, regexes, and so on), so that character set and
character encoding is handled as transparently as possible (i.e, we as
Ruby programmers don't have to lift a finger),
nikolai

 

[Lee Braiden]

* Lee Braiden (Mar 07, 2005 14:40):

Ruby 2.0
will fix this, hopefully in a wonderful way (I'm talking total coverage,

Ahh. Is there an official guess on when 2.0 will happen? Best estimate I can
find is 2006-ish?

 

[Nikolai Weibull]

* Lee Braiden (Mar 07, 2005 16:10):

Ahh. Is there an official guess on when 2.0 will happen? Best
estimate I can find is 2006-ish?

That sounds like a wishful estimate. However, if I get the time (which
I won't), I'll probably hack something together as a suggestion for
Unicode support. matz has commented on some parts of the future
semantics of Strings, but there's still a lot to figure out, I'd say,
nikolai

 

[Simon Strandgaard]

On Mon, 7 Mar 2005 21:48:54 +0900, Nikolai Weibull

You need ncursesw (configure --enable-widec).

Check the archive (http://www.ruby-talk.org/) for previous discussions
(Simon Strandgaard has done some investigation into using it from Ruby).

I wrote a small .c wrapper for ncursesw.
See code here:
http://rubyforge.org/cgi-bin/viewcv.../buffer2/source/tui.c?rev=1.4&cvsroot=aeditor

 

[Brian Schröder]

On Mon, 7 Mar 2005 21:48:54 +0900, Nikolai Weibull


I wrote a small .c wrapper for ncursesw.
See code here:
http://rubyforge.org/cgi-bin/viewcv.../buffer2/source/tui.c?rev=1.4&cvsroot=aeditor

Thank you, that is very helpfull.

best regards,

Brian

 

[Cs. Henk]

* Lee Braiden (Mar 07, 2005 14:40):

There is none, beyond a very hackish one for regular expression

I just wonder how is it possible that a language which originated in
Japan doesn't support unicode & co. out of the box. They do need unicode
for proper handling of their character sets, don't they? How is it
possible historically that noone cared about it?

Csaba

 

[Dido Sevilla]

I just wonder how is it possible that a language which originated in
Japan doesn't support unicode & co. out of the box. They do need unicode
for proper handling of their character sets, don't they?

No. Before Unicode there were already several different ways of
encoding Japanese text (just as we English speakers have had ASCII and
EBCDIC in the past), and Unicode is only one of them. Apparently
Unicode isn't even the dominant way of doing so just yet: the JIS
encodings seem to be more popular at the moment, and there are even
three variations of JIS (Shift-JIS is by far the most widely used for
Japanese web pages and such, and ISO-2022-JP is most widely used for
Japanese email, and Japanese locales on Unix/Linux use the JIS EUC
encoding).

See this site for more information on this complicated topic:

http://www.cs.mcgill.ca/~aelias4/encodings.html

 

[gabriele renzi]

Dido Sevilla ha scritto:

No. Before Unicode there were already several different ways of
encoding Japanese text (just as we English speakers have had ASCII and
EBCDIC in the past), and Unicode is only one of them. Apparently
Unicode isn't even the dominant way of doing so just yet: the JIS
encodings seem to be more popular at the moment, and there are even
three variations of JIS (Shift-JIS is by far the most widely used for
Japanese web pages and such, and ISO-2022-JP is most widely used for
Japanese email, and Japanese locales on Unix/Linux use the JIS EUC
encoding).

See this site for more information on this complicated topic:

http://www.cs.mcgill.ca/~aelias4/encodings.html

Also notice that there is an old debate about unicode *not* being good
enough for japanese and other languages, for example:
http://tronweb.super-nova.co.jp/unicoderevisited.html
(I really don't understand a lot about encoding, anyway)

 

[Cs. Henk]

No. Before Unicode there were already several different ways of
encoding Japanese text (just as we English speakers have had ASCII and
EBCDIC in the past), and Unicode is only one of them. Apparently
Unicode isn't even the dominant way of doing so just yet: the JIS
encodings seem to be more popular at the moment, and there are even
three variations of JIS (Shift-JIS is by far the most widely used for
Japanese web pages and such, and ISO-2022-JP is most widely used for
Japanese email, and Japanese locales on Unix/Linux use the JIS EUC
encoding).

See this site for more information on this complicated topic:

http://www.cs.mcgill.ca/~aelias4/encodings.html

This is interesting, thanks for the pointer! But then, how are these
popular schemes supported by ruby? Isn't it possible/desirable to use a
generic scheme which can be realized then either as unicode or as one of
the above encodings?

Csaba

 

[Yukihiro Matsumoto]

Hi,

In message "Re: ncurses ruby and utf-8"

|This is interesting, thanks for the pointer! But then, how are these
|popular schemes supported by ruby? Isn't it possible/desirable to use a
|generic scheme which can be realized then either as unicode or as one of
|the above encodings?

Ruby Regexp can handle these encodings (i.e. EUC-JP, Shift_JIS, UTF-8).

matz.

 

[Tobias Peters]

I wrote a small .c wrapper for ncursesw.
See code here:
http://rubyforge.org/cgi-bin/viewcv.../buffer2/source/tui.c?rev=1.4&cvsroot=aeditor

Looking through your wrapper, I have not recognized anything that would
be specific to ncursesw, and differ from narrow-character ncurses (apart
from wrapping wcwidth, which seems to live in libc).

So I wonder what it would take to make ncurses-ruby wide-character
compatible. Could you give me a hint where to start?

Thanks,
Tobias