Extension Language for a Text Editor

[Nikolai Weibull]

OK. So I'm going to write a text editor for my masters' thesis. The
general idea of it is fixed but the extension language has not been
settled on, yet. I have thought of, in the footsteps of editors such as
Emacs and its clones, using some version of LISP or Scheme, possibly
GNU's Guile[1], or Jade's librep[2]. LISP-like languages have proved
very successful in describing the way we edit text and is thus a sure
bet. It would, however, perhaps be of interest to use another language
to try the waters. In my previous editor (slackedit/sled[3]) I used
Tcl[4] as an extension language, and it proved very easy to use, but I
never got far enough to actually give it a good run.

Anyway, what I'm getting at is:

Do you figure Ruby to be a good extension language for a text editor?
* What would be easier to do using Ruby?
* LISP?
* What language allows the most of the editing commands to be written in
the given language?
* Is Ruby good enough for the task at the moment, performance wise?
* Resource wise? Does anyone have any statistics on this?

I'm not talking about the ease of implementing a web browser in the
target language, rather the ease of structuring an extensible framework
in it.

Also, is there any way of redefining the // operator for constructing
regular expression objects? I'm planning on implementing a new regex
syntax for the editor (to make searches/substitutions easier to
describe).

thanks in advance,
nikolai

[1] http://www.gnu.org/software/guile/
[2] http://librep.sourceforge.net/
[3] http://www.pcppopper.org/code/win/sled/
[4] http://www.scriptics.com/

 

[Ryan Pavlik]

OK. So I'm going to write a text editor for my masters' thesis. The
general idea of it is fixed but the extension language has not been
settled on, yet.

<snip>

The thing with lisp, though, is that everyone has used lisp already,
and while I love lisp, there wouldn't be much of a reason to switch
from XEmacs and its twenty million extensions.

Anyway, what I'm getting at is:

Do you figure Ruby to be a good extension language for a text
editor?

YES. I've been wanting "erubs" (Editor for Ruby Scripts ;-) for
awhile; something like emacs, but s/lisp/ruby. Same design otherwise.

* What would be easier to do using Ruby?

Well, the main thing is that ruby has a lot of very convenient pattern
and text matching functionality, and a boatload of extensions. And
it's a very easy language, so anyone can pick it up and start
extending the editor, unlike emacs, where fewer brave the waters.

For instance, I recently wrote a bit of code to align things:

a = b ___\ a = b
foo = bar / foo = bar

This was a few simple lines of ruby. Not counting comments, it's 44
lines of lisp that don't quite work perfectly.

* LISP?

As much as I love lisp, I can't really think of things that would be
easier to in any of the above scheme flavors that wouldn't be easier
in ruby.

* What language allows the most of the editing commands to be written in
the given language?

Ruby certainly allows this. It's extremely trivial to extend and use
Ruby in C. They go hand-in-hand. Plus there are existing GUI
packages you could interface to if you wanted to provide UI handling
in ruby without a lot of work.

* Is Ruby good enough for the task at the moment, performance wise?
* Resource wise? Does anyone have any statistics on this?

This is a decent benchmark:

http://www.bagley.org/~doug/shootout/craps.shtml

Is ruby at the top of that list? No. (Note that most of the
languages above it are compiled, including the bigloo scheme
implementation). But XEmacs elisp falls near the bottom, and I use
that for editing tasks daily. Tcl and Rep both fall below.

So, I wouldn't worry about performance. And, memory-wise, I can't
really offer benchmarks, but I have a system managing tens of
thousands of objects that doesn't present a problem.

I'm not talking about the ease of implementing a web browser in the
target language, rather the ease of structuring an extensible framework
in it.

Then Ruby is ideal. Extending and embedding it is extremely easy.
It's a really great scripting framework, even though many people
including myself use it for entire applications.

Also, is there any way of redefining the // operator for constructing
regular expression objects? I'm planning on implementing a new regex
syntax for the editor (to make searches/substitutions easier to
describe).

No, but I would encourage you leave the existing regular expression
format intact, as, while somewhat write-only, is familiar to most
people who will be using your editor. Redefining // would break a lot
of extension code, and alienate a large part of your audience.

However, that's not to say you can't make your own matching format.
I'm not sure what you have in mind, but there are a number of ways to
bend the syntax to integrate such things.

thanks in advance,

If you write a ruby editor... thank _you_. ;-)

hth,

 

[Simon Strandgaard]

OK. So I'm going to write a text editor for my masters' thesis.

What a coincidence.. I am also writing a programmers editor (AEditor)
in Ruby.
http://aeditor.rubyforge.org/

In my previous editor (slackedit/sled[3]) I used
Tcl[4] as an extension language, and it proved very easy to use, but I
never got far enough to actually give it a good run.

where is the homepage ? ;-)
http://sourceforge.net/projects/slackedit
Your Trove-info says 'gnome', but no unix install.

Do you figure Ruby to be a good extension language for a text editor?

My original plan were to use Ruby as an extension language and write
the core in C++... At some point started experimenting using Ruby for
everything, and now I have actually written AEditor completely in Ruby!!

* What would be easier to do using Ruby?

undo/redo/macros.. The code which is responsible for this only takes up
119 lines of Ruby code. This is more complex in C++ because you cannot
#clone arbitrary objects.

unittesting: AEditor has about 440 testcases and 1400 assertions.
unittesting is annoying in C++. In Ruby its easy.

It would be easier for users of your editor to make changes
to the internals of your editor (because Ruby is intuitive).
If you have a testharness then they will also could test if
their changes has unforeseen sideeffects.

* LISP?

not enough intuitive.

* What language allows the most of the editing commands to be written in
the given language?

If your editor core is written in a different language than your extension
language. Then you will no matter what language chosen, have to make a
bridge between. There are shortcuts, such as SWIG.
SWIG can generate wrappers for *many* script languages.
My vote would be Ruby

* Is Ruby good enough for the task at the moment, performance wise?

You can try out AEditor, to feel if Ruby is fast enough.
On a 100MHz machine AEditor feels quite Okay.
On a 33MHz 486 it can be slow.
I can live with this hardware requirements, thats why I choose to write
everything in Ruby

* Resource wise? Does anyone have any statistics on this?

My datastructure has a huge amount of memory overhead (because it uses an
Array of characters). I am in the process of rethinking the datastructure
and maybe write a datastructure Ruby-extension module in C++..?

Also, is there any way of redefining the // operator for constructing
regular expression objects? I'm planning on implementing a new regex
syntax for the editor (to make searches/substitutions easier to
describe).

Overloading Regexp... I don't know.
I am currently writing my own regex engine.



What features do you plan for your editor ?
Is it a programmers editor, or a propertional font editor ?

 

[Nikolai Weibull]

The thing with lisp, though, is that everyone has used lisp already,
and while I love lisp, there wouldn't be much of a reason to switch
from XEmacs and its twenty million extensions.

Yes, this is one of the downsides in a way. You won't be able to get
people to switch very easily when everything is the same ;-)

YES. I've been wanting "erubs" (Editor for Ruby Scripts ;-) for
awhile; something like emacs, but s/lisp/ruby. Same design otherwise.

hehe, that won't be the name, but yeah, that's in a sense what I want.
Except s/Emacs/Vim/ ;-).

Well, the main thing is that ruby has a lot of very convenient pattern
and text matching functionality, and a boatload of extensions. And
it's a very easy language, so anyone can pick it up and start
extending the editor, unlike emacs, where fewer brave the waters.

Well, no dependencies on Ruby extensions would be necesarry, but I
guess, since they exist, they could be used.

For instance, I recently wrote a bit of code to align things:

a = b ___\ a = b
foo = bar / foo = bar

This was a few simple lines of ruby. Not counting comments, it's 44
lines of lisp that don't quite work perfectly.

Could you provide the code? I'd love to see the comparison.

As much as I love lisp, I can't really think of things that would be
easier to in any of the above scheme flavors that wouldn't be easier
in ruby.

OK. The way I see it, Ruby is OO-programming personified. LISP is,
well, functional? programming personified. Anyway, I have gotten the
feeling that it's generally easier to think in terms of functional, not
OO, when editing text. I mean, what do tho OO constructs really add?

Ruby certainly allows this. It's extremely trivial to extend and use
Ruby in C. They go hand-in-hand. Plus there are existing GUI
packages you could interface to if you wanted to provide UI handling
in ruby without a lot of work.

Ah, I think you misunderstood the question. Or I'm misunderstanding the
answer ;-). I want the C core to be as small as possible, leaving the
most possible flexibility using the extension language. And I don't
generally see the need for GUI extensions and such.

This is a decent benchmark:

http://www.bagley.org/~doug/shootout/craps.shtml

Is ruby at the top of that list? No. (Note that most of the
languages above it are compiled, including the bigloo scheme
implementation). But XEmacs elisp falls near the bottom, and I use
that for editing tasks daily. Tcl and Rep both fall below. Thanks, great summary!
So, I wouldn't worry about performance. And, memory-wise, I can't
really offer benchmarks, but I have a system managing tens of
thousands of objects that doesn't present a problem. OK. Good.
Then Ruby is ideal. Extending and embedding it is extremely easy.

Yeah, this, I know. My experiences from the Ruby-GNOME2 project have
been great. Very easy to add extensions. I haven't used the embedding
facilities yet, but I'm sure they're good.

No, but I would encourage you leave the existing regular expression
format intact, as, while somewhat write-only, is familiar to most
people who will be using your editor. Redefining // would break a lot
of extension code, and alienate a large part of your audience.

However, that's not to say you can't make your own matching format.
I'm not sure what you have in mind, but there are a number of ways to
bend the syntax to integrate such things.

Yeah, but that's just the thing. This is one of the real selling
points, if you will ;-). Oh well, I guess one can always do some
MyRegex.new(string)
but then you wind up with the string interpolation problems (\n and
friends). :-(

If you write a ruby editor... thank _you_. ;-) hehe, we'll see ;-)

hth,

thanks for your long and quick response. It's good to know that at
least someone is interested ,
nikolai

 

[gabriele renzi]

il Thu, 9 Oct 2003 05:06:32 +0900, Nikolai Weibull
<[email protected]> ha scritto::


my 2c:

Anyway, what I'm getting at is:

Do you figure Ruby to be a good extension language for a text editor?
* What would be easier to do using Ruby?

the learning curve for casual user would be really less hard, and it
won't collide with already existing lisp based editors

* What language allows the most of the editing commands to be written in
the given language?

Well, given that tab expansion, string replacing/finding,
byte/word/line count, justification and other stuff can be easily done
with one line of ruby...
Anyway, I think the apis you make available from the core of the
editor would be something really important in this analisys.

* Is Ruby good enough for the task at the moment, performance wise?

Sure

* Resource wise? Does anyone have any statistics on this? dunno

Also, is there any way of redefining the // operator for constructing
regular expression objects? I'm planning on implementing a new regex
syntax for the editor (to make searches/substitutions easier to
describe).

well, you have the soruce code, dig in it and change what you need, if
you really need it
Anyway, why would you do such a thing ? the regex language is quite
standardized now, why would you change it?

 

[Dale Martenson]

--=-k1N1cbYdS6APwxuOu+Us
Content-Type: text/plain
Content-Transfer-Encoding: 7bit

You may want to look at the VIM's use of Ruby for writing extensions.
Documentation available inside of VIM via ":help ruby".

For an up-to-date copy of VIM see: http://www.vim.org

--Dale

--=-k1N1cbYdS6APwxuOu+Us
Content-Type: text/html; charset=utf-8

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
<META NAME="GENERATOR" CONTENT="GtkHTML/1.0.2">
</HEAD>
<BODY>
You may want to look at the VIM's use of Ruby for writing extensions. Documentation available inside of VIM via &quot;:help ruby&quot;.
<BR>

<BR>
For an up-to-date copy of VIM see: <A HREF="http://www.vim.org">http://www.vim.org</A>
<BR>

<BR>
--Dale
</BODY>
</HTML>

--=-k1N1cbYdS6APwxuOu+Us--

 

[Ryan Pavlik]

* Ryan Pavlik <[email protected]> [Oct, 08 2003 22:30]:

YES. I've been wanting "erubs" (Editor for Ruby Scripts ;-) for
awhile; something like emacs, but s/lisp/ruby. Same design
otherwise.

hehe, that won't be the name, but yeah, that's in a sense what I want.
Except s/Emacs/Vim/ ;-).

Well, whichever, just be aware that emacs is designed as an extensible
editor, and vim is not, even though such things have crept in with
varying degrees of usefulness.

Well, no dependencies on Ruby extensions would be necesarry, but I
guess, since they exist, they could be used.

Right. They're there, people can write extensions that interface to
the web, or whatever.

Could you provide the code? I'd love to see the comparison.

See http://ogmo.mephle.org/tabular-alignment.org for the Lisp
version. The ruby one I deleted, as it was pretty simple to
reproduce, I'm sure someoone can whip up an example.

OK. The way I see it, Ruby is OO-programming personified. LISP is,
well, functional? programming personified. Anyway, I have gotten the
feeling that it's generally easier to think in terms of functional, not
OO, when editing text. I mean, what do tho OO constructs really
add?

I don't really find that. I don't think functional programming is any
easier for editor-related tasks. I'm not even sure how you would come
up with such an assumption.

Ah, I think you misunderstood the question. Or I'm misunderstanding the
answer ;-). I want the C core to be as small as possible, leaving the
most possible flexibility using the extension language. And I don't
generally see the need for GUI extensions and such.

Right, tiny C core like emacs, everything higher-level in the language
of choice. Ruby is highly suited for this task.

People could even write high-performance ruby extensions in C...

Yeah, but that's just the thing. This is one of the real selling
points, if you will ;-). Oh well, I guess one can always do some

Well, to be blunt, whatever you come up with won't be as popular or
useful as the existing regular expressions, just because they'll be a
nonstandard replacement of something already very common. PCRE
regexps are extremely flexible and well-known.

That isn't to say people won't use them, especially if they're
simpler, but it probably won't be the main selling point of your
editor to _other people_.

MyRegex.new(string)

but then you wind up with the string interpolation problems (\n and
friends). :-(

I'm not sure how that's a problem. The same applies to // regexps.
They're just basically strings, except stored in a different type of
object with a few flags.

hth,

 

[Simon Strandgaard]

Anyway, why would you do such a thing ? the regex language is quite
standardized now, why would you change it?

I am in the process of writing my own regex engine (for AEditor).
The reason why I cannot use Ruby's are, that I must run regex's on
a wide range of iterators.

I want to use same syntax as Ruby's regex, and perhaps extend it
with some editor stuff, perhaps like:
* how to traverse the datastructure, jump over folds, jump into folds.
* regex on columnar selections.
* mechanism for getting the syntax-state: perhaps a regex which
only operates inside comments, apply regex to variable-names, etc.

Status for my regex project:
DONE studies of how Rubys GNU-regex engine behave.
DONE nfa 2 dfa.
DONE regex-datastructure 2 nfa.
TODO translate regex-string into regex-datastructure

There is probably some item which I have overseen.

You can check it out here:
http://rubyforge.org/cgi-bin/cvsweb.cgi/projects/experimental/nfa_to_dfa/?cvsroot=aeditor

 

[Seth Kurtzberg]

Well, for one thing, doing it in lisp would be reinventing the wheel, as
emacs has been around for ages.

One main advantage that I see to ruby is that one could modify existing
code and add new code much more easily if the modifier doesn't have a
lisp background. lisp code is horribly unreadable.

If you do want to use a functional language Haskell is a much better
choice, and would in fact be interesting in this context.

OK. So I'm going to write a text editor for my masters' thesis. The
general idea of it is fixed but the extension language has not been
settled on, yet. I have thought of, in the footsteps of editors such as
Emacs and its clones, using some version of LISP or Scheme, possibly
GNU's Guile[1], or Jade's librep[2]. LISP-like languages have proved
very successful in describing the way we edit text and is thus a sure
bet. It would, however, perhaps be of interest to use another language
to try the waters. In my previous editor (slackedit/sled[3]) I used
Tcl[4] as an extension language, and it proved very easy to use, but I
never got far enough to actually give it a good run.

Anyway, what I'm getting at is:

Do you figure Ruby to be a good extension language for a text editor?
* What would be easier to do using Ruby?
* LISP?
* What language allows the most of the editing commands to be written in
the given language?
* Is Ruby good enough for the task at the moment, performance wise?
* Resource wise? Does anyone have any statistics on this?

I'm not talking about the ease of implementing a web browser in the
target language, rather the ease of structuring an extensible framework
in it.

Also, is there any way of redefining the // operator for constructing
regular expression objects? I'm planning on implementing a new regex
syntax for the editor (to make searches/substitutions easier to
describe).

thanks in advance,
nikolai

[1] http://www.gnu.org/software/guile/
[2] http://librep.sourceforge.net/
[3] http://www.pcppopper.org/code/win/sled/
[4] http://www.scriptics.com/

--
::: name: Nikolai Weibull :: aliases: pcp / lone-star / aka :::
::: born: Chicago, IL USA :: loc atm: Gothenburg, Sweden :::
::: page: www.pcppopper.org :: fun atm: gf,lps,ruby,lisp,war3 :::
main(){printf(&linux["\021%six\012\0"],(linux)["have"]+"fun"-97);}


.

--
Seth Kurtzberg
CTO
Scottsdale Research and Network Operations Center
ISEC.us
(e-mail address removed)

 

[Ryan Pavlik]

On Thu, 9 Oct 2003 07:10:42 +0900

I want to use same syntax as Ruby's regex, and perhaps extend it
with some editor stuff, perhaps like:

There seem to be simpler/more elegant solutions to these problems than
reimplementing pcre:

* how to traverse the datastructure, jump over folds, jump into
folds.
* regex on columnar selections.
* mechanism for getting the syntax-state: perhaps a regex which
only operates inside comments, apply regex to variable-names, etc.

All three of these can easily be accomplished by providing constrained
input to the matcher. For folds etc, first you grab the pieces you
want (folds, no folds), then apply the re. For colums, same deal.
Extract the column, then apply. I'm sure you can guess where this is
going for the last one... extract and apply the regexp only to
comments or variable names.

You could have a simple "Region" class which selects the appropriate
areas, make an array of applicable regions, and then apply the regexp
to each. This gives you three exceedingly simple tasks:

1) Region objects defined by <start> and <end>.
2) Regexp application to a region.
3) Matching code to produce Regions. (Obviously, you'll need
this in any case, to select only comments, etc.)

I'd say this is way simpler, more effective, and more extensible than
redefining all-new regexp syntax. Especially since the existing
syntax is hairy enough. ;-)

 

[Gavin Sinclair]

You may want to look at the VIM's use of Ruby for writing extensions.
Documentation available inside of VIM via ":help ruby".

For an up-to-date copy of VIM see: http://www.vim.org

There's a little discussion of this at

http://www.rubygarden.org/ruby?VimRubyInterface

This capability of Vim doesn't seem to be heavily used. I certainly
don't use it. Partially because the integration is a bit lacking (but
you could do Ryan's aligner very easily - but then there's an
excellent and well-supported vim plugin for that anyway). Partially
because I couldn't be bothered.

Gavin

 

[Simon Strandgaard]

On Thu, 9 Oct 2003 07:10:42 +0900



There seem to be simpler/more elegant solutions to these problems than
reimplementing pcre:

Agree, it may be sufficiently to only replace "regex.c/re_search()" with
something which can operate on an iterator-range. But I am not that strong
in the internals of Ruby and
I prefer C++ over C. I rather make a full-blown regex engine in Ruby than
interfacing to GNU regex in C. It may take longer, but the reward is many:
* exercise my regex knowledge.
* a regex engine with Ruby license.
I am not interested in doing what may be the easiest. I just do what I like

All three of these can easily be accomplished by providing constrained
input to the matcher. For folds etc, first you grab the pieces you
want (folds, no folds), then apply the re. For colums, same deal.
Extract the column, then apply. I'm sure you can guess where this is
going for the last one... extract and apply the regexp only to
comments or variable names.

Your are talking about traversing, and you are absolutly right..
But I am talking about the syntax of the regex. Misunderstanding

[snip more text about traversing]
BTW: I use many different kinds of iterators for traversing.

I'd say this is way simpler, more effective, and more extensible than
redefining all-new regexp syntax. Especially since the existing
syntax is hairy enough. ;-)

Yes this isn't going to be easy. I am the kind of type which like hairy
problems

 

[Robert Klemme]

Since others have commented on various aspects of your posting I'll only
throw my 2c at this:

Also, is there any way of redefining the // operator for constructing
regular expression objects? I'm planning on implementing a new regex
syntax for the editor (to make searches/substitutions easier to
describe).

Why do you want to change it? I find it quite flexible and expressive
(especially when using flag "x"). And especially if you want to attract
rubyists to use TREC (The Ruby Editor to Come), then you should ensure
that their knowledge of ruby regexps is not obsoleted, since rx's will
likely play an important role in user defined extensions!

You can always define methods in Kernel like this:

irb(main):002:0* module Kernel
irb(main):003:1> def rx(str)
irb(main):004:2> puts "building rx from '#{str}'"
irb(main):005:2> end
irb(main):006:1> end
=> nil
irb(main):007:0>
irb(main):008:0* rx %q{(foo)+ \s+ (\w+)}
building rx from '(foo)+ \s+ (\w+)'
=> nil
irb(main):009:0> rx '(foo)+ \s+ (\w+)'
building rx from '(foo)+ \s+ (\w+)'
=> nil
irb(main):010:0>

Which isn't too bad IMHO.

Regards

robert

 

[Frank Schmitt]

undo/redo/macros.. The code which is responsible for this only takes up
119 lines of Ruby code. This is more complex in C++ because you cannot
#clone arbitrary objects.

If you are talking about user-defined class hierarchies, you *can* clone
them, as well - it just requires extra work:

class Base {
virtual Base* clone() = 0;
};

class Derived1: public Base {
virtual Base* clone() { return new Derived1(*this); }
};

class Derived2: public Base {
virtual Base* clone() { return new Derived2(*this); }
};

unittesting: AEditor has about 440 testcases and 1400 assertions.
unittesting is annoying in C++. In Ruby its easy.

Why is unittesting in C++ annoying? I have an application with 200+
testcases and 400+ assertions, written completely in C++.
The only thing that annoys me is that I can't easily add/remove tests
without recompiling - some day I'll have to write a ruby wrapper to do this

kind regards
frank

 

[Simon Strandgaard]

If you are talking about user-defined class hierarchies, you *can* clone
them, as well - it just requires extra work:

I know.. cloning in C++ get tiresome at that point when you have
many childred.

class Base {
virtual Base* clone() = 0;
};

class Derived1: public Base {
virtual Base* clone() { return new Derived1(*this); }
};

This method requires typecasting if you want to do something like this:

Derived *a = new Derived1();
Derived *b = dynamic_cast<Derived*>(a->clone());


Covariant return types is unfortunatly a relative new thing in the C++
world.

class Derived1: public Base {
virtual Derived1* clone() { return new Derived1(*this); }
};

Derived *a = new Derived1();
Derived *b = a->clone();

Why is unittesting in C++ annoying? I have an application with 200+
testcases and 400+ assertions, written completely in C++.
The only thing that annoys me is that I can't easily add/remove tests
without recompiling - some day I'll have to write a ruby wrapper to do

this

Compiling is expensive on my Pentium350 which is the fastest machine I have.
GCC2.96 was quite fast... I have not yet seen any fast result with GCC3.
I like templates, I like many things in C++. If I had a faster machine, then
there would be no problems

 

[Nikolai Weibull]

Well, whichever, just be aware that emacs is designed as an extensible
editor, and vim is not, even though such things have crept in with
varying degrees of usefulness.

Yes, that, as I see it, is Emacs biggest win. The big problem I see
with Vim is that it hasn't undergone a major overhaul. It is basically
Emacs written in C now. Vim is, in my opinion, probably the best editor
that exists right now. It is, however, going to reach a point where
adding new features will demand some sort of rewrite. With Emacs,
things like these are easy to alter. Anyway, Vim is extensible, it is,
however, not _changable_. By that I mean that, if you want to change
the way folding works, you must rewrite the core of Vim. And, as Bram
pointed out in some interview, that means altering basically every file
in the Vim distribution. To Vim's salvation comes the ability to easily
define new syntax definitions and indentation definitions, which one has
to agree, are a lot easier to create and alter than with Emacs (Emacs
being perhaps more powerful though).
[stuff about Ruby libraries and such]

Right. They're there, people can write extensions that interface to
the web, or whatever.

Yes, but I don't want an Operating System. I guess, to a certain degree
the library you get helps you, but it can also detract you from the
central topic, namely editing text.

See http://ogmo.mephle.org/tabular-alignment.org for the Lisp
version. The ruby one I deleted, as it was pretty simple to
reproduce, I'm sure someoone can whip up an example.

Yeah, OK. I see your point. It is, however, very easy to alter to fit
your own needs. Change some global variables and you can make it work
for almost anything. I can't tell, but I'm guessing your code in Ruby
wouldn't allow for this? I'm not trying to contract you, only point out
that Emacs extensions are, as oposed to Vim extensions, very flexible
and well thought out. This does, of course, not mean that it can't be
done in Ruby. I just get the feeling that Lisp excels at this.
[stuff about functional vs. OO being more well suited for editing text]

I don't really find that. I don't think functional programming is any
easier for editor-related tasks. I'm not even sure how you would come
up with such an assumption.

My real point was that having OO around doesn't really help either. It
doesn't add anything. Sure, you can make classes like Buffer and Window,
were is the real gain? I have tried to envision some OO structure for
implementing Emacs like Major/Minor Modes and such, but I haven't been
able to come to any satisfactory results. I mean, how is a Major Mode
an object? Really? I guess it has a syntax definition, a separate
keybinding mapping, an indentation callback, maybe something else? I
just don't feel it adds anything though. I am, of course open to
suggestions ;-).

Right, tiny C core like emacs, everything higher-level in the language
of choice. Ruby is highly suited for this task.

Yeah, this is true. Ruby would be well suited for this I do believe.
But note that Emacs C core isn't very small ;-)

People could even write high-performance ruby extensions in C...

Yeah, this would be easy to do as well. There is, of course, the
inherent risk of not being portable enough. Vim supports this in a way,
and I have never seen it used to date.
[stuff about regular expressions]

Well, to be blunt, whatever you come up with won't be as popular or
useful as the existing regular expressions, just because they'll be a
nonstandard replacement of something already very common. PCRE
regexps are extremely flexible and well-known.

As useful? Please, my dear sir, there has to be something better than
the way we describe regular expressions now. At least for searching
text. The syntax we have today for regular expressions is basically the
same, only extended, as that that Ken Thompson uses in his 1968 paper on
it. Or that of _real_ regular expressions long before it. And
remember, real regular expressions only have * (Kleene star) and no +.
There has to be a simpler syntax that can be useful for interactive text
search-and-replaces. Look at Vim, Emacs, and Perl (and thus,
basically, Ruby)'s syntax. They are all extensions of this, adding new
short cryptic ways of saying things that you often don't need, and if
you did you wouldn't want to do it that way anyway. The real example of
how it has gotten out of hand is the overuse of backslash (\). It is
everywhere. having to move my hand to the upper right corner of my
keyboard all the time is a real pain.
Of course we'll have to see if I'm actually able to come up with
anything better. It's probably not going to be as easy as I'd like to
suggest here. However, look at the Perl 6 Apocalypse 5[1] to see one way
of moving away from cryptic (?:...) metasyntaxes.

That isn't to say people won't use them, especially if they're
simpler, but it probably won't be the main selling point of your
editor to _other people_.

Nah OK. You've got a point. But, as with most free software, this
one's for me ;-). If anyone wants to tag along later on, fine. But I
won't care if no one is interested, Emacs and Vim are fine editors.
Even notepad has its uses. It can, for example, tell you if a file is
smaller or greater than 65535 bytes very easily ;-).
I have, perhaps, failed to describe the real winning here. (Alas, I
realize I forgot to mention it.) As you perhaps know, Vim, and most
other UNIX software, operate on a line-by-line basis. This restriction
would not impede the command language I'm contemplating. If you take a
look at the Sam editor[2], this is its main selling point, and this is
another one I want to include.

I'm not sure how that's a problem. The same applies to // regexps.
They're just basically strings, except stored in a different type of
object with a few flags.

Nono, they don't do string escapes. \n in a regex (//) means match a
newline, not substitute this for 0x0a. So, you don't have to quote it
with an extra backslash to get that meaning. Eh, maybe I'm not making
myself clear. See, in Emacs, you have to write it as
"\\n"
since, otherwise you get the mentioned effect. This may be OK most of
the time, but it has implications. Also, if you want to match a word
boundary in Emacs you'll have to write
"\\<"
and to match a backslash itself:
"\\\\"
which is horrendous. In a Ruby regex, /\\/ suffices.
nikolai

[1] http://www.perl.com/pub/a/2002/06/04/apo5.html
[2] http://plan9.bell-labs.com/sys/doc/sam/sam.html or as PostScript:
http://plan9.bell-labs.com/sys/doc/sam/sam.ps

 

[Nikolai Weibull]

What a coincidence.. I am also writing a programmers editor (AEditor)
in Ruby.
http://aeditor.rubyforge.org/

Yes, I have seen it, and tried it (see below).

In my previous editor (slackedit/sled[3]) I used
Tcl[4] as an extension language, and it proved very easy to use, but I
never got far enough to actually give it a good run.

where is the homepage ? ;-)

eh, it was included in the end of the mail (marked [3]) ;-)
http://www.pcppopper.org/code/win/sled/

http://sourceforge.net/projects/slackedit

This is very old. Do not use ;-)

Your Trove-info says 'gnome', but no unix install.

No, the plan was to port it to Gtk/GLib, but that never happened.

My original plan were to use Ruby as an extension language and write
the core in C++... At some point started experimenting using Ruby for
everything, and now I have actually written AEditor completely in Ruby!!

Yeah, it is very nice and simply done. I like your code. I see you've
read your GOF well ;-) (especially the first chapter I presume ;-). I
do, however, feel that you've gone into it a bit too much. I mean, It's
fine to represent lines as object, but individual characters? It gets a
bit big at that point ;-)

undo/redo/macros.. The code which is responsible for this only takes up
119 lines of Ruby code. This is more complex in C++ because you cannot
#clone arbitrary objects.

Well, if you take a look at the undo/redo code for Wily and Sam, they
are actually not very large. Using the Command pattern is very clean
and OO though.

It would be easier for users of your editor to make changes
to the internals of your editor (because Ruby is intuitive).
If you have a testharness then they will also could test if
their changes has unforeseen sideeffects.

Intuitive in general, yes. But is it intuitive in this context. Are
there any obvious winnings by using it? Can you forsee syntax
definitions and indentation definitions easier to write in Ruby than in
LISP? Why? How?

not enough intuitive.

Hehe, OK ;-). I've just started getting interested in Lisp, shunning it
in the past as an Old-Fart Language. It just seemed to arcane for me.
Then I realized that it's a language that's had all these features we
only now see making their way into mainstream languages. I mean, look
what Matz has to say about it:
"Some may say Ruby is a bad rip-off of Lisp or Smalltalk, and I
admit that. But it is nicer to ordinary people."
- Matz, LL2
OK, this may be true. 1 + 2 is easier to read and understand than (+ 1
2), but you can't do 1 + 2 3 and expect it to work ((+ 1 2 3) does). I
guess, that, in the end I'm just pushing Lisp here because I am
interested in it and want to give it a try. Ruby is great, but I
already know Ruby ;-). I'm trying to get someone to push me back into
the Ruby camp :-D.

If your editor core is written in a different language than your extension
language. Then you will no matter what language chosen, have to make a
bridge between. There are shortcuts, such as SWIG.
SWIG can generate wrappers for *many* script languages.
My vote would be Ruby

Yeah, but I want the language that makes it simpler to move as much out
to the extension language as possible.

You can try out AEditor, to feel if Ruby is fast enough.
On a 100MHz machine AEditor feels quite Okay.
On a 33MHz 486 it can be slow.
I can live with this hardware requirements, thats why I choose to write
everything in Ruby

Yeah, I tried opening this rather large text file (In The Beginning Was
The Command Line) and it started thrashing my hard-drive (just loading
it (a file of perhaps 200K?) since it required 130MB to load it started
swapping.

My datastructure has a huge amount of memory overhead (because it uses an
Array of characters). I am in the process of rethinking the datastructure
and maybe write a datastructure Ruby-extension module in C++..?
OK.
Overloading Regexp... I don't know.
I am currently writing my own regex engine. OK, interesting.
What features do you plan for your editor ?
Is it a programmers editor, or a propertional font editor ?

It's a general editor, of course aimed at programming. Proportional
text is for whimps ;-)
nikolai

 

[Nikolai Weibull]

the learning curve for casual user would be really less hard, and it
won't collide with already existing lisp based editors

True, but this isn't going to be an editor for the casual user either
;-)

Well, given that tab expansion, string replacing/finding,
byte/word/line count, justification and other stuff can be easily done
with one line of ruby...

But remember that we're not going to operate on a File, or a String as
such.

Anyway, I think the apis you make available from the core of the
editor would be something really important in this analisys.

Yeah, but one could assume that they'd be the same for both languages.

Sure

Heh, short and to the point ;-).

well, you have the soruce code, dig in it and change what you need, if
you really need it

If I go with Ruby I don't want to alter the core of the language.

Anyway, why would you do such a thing ? the regex language is quite
standardized now, why would you change it?

Because the standardized regex language we have now sucks, like German,
or French. (Only kidding, only kidding. I love French. German sucks
though. (Am I still kidding?) ;-)
Seriously, see the earlier reply I wrote. I want simple, easy, yet
expressive. I don't want (?:...) and "\\\\".
nikolai

 

[Nikolai Weibull]

You can check it out here:
http://rubyforge.org/cgi-bin/cvsweb.cgi/projects/experimental/nfa_to_dfa/?cvsroot=aeditor

Looks interesting, I'll give it a deeper study later.
nikolai

 

[Nikolai Weibull]

There seem to be simpler/more elegant solutions to these problems than
reimplementing pcre:

You don't have to reimplement it, just change its syntax.

I'd say this is way simpler, more effective, and more extensible than
redefining all-new regexp syntax. Especially since the existing
syntax is hairy enough. ;-)

You should take a look at the Sam editor. It works like this in a way.
It has very, very simple regexes, but the way you get advanced is by
combining different regexes and operators to get to the real match.
nikolai