Extension Language for a Text Editor

Discussion in 'Ruby' started by Nikolai Weibull, Oct 8, 2003.

  1. OK. So I'm going to write a text editor for my masters' thesis. The
    general idea of it is fixed but the extension language has not been
    settled on, yet. I have thought of, in the footsteps of editors such as
    Emacs and its clones, using some version of LISP or Scheme, possibly
    GNU's Guile[1], or Jade's librep[2]. LISP-like languages have proved
    very successful in describing the way we edit text and is thus a sure
    bet. It would, however, perhaps be of interest to use another language
    to try the waters. In my previous editor (slackedit/sled[3]) I used
    Tcl[4] as an extension language, and it proved very easy to use, but I
    never got far enough to actually give it a good run.

    Anyway, what I'm getting at is:

    Do you figure Ruby to be a good extension language for a text editor?
    * What would be easier to do using Ruby?
    * LISP?
    * What language allows the most of the editing commands to be written in
    the given language?
    * Is Ruby good enough for the task at the moment, performance wise?
    * Resource wise? Does anyone have any statistics on this?

    I'm not talking about the ease of implementing a web browser in the
    target language, rather the ease of structuring an extensible framework
    in it.

    Also, is there any way of redefining the // operator for constructing
    regular expression objects? I'm planning on implementing a new regex
    syntax for the editor (to make searches/substitutions easier to
    describe).

    thanks in advance,
    nikolai

    [1] http://www.gnu.org/software/guile/
    [2] http://librep.sourceforge.net/
    [3] http://www.pcppopper.org/code/win/sled/
    [4] http://www.scriptics.com/

    --
    ::: name: Nikolai Weibull :: aliases: pcp / lone-star / aka :::
    ::: born: Chicago, IL USA :: loc atm: Gothenburg, Sweden :::
    ::: page: www.pcppopper.org :: fun atm: gf,lps,ruby,lisp,war3 :::
    main(){printf(&linux["\021%six\012\0"],(linux)["have"]+"fun"-97);}
    Nikolai Weibull, Oct 8, 2003
    #1
    1. Advertising

  2. Nikolai Weibull

    Ryan Pavlik Guest

    On Thu, 9 Oct 2003 05:06:32 +0900
    Nikolai Weibull <> wrote:

    > OK. So I'm going to write a text editor for my masters' thesis. The
    > general idea of it is fixed but the extension language has not been
    > settled on, yet.

    <snip>

    The thing with lisp, though, is that everyone has used lisp already,
    and while I love lisp, there wouldn't be much of a reason to switch
    from XEmacs and its twenty million extensions.

    > Anyway, what I'm getting at is:
    >
    > Do you figure Ruby to be a good extension language for a text
    > editor?


    YES. I've been wanting "erubs" (Editor for Ruby Scripts ;-) for
    awhile; something like emacs, but s/lisp/ruby. Same design otherwise.

    > * What would be easier to do using Ruby?


    Well, the main thing is that ruby has a lot of very convenient pattern
    and text matching functionality, and a boatload of extensions. And
    it's a very easy language, so anyone can pick it up and start
    extending the editor, unlike emacs, where fewer brave the waters.

    For instance, I recently wrote a bit of code to align things:

    a = b ___\ a = b
    foo = bar / foo = bar

    This was a few simple lines of ruby. Not counting comments, it's 44
    lines of lisp that don't quite work perfectly.

    > * LISP?


    As much as I love lisp, I can't really think of things that would be
    easier to in any of the above scheme flavors that wouldn't be easier
    in ruby.

    > * What language allows the most of the editing commands to be written in
    > the given language?


    Ruby certainly allows this. It's extremely trivial to extend and use
    Ruby in C. They go hand-in-hand. Plus there are existing GUI
    packages you could interface to if you wanted to provide UI handling
    in ruby without a lot of work.

    > * Is Ruby good enough for the task at the moment, performance wise?
    > * Resource wise? Does anyone have any statistics on this?


    This is a decent benchmark:

    http://www.bagley.org/~doug/shootout/craps.shtml

    Is ruby at the top of that list? No. (Note that most of the
    languages above it are compiled, including the bigloo scheme
    implementation). But XEmacs elisp falls near the bottom, and I use
    that for editing tasks daily. Tcl and Rep both fall below.

    So, I wouldn't worry about performance. And, memory-wise, I can't
    really offer benchmarks, but I have a system managing tens of
    thousands of objects that doesn't present a problem.

    > I'm not talking about the ease of implementing a web browser in the
    > target language, rather the ease of structuring an extensible framework
    > in it.


    Then Ruby is ideal. Extending and embedding it is extremely easy.
    It's a really great scripting framework, even though many people
    including myself use it for entire applications.

    > Also, is there any way of redefining the // operator for constructing
    > regular expression objects? I'm planning on implementing a new regex
    > syntax for the editor (to make searches/substitutions easier to
    > describe).


    No, but I would encourage you leave the existing regular expression
    format intact, as, while somewhat write-only, is familiar to most
    people who will be using your editor. Redefining // would break a lot
    of extension code, and alienate a large part of your audience.

    However, that's not to say you can't make your own matching format.
    I'm not sure what you have in mind, but there are a number of ways to
    bend the syntax to integrate such things.

    > thanks in advance,


    If you write a ruby editor... thank _you_. ;-)

    hth,

    --
    Ryan Pavlik <>

    "You'd be surprised what a platoon of heartless ninja lawyers
    can do in favor of a position." - 8BT
    Ryan Pavlik, Oct 8, 2003
    #2
    1. Advertising

  3. On Thu, 09 Oct 2003 06:06:32 +0900, Nikolai Weibull wrote:

    > OK. So I'm going to write a text editor for my masters' thesis.


    What a coincidence.. I am also writing a programmers editor (AEditor)
    in Ruby.
    http://aeditor.rubyforge.org/


    > In my previous editor (slackedit/sled[3]) I used
    > Tcl[4] as an extension language, and it proved very easy to use, but I
    > never got far enough to actually give it a good run.


    where is the homepage ? ;-)
    http://sourceforge.net/projects/slackedit
    Your Trove-info says 'gnome', but no unix install.


    > Do you figure Ruby to be a good extension language for a text editor?


    My original plan were to use Ruby as an extension language and write
    the core in C++... At some point started experimenting using Ruby for
    everything, and now I have actually written AEditor completely in Ruby!!


    > * What would be easier to do using Ruby?


    undo/redo/macros.. The code which is responsible for this only takes up
    119 lines of Ruby code. This is more complex in C++ because you cannot
    #clone arbitrary objects.

    unittesting: AEditor has about 440 testcases and 1400 assertions.
    unittesting is annoying in C++. In Ruby its easy.

    It would be easier for users of your editor to make changes
    to the internals of your editor (because Ruby is intuitive).
    If you have a testharness then they will also could test if
    their changes has unforeseen sideeffects.


    > * LISP?


    not enough intuitive.


    > * What language allows the most of the editing commands to be written in
    > the given language?


    If your editor core is written in a different language than your extension
    language. Then you will no matter what language chosen, have to make a
    bridge between. There are shortcuts, such as SWIG.
    SWIG can generate wrappers for *many* script languages.
    My vote would be Ruby :)


    > * Is Ruby good enough for the task at the moment, performance wise?


    You can try out AEditor, to feel if Ruby is fast enough.
    On a 100MHz machine AEditor feels quite Okay.
    On a 33MHz 486 it can be slow.
    I can live with this hardware requirements, thats why I choose to write
    everything in Ruby :)


    > * Resource wise? Does anyone have any statistics on this?


    My datastructure has a huge amount of memory overhead (because it uses an
    Array of characters). I am in the process of rethinking the datastructure
    and maybe write a datastructure Ruby-extension module in C++..?


    > Also, is there any way of redefining the // operator for constructing
    > regular expression objects? I'm planning on implementing a new regex
    > syntax for the editor (to make searches/substitutions easier to
    > describe).


    Overloading Regexp... I don't know.
    I am currently writing my own regex engine.



    What features do you plan for your editor ?
    Is it a programmers editor, or a propertional font editor ?

    --
    Simon Strandgaard
    Simon Strandgaard, Oct 8, 2003
    #3
  4. * Ryan Pavlik <> [Oct, 08 2003 22:30]:
    > The thing with lisp, though, is that everyone has used lisp already,
    > and while I love lisp, there wouldn't be much of a reason to switch
    > from XEmacs and its twenty million extensions.

    Yes, this is one of the downsides in a way. You won't be able to get
    people to switch very easily when everything is the same ;-)
    > YES. I've been wanting "erubs" (Editor for Ruby Scripts ;-) for
    > awhile; something like emacs, but s/lisp/ruby. Same design otherwise.

    hehe, that won't be the name, but yeah, that's in a sense what I want.
    Except s/Emacs/Vim/ ;-).
    > Well, the main thing is that ruby has a lot of very convenient pattern
    > and text matching functionality, and a boatload of extensions. And
    > it's a very easy language, so anyone can pick it up and start
    > extending the editor, unlike emacs, where fewer brave the waters.

    Well, no dependencies on Ruby extensions would be necesarry, but I
    guess, since they exist, they could be used.
    > For instance, I recently wrote a bit of code to align things:
    >
    > a = b ___\ a = b
    > foo = bar / foo = bar
    >
    > This was a few simple lines of ruby. Not counting comments, it's 44
    > lines of lisp that don't quite work perfectly.

    Could you provide the code? I'd love to see the comparison.
    > As much as I love lisp, I can't really think of things that would be
    > easier to in any of the above scheme flavors that wouldn't be easier
    > in ruby.

    OK. The way I see it, Ruby is OO-programming personified. LISP is,
    well, functional? programming personified. Anyway, I have gotten the
    feeling that it's generally easier to think in terms of functional, not
    OO, when editing text. I mean, what do tho OO constructs really add?
    >
    > > * What language allows the most of the editing commands to be written in
    > > the given language?

    >
    > Ruby certainly allows this. It's extremely trivial to extend and use
    > Ruby in C. They go hand-in-hand. Plus there are existing GUI
    > packages you could interface to if you wanted to provide UI handling
    > in ruby without a lot of work.

    Ah, I think you misunderstood the question. Or I'm misunderstanding the
    answer ;-). I want the C core to be as small as possible, leaving the
    most possible flexibility using the extension language. And I don't
    generally see the need for GUI extensions and such.
    > > * Is Ruby good enough for the task at the moment, performance wise?
    > > * Resource wise? Does anyone have any statistics on this?

    >
    > This is a decent benchmark:
    >
    > http://www.bagley.org/~doug/shootout/craps.shtml
    >
    > Is ruby at the top of that list? No. (Note that most of the
    > languages above it are compiled, including the bigloo scheme
    > implementation). But XEmacs elisp falls near the bottom, and I use
    > that for editing tasks daily. Tcl and Rep both fall below.

    Thanks, great summary!
    > So, I wouldn't worry about performance. And, memory-wise, I can't
    > really offer benchmarks, but I have a system managing tens of
    > thousands of objects that doesn't present a problem.

    OK. Good.
    > > I'm not talking about the ease of implementing a web browser in the
    > > target language, rather the ease of structuring an extensible framework
    > > in it.

    > Then Ruby is ideal. Extending and embedding it is extremely easy.

    Yeah, this, I know. My experiences from the Ruby-GNOME2 project have
    been great. Very easy to add extensions. I haven't used the embedding
    facilities yet, but I'm sure they're good.
    > > Also, is there any way of redefining the // operator for constructing
    > > regular expression objects? I'm planning on implementing a new regex
    > > syntax for the editor (to make searches/substitutions easier to
    > > describe).

    >
    > No, but I would encourage you leave the existing regular expression
    > format intact, as, while somewhat write-only, is familiar to most
    > people who will be using your editor. Redefining // would break a lot
    > of extension code, and alienate a large part of your audience.
    >
    > However, that's not to say you can't make your own matching format.
    > I'm not sure what you have in mind, but there are a number of ways to
    > bend the syntax to integrate such things.

    Yeah, but that's just the thing. This is one of the real selling
    points, if you will ;-). Oh well, I guess one can always do some
    MyRegex.new(string)
    but then you wind up with the string interpolation problems (\n and
    friends). :-(
    >
    > > thanks in advance,

    >
    > If you write a ruby editor... thank _you_. ;-)

    hehe, we'll see ;-)
    >
    > hth,
    >

    thanks for your long and quick response. It's good to know that at
    least someone is interested :),
    nikolai

    --
    ::: name: Nikolai Weibull :: aliases: pcp / lone-star / aka :::
    ::: born: Chicago, IL USA :: loc atm: Gothenburg, Sweden :::
    ::: page: www.pcppopper.org :: fun atm: gf,lps,ruby,lisp,war3 :::
    main(){printf(&linux["\021%six\012\0"],(linux)["have"]+"fun"-97);}
    Nikolai Weibull, Oct 8, 2003
    #4
  5. il Thu, 9 Oct 2003 05:06:32 +0900, Nikolai Weibull
    <> ha scritto::


    my 2c:

    >Anyway, what I'm getting at is:
    >
    >Do you figure Ruby to be a good extension language for a text editor?
    >* What would be easier to do using Ruby?


    the learning curve for casual user would be really less hard, and it
    won't collide with already existing lisp based editors ;)


    >* What language allows the most of the editing commands to be written in
    > the given language?

    Well, given that tab expansion, string replacing/finding,
    byte/word/line count, justification and other stuff can be easily done
    with one line of ruby...
    Anyway, I think the apis you make available from the core of the
    editor would be something really important in this analisys.

    >* Is Ruby good enough for the task at the moment, performance wise?

    Sure :)

    >* Resource wise? Does anyone have any statistics on this?

    dunno

    >Also, is there any way of redefining the // operator for constructing
    >regular expression objects? I'm planning on implementing a new regex
    >syntax for the editor (to make searches/substitutions easier to
    >describe).


    well, you have the soruce code, dig in it and change what you need, if
    you really need it :)
    Anyway, why would you do such a thing ? the regex language is quite
    standardized now, why would you change it?
    gabriele renzi, Oct 8, 2003
    #5
  6. --=-k1N1cbYdS6APwxuOu+Us
    Content-Type: text/plain
    Content-Transfer-Encoding: 7bit

    You may want to look at the VIM's use of Ruby for writing extensions.
    Documentation available inside of VIM via ":help ruby".

    For an up-to-date copy of VIM see: http://www.vim.org

    --Dale

    --=-k1N1cbYdS6APwxuOu+Us
    Content-Type: text/html; charset=utf-8

    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
    <HTML>
    <HEAD>
    <META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
    <META NAME="GENERATOR" CONTENT="GtkHTML/1.0.2">
    </HEAD>
    <BODY>
    You may want to look at the VIM's use of Ruby for writing extensions. Documentation available inside of VIM via &quot;:help ruby&quot;.
    <BR>

    <BR>
    For an up-to-date copy of VIM see: <A HREF="http://www.vim.org">http://www.vim.org</A>
    <BR>

    <BR>
    --Dale
    </BODY>
    </HTML>

    --=-k1N1cbYdS6APwxuOu+Us--
    Dale Martenson, Oct 8, 2003
    #6
  7. Nikolai Weibull

    Ryan Pavlik Guest

    On Thu, 9 Oct 2003 06:09:29 +0900
    Nikolai Weibull <> wrote:

    > * Ryan Pavlik <> [Oct, 08 2003 22:30]:


    > > YES. I've been wanting "erubs" (Editor for Ruby Scripts ;-) for
    > > awhile; something like emacs, but s/lisp/ruby. Same design
    > > otherwise.


    > hehe, that won't be the name, but yeah, that's in a sense what I want.
    > Except s/Emacs/Vim/ ;-).


    Well, whichever, just be aware that emacs is designed as an extensible
    editor, and vim is not, even though such things have crept in with
    varying degrees of usefulness.

    > > Well, the main thing is that ruby has a lot of very convenient pattern
    > > and text matching functionality, and a boatload of extensions. And
    > > it's a very easy language, so anyone can pick it up and start
    > > extending the editor, unlike emacs, where fewer brave the waters.


    > Well, no dependencies on Ruby extensions would be necesarry, but I
    > guess, since they exist, they could be used.


    Right. They're there, people can write extensions that interface to
    the web, or whatever.

    > > This was a few simple lines of ruby. Not counting comments, it's 44
    > > lines of lisp that don't quite work perfectly.


    > Could you provide the code? I'd love to see the comparison.


    See http://ogmo.mephle.org/tabular-alignment.org for the Lisp
    version. The ruby one I deleted, as it was pretty simple to
    reproduce, I'm sure someoone can whip up an example.

    > > As much as I love lisp, I can't really think of things that would be
    > > easier to in any of the above scheme flavors that wouldn't be easier
    > > in ruby.


    > OK. The way I see it, Ruby is OO-programming personified. LISP is,
    > well, functional? programming personified. Anyway, I have gotten the
    > feeling that it's generally easier to think in terms of functional, not
    > OO, when editing text. I mean, what do tho OO constructs really
    > add?


    I don't really find that. I don't think functional programming is any
    easier for editor-related tasks. I'm not even sure how you would come
    up with such an assumption. ;)

    > > Ruby certainly allows this. It's extremely trivial to extend and use
    > > Ruby in C. They go hand-in-hand. Plus there are existing GUI
    > > packages you could interface to if you wanted to provide UI handling
    > > in ruby without a lot of work.


    > Ah, I think you misunderstood the question. Or I'm misunderstanding the
    > answer ;-). I want the C core to be as small as possible, leaving the
    > most possible flexibility using the extension language. And I don't
    > generally see the need for GUI extensions and such.


    Right, tiny C core like emacs, everything higher-level in the language
    of choice. Ruby is highly suited for this task.

    People could even write high-performance ruby extensions in C...

    <snip>
    > > However, that's not to say you can't make your own matching format.
    > > I'm not sure what you have in mind, but there are a number of ways to
    > > bend the syntax to integrate such things.


    > Yeah, but that's just the thing. This is one of the real selling
    > points, if you will ;-). Oh well, I guess one can always do some


    Well, to be blunt, whatever you come up with won't be as popular or
    useful as the existing regular expressions, just because they'll be a
    nonstandard replacement of something already very common. PCRE
    regexps are extremely flexible and well-known.

    That isn't to say people won't use them, especially if they're
    simpler, but it probably won't be the main selling point of your
    editor to _other people_.

    > MyRegex.new(string)


    > but then you wind up with the string interpolation problems (\n and
    > friends). :-(


    I'm not sure how that's a problem. The same applies to // regexps.
    They're just basically strings, except stored in a different type of
    object with a few flags.

    hth,

    --
    Ryan Pavlik <>

    "You'd be surprised what a platoon of heartless ninja lawyers
    can do in favor of a position." - 8BT
    Ryan Pavlik, Oct 8, 2003
    #7
  8. On Wed, 08 Oct 2003 22:39:13 +0000, gabriele renzi wrote:

    > Anyway, why would you do such a thing ? the regex language is quite
    > standardized now, why would you change it?


    I am in the process of writing my own regex engine (for AEditor).
    The reason why I cannot use Ruby's are, that I must run regex's on
    a wide range of iterators.

    I want to use same syntax as Ruby's regex, and perhaps extend it
    with some editor stuff, perhaps like:
    * how to traverse the datastructure, jump over folds, jump into folds.
    * regex on columnar selections.
    * mechanism for getting the syntax-state: perhaps a regex which
    only operates inside comments, apply regex to variable-names, etc.

    Status for my regex project:
    DONE studies of how Rubys GNU-regex engine behave.
    DONE nfa 2 dfa.
    DONE regex-datastructure 2 nfa.
    TODO translate regex-string into regex-datastructure

    There is probably some item which I have overseen.

    You can check it out here:
    http://rubyforge.org/cgi-bin/cvsweb.cgi/projects/experimental/nfa_to_dfa/?cvsroot=aeditor

    --
    Simon Strandgaard
    Simon Strandgaard, Oct 8, 2003
    #8
  9. Well, for one thing, doing it in lisp would be reinventing the wheel, as
    emacs has been around for ages.

    One main advantage that I see to ruby is that one could modify existing
    code and add new code much more easily if the modifier doesn't have a
    lisp background. lisp code is horribly unreadable.

    If you do want to use a functional language Haskell is a much better
    choice, and would in fact be interesting in this context.

    Nikolai Weibull wrote:

    >OK. So I'm going to write a text editor for my masters' thesis. The
    >general idea of it is fixed but the extension language has not been
    >settled on, yet. I have thought of, in the footsteps of editors such as
    >Emacs and its clones, using some version of LISP or Scheme, possibly
    >GNU's Guile[1], or Jade's librep[2]. LISP-like languages have proved
    >very successful in describing the way we edit text and is thus a sure
    >bet. It would, however, perhaps be of interest to use another language
    >to try the waters. In my previous editor (slackedit/sled[3]) I used
    >Tcl[4] as an extension language, and it proved very easy to use, but I
    >never got far enough to actually give it a good run.
    >
    >Anyway, what I'm getting at is:
    >
    >Do you figure Ruby to be a good extension language for a text editor?
    >* What would be easier to do using Ruby?
    >* LISP?
    >* What language allows the most of the editing commands to be written in
    > the given language?
    >* Is Ruby good enough for the task at the moment, performance wise?
    >* Resource wise? Does anyone have any statistics on this?
    >
    >I'm not talking about the ease of implementing a web browser in the
    >target language, rather the ease of structuring an extensible framework
    >in it.
    >
    >Also, is there any way of redefining the // operator for constructing
    >regular expression objects? I'm planning on implementing a new regex
    >syntax for the editor (to make searches/substitutions easier to
    >describe).
    >
    >thanks in advance,
    > nikolai
    >
    >[1] http://www.gnu.org/software/guile/
    >[2] http://librep.sourceforge.net/
    >[3] http://www.pcppopper.org/code/win/sled/
    >[4] http://www.scriptics.com/
    >
    >--
    >::: name: Nikolai Weibull :: aliases: pcp / lone-star / aka :::
    >::: born: Chicago, IL USA :: loc atm: Gothenburg, Sweden :::
    >::: page: www.pcppopper.org :: fun atm: gf,lps,ruby,lisp,war3 :::
    >main(){printf(&linux["\021%six\012\0"],(linux)["have"]+"fun"-97);}
    >
    >
    >.
    >
    >
    >



    --
    Seth Kurtzberg
    CTO
    Scottsdale Research and Network Operations Center
    ISEC.us
    Seth Kurtzberg, Oct 8, 2003
    #9
  10. Nikolai Weibull

    Ryan Pavlik Guest

    On Thu, 9 Oct 2003 07:10:42 +0900
    Simon Strandgaard <> wrote:

    <snip>
    > I want to use same syntax as Ruby's regex, and perhaps extend it
    > with some editor stuff, perhaps like:


    There seem to be simpler/more elegant solutions to these problems than
    reimplementing pcre:

    > * how to traverse the datastructure, jump over folds, jump into
    > folds.
    > * regex on columnar selections.
    > * mechanism for getting the syntax-state: perhaps a regex which
    > only operates inside comments, apply regex to variable-names, etc.


    All three of these can easily be accomplished by providing constrained
    input to the matcher. For folds etc, first you grab the pieces you
    want (folds, no folds), then apply the re. For colums, same deal.
    Extract the column, then apply. I'm sure you can guess where this is
    going for the last one... extract and apply the regexp only to
    comments or variable names.

    You could have a simple "Region" class which selects the appropriate
    areas, make an array of applicable regions, and then apply the regexp
    to each. This gives you three exceedingly simple tasks:

    1) Region objects defined by <start> and <end>.
    2) Regexp application to a region.
    3) Matching code to produce Regions. (Obviously, you'll need
    this in any case, to select only comments, etc.)

    I'd say this is way simpler, more effective, and more extensible than
    redefining all-new regexp syntax. Especially since the existing
    syntax is hairy enough. ;-)

    --
    Ryan Pavlik <>

    "You'd be surprised what a platoon of heartless ninja lawyers
    can do in favor of a position." - 8BT
    Ryan Pavlik, Oct 8, 2003
    #10
  11. On Thursday, October 9, 2003, 7:59:07 AM, Dale wrote:

    > You may want to look at the VIM's use of Ruby for writing extensions.
    > Documentation available inside of VIM via ":help ruby".


    > For an up-to-date copy of VIM see: http://www.vim.org


    There's a little discussion of this at

    http://www.rubygarden.org/ruby?VimRubyInterface

    This capability of Vim doesn't seem to be heavily used. I certainly
    don't use it. Partially because the integration is a bit lacking (but
    you could do Ryan's aligner very easily - but then there's an
    excellent and well-supported vim plugin for that anyway). Partially
    because I couldn't be bothered.

    Gavin
    Gavin Sinclair, Oct 9, 2003
    #11
  12. Ryan Pavlik <> skrev i en
    nyhedsmeddelelse:...
    > On Thu, 9 Oct 2003 07:10:42 +0900
    > Simon Strandgaard <> wrote:
    >
    > <snip>
    > > I want to use same syntax as Ruby's regex, and perhaps extend it
    > > with some editor stuff, perhaps like:

    >
    > There seem to be simpler/more elegant solutions to these problems than
    > reimplementing pcre:


    Agree, it may be sufficiently to only replace "regex.c/re_search()" with
    something which can operate on an iterator-range. But I am not that strong
    in the internals of Ruby and
    I prefer C++ over C. I rather make a full-blown regex engine in Ruby than
    interfacing to GNU regex in C. It may take longer, but the reward is many:
    * exercise my regex knowledge.
    * a regex engine with Ruby license.
    I am not interested in doing what may be the easiest. I just do what I like
    :)


    > > * how to traverse the datastructure, jump over folds, jump into
    > > folds.
    > > * regex on columnar selections.
    > > * mechanism for getting the syntax-state: perhaps a regex which
    > > only operates inside comments, apply regex to variable-names, etc.

    >
    > All three of these can easily be accomplished by providing constrained
    > input to the matcher. For folds etc, first you grab the pieces you
    > want (folds, no folds), then apply the re. For colums, same deal.
    > Extract the column, then apply. I'm sure you can guess where this is
    > going for the last one... extract and apply the regexp only to
    > comments or variable names.


    Your are talking about traversing, and you are absolutly right..
    But I am talking about the syntax of the regex. Misunderstanding :)

    [snip more text about traversing]
    BTW: I use many different kinds of iterators for traversing.


    > I'd say this is way simpler, more effective, and more extensible than
    > redefining all-new regexp syntax. Especially since the existing
    > syntax is hairy enough. ;-)


    Yes this isn't going to be easy. I am the kind of type which like hairy
    problems :)

    --
    Simon Strandgaard
    Simon Strandgaard, Oct 9, 2003
    #12
  13. "Nikolai Weibull" <> schrieb im Newsbeitrag
    news:...

    Since others have commented on various aspects of your posting I'll only
    throw my 2c at this:

    > Also, is there any way of redefining the // operator for constructing
    > regular expression objects? I'm planning on implementing a new regex
    > syntax for the editor (to make searches/substitutions easier to
    > describe).


    Why do you want to change it? I find it quite flexible and expressive
    (especially when using flag "x"). And especially if you want to attract
    rubyists to use TREC (The Ruby Editor to Come), then you should ensure
    that their knowledge of ruby regexps is not obsoleted, since rx's will
    likely play an important role in user defined extensions!

    You can always define methods in Kernel like this:

    irb(main):002:0* module Kernel
    irb(main):003:1> def rx(str)
    irb(main):004:2> puts "building rx from '#{str}'"
    irb(main):005:2> end
    irb(main):006:1> end
    => nil
    irb(main):007:0>
    irb(main):008:0* rx %q{(foo)+ \s+ (\w+)}
    building rx from '(foo)+ \s+ (\w+)'
    => nil
    irb(main):009:0> rx '(foo)+ \s+ (\w+)'
    building rx from '(foo)+ \s+ (\w+)'
    => nil
    irb(main):010:0>

    Which isn't too bad IMHO.

    Regards

    robert
    Robert Klemme, Oct 9, 2003
    #13
  14. Simon Strandgaard <> writes:

    > On Thu, 09 Oct 2003 06:06:32 +0900, Nikolai Weibull wrote:
    > > * What would be easier to do using Ruby?

    >
    > undo/redo/macros.. The code which is responsible for this only takes up
    > 119 lines of Ruby code. This is more complex in C++ because you cannot
    > #clone arbitrary objects.


    If you are talking about user-defined class hierarchies, you *can* clone
    them, as well - it just requires extra work:

    class Base {
    virtual Base* clone() = 0;
    };

    class Derived1: public Base {
    virtual Base* clone() { return new Derived1(*this); }
    };

    class Derived2: public Base {
    virtual Base* clone() { return new Derived2(*this); }
    };

    > unittesting: AEditor has about 440 testcases and 1400 assertions.
    > unittesting is annoying in C++. In Ruby its easy.


    Why is unittesting in C++ annoying? I have an application with 200+
    testcases and 400+ assertions, written completely in C++.
    The only thing that annoys me is that I can't easily add/remove tests
    without recompiling - some day I'll have to write a ruby wrapper to do this :)

    kind regards
    frank

    --
    Frank Schmitt
    4SC AG phone: +49 89 700763-0
    e-mail: frankNO DOT SPAMschmitt AT 4sc DOT com
    Frank Schmitt, Oct 9, 2003
    #14
  15. Frank Schmitt <> skrev i en
    nyhedsmeddelelse:4cfzi2lod6.fsf@scxw21.4sc...
    > Simon Strandgaard <> writes:
    >
    > > On Thu, 09 Oct 2003 06:06:32 +0900, Nikolai Weibull wrote:
    > > > * What would be easier to do using Ruby?

    > >
    > > undo/redo/macros.. The code which is responsible for this only takes up
    > > 119 lines of Ruby code. This is more complex in C++ because you cannot
    > > #clone arbitrary objects.

    >
    > If you are talking about user-defined class hierarchies, you *can* clone
    > them, as well - it just requires extra work:


    I know.. cloning in C++ get tiresome at that point when you have
    many childred.

    > class Base {
    > virtual Base* clone() = 0;
    > };
    >
    > class Derived1: public Base {
    > virtual Base* clone() { return new Derived1(*this); }
    > };


    This method requires typecasting if you want to do something like this:

    Derived *a = new Derived1();
    Derived *b = dynamic_cast<Derived*>(a->clone());


    Covariant return types is unfortunatly a relative new thing in the C++
    world.

    class Derived1: public Base {
    virtual Derived1* clone() { return new Derived1(*this); }
    };

    Derived *a = new Derived1();
    Derived *b = a->clone();


    > > unittesting: AEditor has about 440 testcases and 1400 assertions.
    > > unittesting is annoying in C++. In Ruby its easy.

    >
    > Why is unittesting in C++ annoying? I have an application with 200+
    > testcases and 400+ assertions, written completely in C++.
    > The only thing that annoys me is that I can't easily add/remove tests
    > without recompiling - some day I'll have to write a ruby wrapper to do

    this :)

    Compiling is expensive on my Pentium350 which is the fastest machine I have.
    GCC2.96 was quite fast... I have not yet seen any fast result with GCC3.
    I like templates, I like many things in C++. If I had a faster machine, then
    there would be no problems :)

    --
    Simon Strandgaard
    Simon Strandgaard, Oct 9, 2003
    #15
  16. * Ryan Pavlik <> [Oct, 09 2003 09:10]:
    [stuff about Emacs and Vim]
    > Well, whichever, just be aware that emacs is designed as an extensible
    > editor, and vim is not, even though such things have crept in with
    > varying degrees of usefulness.

    Yes, that, as I see it, is Emacs biggest win. The big problem I see
    with Vim is that it hasn't undergone a major overhaul. It is basically
    Emacs written in C now. Vim is, in my opinion, probably the best editor
    that exists right now. It is, however, going to reach a point where
    adding new features will demand some sort of rewrite. With Emacs,
    things like these are easy to alter. Anyway, Vim is extensible, it is,
    however, not _changable_. By that I mean that, if you want to change
    the way folding works, you must rewrite the core of Vim. And, as Bram
    pointed out in some interview, that means altering basically every file
    in the Vim distribution. To Vim's salvation comes the ability to easily
    define new syntax definitions and indentation definitions, which one has
    to agree, are a lot easier to create and alter than with Emacs (Emacs
    being perhaps more powerful though).
    [stuff about Ruby libraries and such]
    > Right. They're there, people can write extensions that interface to
    > the web, or whatever.

    Yes, but I don't want an Operating System. I guess, to a certain degree
    the library you get helps you, but it can also detract you from the
    central topic, namely editing text.
    > See http://ogmo.mephle.org/tabular-alignment.org for the Lisp
    > version. The ruby one I deleted, as it was pretty simple to
    > reproduce, I'm sure someoone can whip up an example.

    Yeah, OK. I see your point. It is, however, very easy to alter to fit
    your own needs. Change some global variables and you can make it work
    for almost anything. I can't tell, but I'm guessing your code in Ruby
    wouldn't allow for this? I'm not trying to contract you, only point out
    that Emacs extensions are, as oposed to Vim extensions, very flexible
    and well thought out. This does, of course, not mean that it can't be
    done in Ruby. I just get the feeling that Lisp excels at this.
    [stuff about functional vs. OO being more well suited for editing text]
    > I don't really find that. I don't think functional programming is any
    > easier for editor-related tasks. I'm not even sure how you would come
    > up with such an assumption. ;)

    My real point was that having OO around doesn't really help either. It
    doesn't add anything. Sure, you can make classes like Buffer and Window,
    were is the real gain? I have tried to envision some OO structure for
    implementing Emacs like Major/Minor Modes and such, but I haven't been
    able to come to any satisfactory results. I mean, how is a Major Mode
    an object? Really? I guess it has a syntax definition, a separate
    keybinding mapping, an indentation callback, maybe something else? I
    just don't feel it adds anything though. I am, of course open to
    suggestions ;-).
    > Right, tiny C core like emacs, everything higher-level in the language
    > of choice. Ruby is highly suited for this task.

    Yeah, this is true. Ruby would be well suited for this I do believe.
    But note that Emacs C core isn't very small ;-)
    > People could even write high-performance ruby extensions in C...

    Yeah, this would be easy to do as well. There is, of course, the
    inherent risk of not being portable enough. Vim supports this in a way,
    and I have never seen it used to date.
    [stuff about regular expressions]
    > Well, to be blunt, whatever you come up with won't be as popular or
    > useful as the existing regular expressions, just because they'll be a
    > nonstandard replacement of something already very common. PCRE
    > regexps are extremely flexible and well-known.

    As useful? Please, my dear sir, there has to be something better than
    the way we describe regular expressions now. At least for searching
    text. The syntax we have today for regular expressions is basically the
    same, only extended, as that that Ken Thompson uses in his 1968 paper on
    it. Or that of _real_ regular expressions long before it. And
    remember, real regular expressions only have * (Kleene star) and no +.
    There has to be a simpler syntax that can be useful for interactive text
    search-and-replaces. Look at Vim, Emacs, and Perl (and thus,
    basically, Ruby)'s syntax. They are all extensions of this, adding new
    short cryptic ways of saying things that you often don't need, and if
    you did you wouldn't want to do it that way anyway. The real example of
    how it has gotten out of hand is the overuse of backslash (\). It is
    everywhere. having to move my hand to the upper right corner of my
    keyboard all the time is a real pain.
    Of course we'll have to see if I'm actually able to come up with
    anything better. It's probably not going to be as easy as I'd like to
    suggest here. However, look at the Perl 6 Apocalypse 5[1] to see one way
    of moving away from cryptic (?:...) metasyntaxes.
    > That isn't to say people won't use them, especially if they're
    > simpler, but it probably won't be the main selling point of your
    > editor to _other people_.

    Nah OK. You've got a point. But, as with most free software, this
    one's for me ;-). If anyone wants to tag along later on, fine. But I
    won't care if no one is interested, Emacs and Vim are fine editors.
    Even notepad has its uses. It can, for example, tell you if a file is
    smaller or greater than 65535 bytes very easily ;-).
    I have, perhaps, failed to describe the real winning here. (Alas, I
    realize I forgot to mention it.) As you perhaps know, Vim, and most
    other UNIX software, operate on a line-by-line basis. This restriction
    would not impede the command language I'm contemplating. If you take a
    look at the Sam editor[2], this is its main selling point, and this is
    another one I want to include.
    > > but then you wind up with the string interpolation problems (\n and
    > > friends). :-(

    > I'm not sure how that's a problem. The same applies to // regexps.
    > They're just basically strings, except stored in a different type of
    > object with a few flags.

    Nono, they don't do string escapes. \n in a regex (//) means match a
    newline, not substitute this for 0x0a. So, you don't have to quote it
    with an extra backslash to get that meaning. Eh, maybe I'm not making
    myself clear. See, in Emacs, you have to write it as
    "\\n"
    since, otherwise you get the mentioned effect. This may be OK most of
    the time, but it has implications. Also, if you want to match a word
    boundary in Emacs you'll have to write
    "\\<"
    and to match a backslash itself:
    "\\\\"
    which is horrendous. In a Ruby regex, /\\/ suffices.
    nikolai

    [1] http://www.perl.com/pub/a/2002/06/04/apo5.html
    [2] http://plan9.bell-labs.com/sys/doc/sam/sam.html or as PostScript:
    http://plan9.bell-labs.com/sys/doc/sam/sam.ps

    --
    ::: name: Nikolai Weibull :: aliases: pcp / lone-star / aka :::
    ::: born: Chicago, IL USA :: loc atm: Gothenburg, Sweden :::
    ::: page: www.pcppopper.org :: fun atm: gf,lps,ruby,lisp,war3 :::
    main(){printf(&linux["\021%six\012\0"],(linux)["have"]+"fun"-97);}
    Nikolai Weibull, Oct 9, 2003
    #16
  17. * Simon Strandgaard <> [Oct, 09 2003 09:09]:
    > What a coincidence.. I am also writing a programmers editor (AEditor)
    > in Ruby.
    > http://aeditor.rubyforge.org/

    Yes, I have seen it, and tried it (see below).
    >
    > > In my previous editor (slackedit/sled[3]) I used
    > > Tcl[4] as an extension language, and it proved very easy to use, but I
    > > never got far enough to actually give it a good run.

    >
    > where is the homepage ? ;-)

    eh, it was included in the end of the mail (marked [3]) ;-)
    http://www.pcppopper.org/code/win/sled/
    > http://sourceforge.net/projects/slackedit

    This is very old. Do not use ;-)
    > Your Trove-info says 'gnome', but no unix install.

    No, the plan was to port it to Gtk/GLib, but that never happened.
    >
    > > Do you figure Ruby to be a good extension language for a text editor?

    >
    > My original plan were to use Ruby as an extension language and write
    > the core in C++... At some point started experimenting using Ruby for
    > everything, and now I have actually written AEditor completely in Ruby!!
    >

    Yeah, it is very nice and simply done. I like your code. I see you've
    read your GOF well ;-) (especially the first chapter I presume ;-). I
    do, however, feel that you've gone into it a bit too much. I mean, It's
    fine to represent lines as object, but individual characters? It gets a
    bit big at that point ;-)
    >
    > > * What would be easier to do using Ruby?

    >
    > undo/redo/macros.. The code which is responsible for this only takes up
    > 119 lines of Ruby code. This is more complex in C++ because you cannot
    > #clone arbitrary objects.
    >

    Well, if you take a look at the undo/redo code for Wily and Sam, they
    are actually not very large. Using the Command pattern is very clean
    and OO though.
    >
    > It would be easier for users of your editor to make changes
    > to the internals of your editor (because Ruby is intuitive).
    > If you have a testharness then they will also could test if
    > their changes has unforeseen sideeffects.
    >

    Intuitive in general, yes. But is it intuitive in this context. Are
    there any obvious winnings by using it? Can you forsee syntax
    definitions and indentation definitions easier to write in Ruby than in
    LISP? Why? How?
    >
    > > * LISP?

    >
    > not enough intuitive.
    >

    Hehe, OK ;-). I've just started getting interested in Lisp, shunning it
    in the past as an Old-Fart Language. It just seemed to arcane for me.
    Then I realized that it's a language that's had all these features we
    only now see making their way into mainstream languages. I mean, look
    what Matz has to say about it:
    "Some may say Ruby is a bad rip-off of Lisp or Smalltalk, and I
    admit that. But it is nicer to ordinary people."
    - Matz, LL2
    OK, this may be true. 1 + 2 is easier to read and understand than (+ 1
    2), but you can't do 1 + 2 3 and expect it to work ((+ 1 2 3) does). I
    guess, that, in the end I'm just pushing Lisp here because I am
    interested in it and want to give it a try. Ruby is great, but I
    already know Ruby ;-). I'm trying to get someone to push me back into
    the Ruby camp :-D.
    >
    > > * What language allows the most of the editing commands to be written in
    > > the given language?

    >
    > If your editor core is written in a different language than your extension
    > language. Then you will no matter what language chosen, have to make a
    > bridge between. There are shortcuts, such as SWIG.
    > SWIG can generate wrappers for *many* script languages.
    > My vote would be Ruby :)
    >

    Yeah, but I want the language that makes it simpler to move as much out
    to the extension language as possible.
    >
    > > * Is Ruby good enough for the task at the moment, performance wise?

    >
    > You can try out AEditor, to feel if Ruby is fast enough.
    > On a 100MHz machine AEditor feels quite Okay.
    > On a 33MHz 486 it can be slow.
    > I can live with this hardware requirements, thats why I choose to write
    > everything in Ruby :)
    >

    Yeah, I tried opening this rather large text file (In The Beginning Was
    The Command Line) and it started thrashing my hard-drive (just loading
    it (a file of perhaps 200K?) since it required 130MB to load it started
    swapping.
    >
    > > * Resource wise? Does anyone have any statistics on this?

    >
    > My datastructure has a huge amount of memory overhead (because it uses an
    > Array of characters). I am in the process of rethinking the datastructure
    > and maybe write a datastructure Ruby-extension module in C++..?
    >

    OK.
    > Overloading Regexp... I don't know.
    > I am currently writing my own regex engine.

    OK, interesting.
    > What features do you plan for your editor ?
    > Is it a programmers editor, or a propertional font editor ?

    It's a general editor, of course aimed at programming. Proportional
    text is for whimps ;-)
    nikolai

    --
    ::: name: Nikolai Weibull :: aliases: pcp / lone-star / aka :::
    ::: born: Chicago, IL USA :: loc atm: Gothenburg, Sweden :::
    ::: page: www.pcppopper.org :: fun atm: gf,lps,ruby,lisp,war3 :::
    main(){printf(&linux["\021%six\012\0"],(linux)["have"]+"fun"-97);}
    Nikolai Weibull, Oct 9, 2003
    #17
  18. * gabriele renzi <> [Oct, 09 2003 09:10]:
    > the learning curve for casual user would be really less hard, and it
    > won't collide with already existing lisp based editors ;)

    True, but this isn't going to be an editor for the casual user either
    ;-)
    > Well, given that tab expansion, string replacing/finding,
    > byte/word/line count, justification and other stuff can be easily done
    > with one line of ruby...

    But remember that we're not going to operate on a File, or a String as
    such.
    > Anyway, I think the apis you make available from the core of the
    > editor would be something really important in this analisys.

    Yeah, but one could assume that they'd be the same for both languages.
    > >* Is Ruby good enough for the task at the moment, performance wise?

    > Sure :)

    Heh, short and to the point ;-).
    > >Also, is there any way of redefining the // operator for constructing
    > >regular expression objects? I'm planning on implementing a new regex
    > >syntax for the editor (to make searches/substitutions easier to
    > >describe).

    > well, you have the soruce code, dig in it and change what you need, if
    > you really need it :)

    If I go with Ruby I don't want to alter the core of the language.
    > Anyway, why would you do such a thing ? the regex language is quite
    > standardized now, why would you change it?

    Because the standardized regex language we have now sucks, like German,
    or French. (Only kidding, only kidding. I love French. German sucks
    though. (Am I still kidding?) ;-)
    Seriously, see the earlier reply I wrote. I want simple, easy, yet
    expressive. I don't want (?:...) and "\\\\".
    nikolai

    --
    ::: name: Nikolai Weibull :: aliases: pcp / lone-star / aka :::
    ::: born: Chicago, IL USA :: loc atm: Gothenburg, Sweden :::
    ::: page: www.pcppopper.org :: fun atm: gf,lps,ruby,lisp,war3 :::
    main(){printf(&linux["\021%six\012\0"],(linux)["have"]+"fun"-97);}
    Nikolai Weibull, Oct 9, 2003
    #18
  19. Nikolai Weibull, Oct 9, 2003
    #19
  20. * Ryan Pavlik <> [Oct, 09 2003 09:10]:
    > There seem to be simpler/more elegant solutions to these problems than
    > reimplementing pcre:

    You don't have to reimplement it, just change its syntax.
    > I'd say this is way simpler, more effective, and more extensible than
    > redefining all-new regexp syntax. Especially since the existing
    > syntax is hairy enough. ;-)

    You should take a look at the Sam editor. It works like this in a way.
    It has very, very simple regexes, but the way you get advanced is by
    combining different regexes and operators to get to the real match.
    nikolai

    --
    ::: name: Nikolai Weibull :: aliases: pcp / lone-star / aka :::
    ::: born: Chicago, IL USA :: loc atm: Gothenburg, Sweden :::
    ::: page: www.pcppopper.org :: fun atm: gf,lps,ruby,lisp,war3 :::
    main(){printf(&linux["\021%six\012\0"],(linux)["have"]+"fun"-97);}
    Nikolai Weibull, Oct 9, 2003
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Stylus Studio
    Replies:
    0
    Views:
    622
    Stylus Studio
    Aug 3, 2004
  2. Anand
    Replies:
    3
    Views:
    866
    Tim Daneliuk
    Nov 8, 2003
  3. Christian Seberino
    Replies:
    3
    Views:
    1,157
    Christian Seberino
    Feb 5, 2004
  4. Steve
    Replies:
    3
    Views:
    26,926
    Steve
    Aug 25, 2006
  5. Elhanan
    Replies:
    0
    Views:
    534
    Elhanan
    Jan 23, 2007
Loading...

Share This Page