Most popular wiki in Ruby seeks kind maintainer

Discussion in 'Ruby' started by David Heinemeier Hansson, Nov 23, 2004.

  1. Okay, okay, okay. Instiki does need a new maintainer. At least a
    temporary steward until I'm ready to resume that role. Are you it?

    The kick: Instiki just surpassed 8,000 downloads on RubyForge, which
    makes it the most downloaded piece of software there if you don't count
    Ruby itself.

    But Instiki has special needs, which is why I've been hesitant to call
    for a maintainer. See, Instiki needs to be and do less. But do it
    better.

    So while all the pending patches are pretty cool in one aspect or
    another, they're mostly about adding stuff to Instiki.

    Here's what I'd like a new Instiki maintainer to work on:

    * Fix the bugs. There are quite a few. Especially render bugs.

    * Switching from Marshal to YAML outputs. The biggest fear in Instiki
    is that something horrible goes wrong with the persisted data and all
    is lost in a big ball of Marshalled, binary mud. Having a humanly
    readable format would be very nice. This needs A LOT of testing,
    though.

    * Image uploads... perhaps even file uploads. This one is really
    tricky, though. It's very easy to create a big honking thing. We need
    an Instiki approach to the problem. Bill Atkins and I already had a few
    thoughts on the subject.

    Who's willing for glory and honor?
    --
    David Heinemeier Hansson,
    http://www.basecamphq.com/ -- Web-based Project Management
    http://www.rubyonrails.org/ -- Web-application framework for Ruby
    http://macromates.com/ -- TextMate: Code and markup editor (OS X)
    http://www.loudthinking.com/ -- Broadcasting Brain
    David Heinemeier Hansson, Nov 23, 2004
    #1
    1. Advertising

  2. David Heinemeier Hansson wrote:

    > Okay, okay, okay. Instiki does need a new maintainer. At least a
    > temporary steward until I'm ready to resume that role. Are you it?



    David,

    As I said in the chat, I'm sure somebody more qualified than me will
    take it up. But just in case nobody does, I can.

    Alex
    Alexey Verkhovsky, Nov 23, 2004
    #2
    1. Advertising

  3. On Wed, 24 Nov 2004 02:35:21 +0900, David Heinemeier Hansson
    <> wrote:
    > * Switching from Marshal to YAML outputs. The biggest fear in Instiki
    > is that something horrible goes wrong with the persisted data and all
    > is lost in a big ball of Marshalled, binary mud. Having a humanly
    > readable format would be very nice. This needs A LOT of testing,
    > though.


    I recommend *not* using YAML. In my experiences with Ruwiki, YAML support is
    not yet stable enough -- especially for large data -- for general use.

    -austin, trying to get some of these same features in Ruwiki...
    --
    Austin Ziegler *
    * Alternate:
    Austin Ziegler, Nov 23, 2004
    #3
  4. Marshal vs. YAML vs. something else (Re: Most popular wiki in Ruby seeks kind maintainer)

    More generally, let me ask: What formats are people using to persist
    Ruby objects to disk? What are pluses and minuses? I can't figure out
    when I should use something like YAML and when I should use the Marshal
    module and if there's anything else out there that people are using,
    I'm probably going to get sort of confused but should probably hear
    about them anyway.

    On Nov 23, 2004, at 4:49 PM, Austin Ziegler wrote:

    > On Wed, 24 Nov 2004 02:35:21 +0900, David Heinemeier Hansson
    > <> wrote:
    >> * Switching from Marshal to YAML outputs. The biggest fear in Instiki
    >> is that something horrible goes wrong with the persisted data and all
    >> is lost in a big ball of Marshalled, binary mud. Having a humanly
    >> readable format would be very nice. This needs A LOT of testing,
    >> though.

    >
    > I recommend *not* using YAML. In my experiences with Ruwiki, YAML
    > support is
    > not yet stable enough -- especially for large data -- for general use.
    >
    > -austin, trying to get some of these same features in Ruwiki...
    > --
    > Austin Ziegler *
    > * Alternate:
    >
    >


    Francis Hwang
    http://fhwang.net/
    Francis Hwang, Nov 24, 2004
    #4
  5. Re: Marshal vs. YAML vs. something else (Re: Most popular wiki in Ruby seeks kind maintainer)

    Hello Francis,

    FH> More generally, let me ask: What formats are people using to persist
    FH> Ruby objects to disk? What are pluses and minuses? I can't figure out
    FH> when I should use something like YAML and when I should use the Marshal
    FH> module and if there's anything else out there that people are using,
    FH> I'm probably going to get sort of confused but should probably hear
    FH> about them anyway.

    If you want a readable object file that can edited by a human
    (configuration file) use YAML otherwise use Marshal.

    You can read YAML file also with Python, Perl und PHP applications

    --
    Best regards, emailto: scholz at scriptolutions dot com
    Lothar Scholz http://www.ruby-ide.com
    CTO Scriptolutions Ruby, PHP, Python IDE 's
    Lothar Scholz, Nov 24, 2004
    #5
  6. Re: Marshal vs. YAML vs. something else (Re: Most popular wiki in Ruby seeks kind maintainer)

    On Nov 23, 2004, at 9:41 PM, Lothar Scholz wrote:

    > Hello Francis,
    >
    > FH> More generally, let me ask: What formats are people using to
    > persist
    > FH> Ruby objects to disk? What are pluses and minuses? I can't figure
    > out
    > FH> when I should use something like YAML and when I should use the
    > Marshal
    > FH> module and if there's anything else out there that people are
    > using,
    > FH> I'm probably going to get sort of confused but should probably hear
    > FH> about them anyway.
    >
    > If you want a readable object file that can edited by a human
    > (configuration file) use YAML otherwise use Marshal.
    >
    > You can read YAML file also with Python, Perl und PHP applications
    >


    Why wouldn't I just use YAML all the time? Are there certain types of
    classes that aren't going to turn into YAML well? Stability issues with
    YAML libs? (I've mostly used Marshal in the past, so pardon if my YAML
    questions are naive.)

    Francis Hwang
    http://fhwang.net/
    Francis Hwang, Nov 24, 2004
    #6
  7. David Heinemeier Hansson

    John Wilger Guest

    Re: Marshal vs. YAML vs. something else (Re: Most popular wiki in Ruby seeks kind maintainer)

    What's wrong with good old fashined XML? Isn't this sort of thing what
    XML was designed for?


    On Wed, 24 Nov 2004 12:07:07 +0900, Francis Hwang <> wrote:
    >
    >
    >
    > On Nov 23, 2004, at 9:41 PM, Lothar Scholz wrote:
    >
    > > Hello Francis,
    > >
    > > FH> More generally, let me ask: What formats are people using to
    > > persist
    > > FH> Ruby objects to disk? What are pluses and minuses? I can't figure
    > > out
    > > FH> when I should use something like YAML and when I should use the
    > > Marshal
    > > FH> module and if there's anything else out there that people are
    > > using,
    > > FH> I'm probably going to get sort of confused but should probably hear
    > > FH> about them anyway.
    > >
    > > If you want a readable object file that can edited by a human
    > > (configuration file) use YAML otherwise use Marshal.
    > >
    > > You can read YAML file also with Python, Perl und PHP applications
    > >

    >
    > Why wouldn't I just use YAML all the time? Are there certain types of
    > classes that aren't going to turn into YAML well? Stability issues with
    > YAML libs? (I've mostly used Marshal in the past, so pardon if my YAML
    > questions are naive.)
    >
    >
    >
    > Francis Hwang
    > http://fhwang.net/
    >
    >



    --
    Regards,
    John Wilger

    -----------
    Alice came to a fork in the road. "Which road do I take?" she asked.
    "Where do you want to go?" responded the Cheshire cat.
    "I don't know," Alice answered.
    "Then," said the cat, "it doesn't matter."
    - Lewis Carrol, Alice in Wonderland
    John Wilger, Nov 24, 2004
    #7
  8. Re: Marshal vs. YAML vs. something else (Re: Most popular wiki in Ruby seeks kind maintainer)

    On Wed, 24 Nov 2004 11:29:51 +0900, Francis Hwang <>
    wrote:
    > More generally, let me ask: What formats are people using to
    > persist Ruby objects to disk? What are pluses and minuses? I can't
    > figure out when I should use something like YAML and when I should
    > use the Marshal module and if there's anything else out there that
    > people are using, I'm probably going to get sort of confused but
    > should probably hear about them anyway.


    Well, I should qualify my statements some, because I don't want to
    maling _why's work, which is nothing short of amazing, ultimately.
    For short configuration items, YAML is rather impressive. It is
    human editable, it's rich, and it's reasonably stable.

    However, syck -- the YAML parser built into Ruby -- is unusable in
    Ruby 1.8.1 (which is still the officially released version of Ruby)
    on Windows and has other issues in versions of Ruby up through 1.8.2
    preview 2. I have not yet verified whether the problems I had in
    1.8.2p2 with syck have been fixed, and I've not been in a position
    to verify them lately.

    Ruwiki has specific needs that may or may not be present in a
    generic application that needs something externally editable. In
    particular, Ruwiki files can be large because of the content --
    which is a large \n-delimited string. For a Ruwiki file at work and
    for the Ruwiki::WikiMarkup pages, syck failed (as in coredump) in
    Ruby 1.8.1 on Windows on writing strings longer than about 7k and
    reading strings longer than about 4k. I don't think it was a
    Windows-only problem. In Ruby 1.8.2p2, syck was confused by \n and
    sometimes escaped them when it shouldn't, resulting in unusable
    code.

    Because of this, and because I needed a human-editable format that
    was simple, reasonably quick, and reliable, I created what has since
    become Ruwiki::Exportable. This is a general-purpose markup, but it
    is not a type-smart format (like YAML), leaving it up to the reader
    and the writer to ensure that the data it writes out will be
    meaningfully read back in. The format looks something like (this is
    actually the default ruwiki.conf):

    ruwiki-config!css: ruwiki.css
    ruwiki-config!date-format: %Y.%m.%d
    ruwiki-config!datetime-format: %Y.%m.%d %H:%M:%S
    ruwiki-config!debug: true
    ruwiki-config!default-page: ProjectIndex
    ruwiki-config!default-project: Default
    ruwiki-config!language: en
    ruwiki-config!storage-options: flatfiles!data-path: ./data
    flatfiles!extension: ruwiki
    ruwiki-config!storage-type: flatfiles
    ruwiki-config!template-path: ./templates/
    ruwiki-config!template-set: default
    ruwiki-config!time-format: %H:%M:%S
    ruwiki-config!title: Ruwiki
    ruwiki-config!webmaster:
    webrick-config!addresses:
    webrick-config!do-log: true
    webrick-config!log-dest: <STDERR>
    webrick-config!mount: /
    webrick-config!port: 8808
    webrick-config!threads: 1

    When read, this will look like (in Ruby):

    { 'ruwiki-config' =>
    { 'css' => 'ruwiki.css',
    'date-format' => "%Y.%m.%d",
    #...
    },
    'webrick-config' =>
    { 'addresses' => "",
    #...
    }
    }

    I have to know that webrick-config!addresses resulting in ""
    actually means []. I have to know in a .ruwiki file that
    "properties!edit-date: 1101267172" actually means "Tue Nov 23
    22:32:52 Eastern Standard Time 2004" because it's Time.now.to_i.

    I also have to know that the value listed for "storage-options" is
    actually a nested Ruwiki::Exportable document. The formal definition
    for a Ruwiki::Exportable document is:

    <group-id>!<item-id>:<1whitespace>VALUE
    [<1whitespace>CONTINUED VALUE]*

    That is, if I want to continue a value, I simply continue it by not
    specifying another <group-id> at the beginning and indenting the
    embedded value with ONE whitespace -- either a tab or a space (it
    was originally only tabs, which is still what Ruwiki::Exportable
    uses by default on export to string).

    YAML knows some of the type stuff -- and most of the problems I have
    seen are when syck is wrong on that guess or otherwise mangles it.

    I'm actually quite pleased with Ruwiki::Exportable -- it's
    reasonably fast and simple to understand, but it is one level
    separated from the actual object (it uses a hash as the canonical
    format for the data, not the object itself, as Marshal and YAML do).

    I prefer YAML to XML for most editable configuration files, but YAML
    has its own issues.

    -austin

    > On Nov 23, 2004, at 4:49 PM, Austin Ziegler wrote:
    >> On Wed, 24 Nov 2004 02:35:21 +0900, David Heinemeier Hansson
    >> <> wrote:
    >>> * Switching from Marshal to YAML outputs. The biggest fear in
    >>> Instiki is that something horrible goes wrong with the
    >>> persisted data and all is lost in a big ball of Marshalled,
    >>> binary mud. Having a humanly readable format would be very
    >>> nice. This needs A LOT of testing, though.

    >> I recommend *not* using YAML. In my experiences with Ruwiki, YAML
    >> support is not yet stable enough -- especially for large data --
    >> for general use.

    --
    Austin Ziegler *
    * Alternate:
    Austin Ziegler, Nov 24, 2004
    #8
  9. Re: Marshal vs. YAML vs. something else (Re: Most popular wiki in Ruby seeks kind maintainer)

    Hello John,

    JW> What's wrong with good old fashined XML? Isn't this sort of thing what
    JW> XML was designed for?

    Yes, it's only there because some people like the YAML flavour a
    little bit more then old fashined XML.

    It is easier to read and it is more dense. Also it does not try to do
    sophisticated things like namespaces etc.

    But i never used it, i also prefer good old fashined XML.

    --
    Best regards, emailto: scholz at scriptolutions dot com
    Lothar Scholz http://www.ruby-ide.com
    CTO Scriptolutions Ruby, PHP, Python IDE 's
    Lothar Scholz, Nov 24, 2004
    #9
  10. Re: Marshal vs. YAML vs. something else (Re: Most popular wiki in Ruby seeks kind maintainer)

    On Wed, 24 Nov 2004 12:30:11 +0900, John Wilger <> wrote:
    > What's wrong with good old fashined XML? Isn't this sort of thing what
    > XML was designed for?


    The biggest problem with XML is verbosity and impedance mismatch with
    Ruby objects for Marshal-ing.

    -austin
    --
    Austin Ziegler *
    * Alternate:
    Austin Ziegler, Nov 24, 2004
    #10
  11. YAML Problems Clearinghouse (Re: Marshal vs. YAML...)

    Austin Ziegler wrote:
    > Well, I should qualify my statements some, because I don't want to
    > maling _why's work, which is nothing short of amazing, ultimately.


    Austin, you need not worry. I'd really rather hear frustrations come
    out openly and bluntly. Having them concealed and timid only prevents
    me from knowing how to improve my libraries. I appreciate your being
    forthright.

    > Ruwiki has specific needs that may or may not be present in a
    > generic application that needs something externally editable. In
    > particular, Ruwiki files can be large because of the content --
    > which is a large \n-delimited string. For a Ruwiki file at work and
    > for the Ruwiki::WikiMarkup pages, syck failed (as in coredump) in
    > Ruby 1.8.1 on Windows on writing strings longer than about 7k and
    > reading strings longer than about 4k. I don't think it was a
    > Windows-only problem. In Ruby 1.8.2p2, syck was confused by \n and
    > sometimes escaped them when it shouldn't, resulting in unusable
    > code.


    I'm sure syck can handle large quantities of text, as well as continuous
    strings, since the (Poignant) Guide and my blog have both been using
    syck since February of this year. But I definitely don't use the
    emitter (the outputting component of syck) as much as your wiki must.

    I feel that syck's parser is quite strong, but the emitter is very poor
    in comparison. Would you offer up your time in helping me test and kill
    further bugs in the emitter? (This is an open invitation to any of you
    out there who have an interest in improving YAML support.)

    > I prefer YAML to XML for most editable configuration files, but YAML
    > has its own issues.


    You've mentioned coredumps on outputting YAML and oddities with line
    endings. Did I miss anything else? Could you outline exactly what
    other issues you've encountered?

    And do they revolve centrally around the Syck implementation or are they
    related to the nature of YAML and its specification?

    _why
    why the lucky stiff, Nov 25, 2004
    #11
  12. Re: YAML Problems Clearinghouse (Re: Marshal vs. YAML...)

    On Thu, 25 Nov 2004 16:36:30 +0900, why the lucky stiff
    <> wrote:
    > Austin Ziegler wrote:
    >> Ruwiki has specific needs that may or may not be present in a
    >> generic application that needs something externally editable. In
    >> particular, Ruwiki files can be large because of the content --
    >> which is a large \n-delimited string. For a Ruwiki file at work
    >> and for the Ruwiki::WikiMarkup pages, syck failed (as in
    >> coredump) in Ruby 1.8.1 on Windows on writing strings longer than
    >> about 7k and reading strings longer than about 4k. I don't think
    >> it was a Windows-only problem. In Ruby 1.8.2p2, syck was confused
    >> by \n and sometimes escaped them when it shouldn't, resulting in
    >> unusable code.

    > I'm sure syck can handle large quantities of text, as well as
    > continuous strings, since the (Poignant) Guide and my blog have
    > both been using syck since February of this year. But I definitely
    > don't use the emitter (the outputting component of syck) as much
    > as your wiki must.


    Well, I know that with Ruby 1.8.1 I couldn't read a 15k YAML file
    without coredumping. When I split the 12k content section into
    multiple parts with an array, I found that on Windows the coredump
    happened with a big block of text between 7k and 8k; on Linux it was
    closer to 11k. These tests were several months ago, and they went
    away with Ruby 1.8.2 preview 2. Unfortunately, there was the \n
    problem you fixed after preview 2, and I haven't had time to rerun
    my tests with preview 3: this will be run in the next day or two as
    I prepare to release Ruwiki 0.9.0 (I still have to verify the YAML
    and Marshal storage formats since I support them even though I use
    and prefer the Ruwiki::Exportable format).

    > I feel that syck's parser is quite strong, but the emitter is very
    > poor in comparison. Would you offer up your time in helping me
    > test and kill further bugs in the emitter? (This is an open
    > invitation to any of you out there who have an interest in
    > improving YAML support.)


    I can definitely give you some test data and a program (it's part of
    Ruwiki, actually) to test it so you can look at the results, but I
    don't have a lot of time to commit to testing at this point -- as
    soon as I release Ruwiki 0.9.0, I have to start work on both Ruwiki
    0.10.0 and Diff::LCS 1.2.0.

    >> I prefer YAML to XML for most editable configuration files, but
    >> YAML has its own issues.

    > You've mentioned coredumps on outputting YAML and oddities with
    > line endings. Did I miss anything else? Could you outline exactly
    > what other issues you've encountered?
    >
    > And do they revolve centrally around the Syck implementation or
    > are they related to the nature of YAML and its specification?


    XML is overly verbose and YAML is problematic because of space/tab
    issues. I like YAML because it's cleaner than XML, but I don't like
    the indentation model.

    -austin
    --
    Austin Ziegler *
    * Alternate:
    Austin Ziegler, Nov 25, 2004
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. mike
    Replies:
    4
    Views:
    11,119
    Rob McAninch
    Dec 31, 2005
  2. Ron Stephens
    Replies:
    23
    Views:
    2,788
    Ron Stephens
    Apr 12, 2004
  3. Steve
    Replies:
    0
    Views:
    366
    Steve
    Oct 25, 2004
  4. Neil Benn
    Replies:
    8
    Views:
    332
    Alex Martelli
    Oct 26, 2004
  5. Robert Oschler

    Wiki engine used by Ruby Wiki?

    Robert Oschler, Jun 17, 2004, in forum: Ruby
    Replies:
    3
    Views:
    219
    Erik Veenstra
    Jun 25, 2004
Loading...

Share This Page