Could Ruby-doc be better? -- Proposal for a better system.

[Andrew Thompson]

<Sorry for the crosspost, but I thought I might as well try to reach as
many people as I could >

Hello Everyone,

I've been toying with ruby for the last month or so and I've become a
big fan of the language. My only gripe is the documentation is, at best,
patchy and obtuse.

I don't intend to flame, or disparage the work done on the documentation
so far, but its not really up to scratch when you look at Python's
manual or, my personal favorite, the PHP manual.

I'm thinking of a system that will allow people to edit/add
documentation from the web based on the existing rdoc and source code
comments, and then generate CVS patches based on those changes which we
can then send for inclusion into mainline.

I've been looking at the rdoc comments in the standard library, and it
seems fairly standard, the method used to document the core .c files
seems a bit less clear, but I think its still usable.

My basic plan is to have a php style manual
http://www.php.net/manual/en/ generated from the rdoc comments, with the
ability to have the documentation maintained in a wiki based fashion,
and have user comments along the lines of the PHP manual. This way we
lower the barrier to writing documentation, allow users to comment with
code samples or additional info, and hopefully increase and improve the
amount of documentation ruby has.

I'd imagine the system would work as follows

Checkout latest CVS ruby code
Extract the Rdoc comments
Check for conflicts with with the documentation already in the system
and mark them for manual merging(where the documentation has changed in
CVS since last checkout)
Rinse and repeat
...

And whenever we think we've made enough progress we can generate a CVS
patch to be merged into mainline. I think we'd probably maintain a
'frozen' and an editable version of each item of documentation, the
frozen is the one shown to casual users and the one we use to generate
CVS, the editable one is the one contributors edit, and periodically one
of the moderators would update the frozen version from the editable one.
This way malicious edits don't screw up the publically viewed
documentation, and we can control when a particular item of
documentation goes into CVS. Users would have to register/login to
edit/add documentation.

I'd also think we could generate 'stubs' for undocumented functions when
we parse the code, and flag them as being empty and in need of
documentation.

The comments would probably be fairly straightforward, no registration
required. The PHP manual has some great comments that help a lot with
common problems people experience using that particular function. I
think this would be a valuable addition.

I haven't planned out the technicalities of it yet, but I'm thinking
Ruby on Rails (Everyone seems to be talking about it, but its ruby, so
how bad could it be?) and MySQL or PostgreSQL (whatever's handy). The
apache mod_rewrite might be handy too, but not essential.

Now, I'm willing to take a stab at this (I'm not just talking), but I'd
like to throw this out there for ideas or comments or flames (stony
silence aren't quite as welcome or helpful) and to see if anyone else is
interested in helping with this project. I'm relatively new to Ruby and
I know nothing about Ruby on Rails, but I'm confident it can be done.
(Maybe not well, maybe not quickly, but it can be done.)

Anyway, let me know your thoughts on this.

Andrew

 

[James Britt]

...
Hello Everyone,

I've been toying with ruby for the last month or so and I've become a
big fan of the language. My only gripe is the documentation is, at best,
patchy and obtuse.

I don't intend to flame, or disparage the work done on the documentation
so far, but its not really up to scratch when you look at Python's
manual or, my personal favorite, the PHP manual.

I'm thinking of a system that will allow people to edit/add
documentation from the web based on the existing rdoc and source code
comments, and then generate CVS patches based on those changes which we
can then send for inclusion into mainline.

<snip/>

I'm working on this right now.

I've been looking at the rdoc comments in the standard library, and it
seems fairly standard, the method used to document the core .c files
seems a bit less clear, but I think its still usable.

My basic plan is to have a php style manual
http://www.php.net/manual/en/ generated from the rdoc comments, with the
ability to have the documentation maintained in a wiki based fashion,
and have user comments along the lines of the PHP manual. This way we
lower the barrier to writing documentation, allow users to comment with
code samples or additional info, and hopefully increase and improve the
amount of documentation ruby has.

Yes, this is quite good, and a some of what I'm after.

...

I haven't planned out the technicalities of it yet, but I'm thinking
Ruby on Rails (Everyone seems to be talking about it, but its ruby, so
how bad could it be?) and MySQL or PostgreSQL (whatever's handy). The
apache mod_rewrite might be handy too, but not essential.

I'm using Nitro.


James Britt
--

http://www.ruby-doc.org - The Ruby Documentation Site
http://www.rubyxml.com - News, Articles, and Listings for Ruby & XML
http://www.rubystuff.com - The Ruby Store for Ruby Stuff
http://www.jamesbritt.com - Playing with Better Toys

 

[Premshree Pillai]

My basic plan is to have a php style manual
http://www.php.net/manual/en/ generated from the rdoc comments, with the

I think it's easier for the PHP folks to have a manual like that,
because most (if not all) functions are pretty much built into the
language itself. With Ruby, we have a lot many libraries to take care
about -- consolidating documentation effort won't be that easy.

--=20
Premshree Pillai
http://www.livejournal.com/users/premshree/

 

[Lothar Scholz]

Hello Premshree,


PP> I think it's easier for the PHP folks to have a manual like that,
PP> because most (if not all) functions are pretty much built into the
PP> language itself. With Ruby, we have a lot many libraries to take care
PP> about -- consolidating documentation effort won't be that easy.

Yes but the main problem is the bad implementation of RDOC which
is not able to merge different "rdoc" runs into one "documentation
repository". So if somebody want to spend time i think it's much
better to do write a program that is using a good documentation
database which allows addition and removal of documentation subtrees.

Also everything gets more complicated with the open classes feature of
ruby. So i don't think that a static manual is good enough. We must look
10 years ahead.

And for the core and all c extensions i also think that the rdoc
approach failed. I really like the way Lyle Johnson is doing rdoc
handling for the FXRuby system, writing a separate documentation ruby
file system.

Look at the Tk binding or many other that are using the dynamic
features of Ruby, they will never work well with RDOC and this is
conceptionally.

 

[Andrew Thompson]

So, basically the consensus is that Rdoc is in need of some work,
especially as regards the .c files, that online editing of the docs
directly isn't required, but that the current ruby-doc doc browser is a
bit crufty and could be done better and preferably without frames.

As regards improving the RDoc tool itself, I'm not really sure if I'm
experienced enough to do that, but I'm willing to look at it at least.

At the very least, I think the output from the Rdoc tool could be better
presented and organized on the web, which may be what James is working
on, and we could bolt on a comment system to allow people to suggest
improvements to the documentation or outline pitfalls with the
particular function ( see an example here:
http://www.php.net/manual/en/function.substr-replace.php ).

The RDoc format seems pretty full-featured, about the only thing I'd
like to add would be a tag to define code snippets so they could be
syntax-highlighted for the web (it looks like code snippets are just
rendered in the typewriter style font at the moment).

As far as functionality goes Hugh mentioned call graphing, I guess that
means showing that function X calls function Y which calls function Z?

Yes but the main problem is the bad implementation of RDOC which
is not able to merge different "rdoc" runs into one "documentation
repository". So if somebody want to spend time i think it's much
better to do write a program that is using a good documentation
database which allows addition and removal of documentation subtrees.

I'm not really sure what lothar means here, is he proposing that rdoc
should be able to resolve conflicts between different versions of ruby
its run against?

To summarize, I'm thinking to scrap the online editing idea and just
reformat the output of rdoc and add a comment system for each
class/function/whatever. Also, assuming I can decipher the rdoc code,
and I can decide on what to change/fix/add, some improvements could be
done on rdoc itself.

James, could you fill me in on what you're working on as regards this
whole thing, maybe we can pool our efforts on this...

Oh, it looks like I'm getting replies from ruby-talk and ruby-doc,
anyone have any objections on moving this discussion to ruby-doc only
now that there's a discussion going? The only problem is that ruby-doc
doesn't seem to be publically archived anywhere like all the other ruby
mailing lists are...

Anyway, thanks for the input guys, I really would like to whatever I can
to help ruby out, and this seems as good a place as any to start

Andrew

 

[Lothar Scholz]

Hello Andrew,

AT> As far as functionality goes Hugh mentioned call graphing, I guess that
AT> means showing that function X calls function Y which calls function Z?

Don't forget Class Hierarchy Graphs.

AT> I'm not really sure what lothar means here, is he proposing that rdoc
AT> should be able to resolve conflicts between different versions of ruby
AT> its run against?

I would like to see exactly one location where information about all
libs and internal structures are stored. At the moment it is
fragmented. You have some docs about "activerecord" here and some of the
"actionmailer" there but no overall structure.

I think we should also go away from the idea to generate HTML files.
Instead we should run a rdoc server and generate documentation on
demand as this allows much better interaction and things like search
functionality. Choosing HTML was just a lazy on in my opinion an old
fashioned idea.

 

[Robert Klemme]

Hello Andrew,


Don't forget Class Hierarchy Graphs.

Yep, I also think documentation of inheritance hiearchies could be better.

I would like to see exactly one location where information about all
libs and internal structures are stored. At the moment it is
fragmented. You have some docs about "activerecord" here and some of
the "actionmailer" there but no overall structure.

+1 Definitely!

I think we should also go away from the idea to generate HTML files.
Instead we should run a rdoc server and generate documentation on
demand as this allows much better interaction and things like search
functionality. Choosing HTML was just a lazy on in my opinion an old
fashioned idea.

-1 I prefer static HTML files as these don't use extra resources; it's
also easier to do for multiple platforms. I rather seldom feel the need
for a search functionality - and in these rare cases find-grep does the
job quite well. My 0.02 EUR...

As a compromize there could be an IRB module that retrieves / generates
documentation on the fly.

Kind regards

robert

 

[James Britt]

So, basically the consensus is that Rdoc is in need of some work,
especially as regards the .c files, that online editing of the docs
directly isn't required, but that the current ruby-doc doc browser is a
bit crufty and could be done better and preferably without frames.

The 'ruby-doc browser' is just basically what rdoc emits. The available
Rdoc templates leave something to be desired. There has been talk here
in the past on writing a better set of templates to avoid using frames
(or use them in such a way that at least one can bookmark or easily link
to pages for specific classes/modules/methods).

(Of course, while there is room from improvement for presentation, the
real work is in updating the docs themselves. Coding, however, always
has more appeal.)
...

James, could you fill me in on what you're working on as regards this
whole thing, maybe we can pool our efforts on this...

There are few ways to approach this. The first one I've looked at, and
done some work on, is altering the rdoc template so that each page shown
in the lower frame has a link for adding comments.

The idea would be to also dynamically include existing comments into
that frame as well.

But it occurred to me that a fairly simple alternative is to have each
main page just link to a corresponding wiki page. It is perhaps not as
slick as having the comments appear on the same page as the docs, though
it has the advantage that if you open the wiki link in new window it is
easier to browse both the docs and the comments at the same time.

This seems like such appealing, low-hanging fruit that I may just do
this anyway; if it turns out to be insufficient then it can be changed
later.

(Ruwiki seems a good choice of wiki for this, with a sub-wiki project
for each class or module)

The usual caveats on wiki spam apply.

Another approach might be to integrate the doc pages with the comments
system directly a la php.net

(Some ways of doing that: dynamic template parsing that merges the API
docs with comments fetched from some storage system, which is where my
recent efforts have been; using DHTML and remote scripting to post and
fetch content from some service. I sort of prefer this latter idea,
using XMlHttpReqquest to fetch the content from a wiki's RSS feed.)

Oh, it looks like I'm getting replies from ruby-talk and ruby-doc,
anyone have any objections on moving this discussion to ruby-doc only
now that there's a discussion going? The only problem is that ruby-doc
doesn't seem to be publically archived anywhere like all the other ruby
mailing lists are...

There was , once upon a time, an specific archive. The list is carried
by gmane and they have an archive or something like that:

http://news.gmane.org/gmane.comp.lang.ruby.documentation

James
--

http://www.ruby-doc.org - The Ruby Documentation Site
http://www.rubyxml.com - News, Articles, and Listings for Ruby & XML
http://www.rubystuff.com - The Ruby Store for Ruby Stuff
http://www.jamesbritt.com - Playing with Better Toys

 

[Premshree Pillai]

I think we should also go away from the idea to generate HTML files.

Umm, performance issues?

--=20
Premshree Pillai
http://www.livejournal.com/users/premshree/