Improvements to RDoc (ideas for GSoC)

[Rodrigo Almeida]

I'm a brazilian student wishing to participate in Google Summer of
Code this year, and work on something related to RDoc. Before I finish
my proposal, I'd like some community feedback on the ideas below

First of all, I think documentation of any Ruby library can be split
in two different sections: Reference, which is documenting every class
and method on the library, in a way that's easy for one to access
information on a specific feature, and a more general documentation,
which is not bounded to any Ruby code directly. This part of
documentation would include an overview of the library, tutorials and
stuff like that. More "common" libs, like a String library, probably
don't really need the second part, but as I was designing a ACL plugin
for a Rails project I'm participating, I found that having a separate
section of the documentation for explaining how ACL works, system
concepts and some additional tutorials would be a nice add. RDoc sort
of currently supports through files without extension like README or
CHANGELOG, but that makes this documentation accessible only though
the Files menu.

This leads to another important aspect. I think that RDoc shouldn't
include that Files menu, it should be structured more like a Table of
Contents. So, for me, the best layout would be having a left
navigation menu, and my ACL plugin documentation menu could look
something like that:

Introduction
System Concepts
What is an ACL?
Privileges
Selectors
Tutorials
...
Reference
[Object Browser-like structure]

This kind of layout is already implemented by Noobkit. See:
http://www.noobkit.com/show/ruby/ruby.html

My idea is not to force every RDoc generated to look like above, but
improve current RDoc system so that the layout above can be achieved
by simply using a special template. Currently, RDoc templates are Ruby
Modules which define a lot of string constants, containing
placeholders, which results in template construction and readability
being far from ideal. The already suggested idea of changing RDoc to
use ERb templates would be really appropriate here, we could really
use some inspiration from rails and use partials to render the
different elements in the page, and have a main template file that
puts everything together.

For implementing code-unrelated documentation, one idea would be to
define a default folder (let's say, /add_doc), which, if present on
the source code root folder, would be parsed by RDoc and added to the
documentation table of contents. Each file inside this dir should call
a title directive on its first line, to define that page's title. So,
the TOC above could be generated by the following file structure:

/add_doc/intro.rdoc
/add_doc/concepts.rdoc
/add_doc/concepts/acl.rdoc
/add_doc/concepts/privileges.rdoc
(...)

Currently RDoc already can be extended to add support for other forms
of markup (see doc for RDoc:age), but this could be improved, as to
accept additional markup tags and modifiers in an easy way. Let's say
I used to work with java and wanted to comment a method like in the
example below.

# Method description here
#
# :arams::
# url: an absolute URL giving the base location of the image
# name: the location of the image, relative to the url argument
# ::return::
# the image at specified URL
# ::see::
# Image
#
def load_image(url, name) #:takes: String, String; #:returns: Image

It should be viable for someone to come up with an RDoc extension that
can generate the JavaDoc-like output from the comments above without
excessive digging into RDoc's source.

Those are my preliminary ideas, any feedback on those would be really
appreciated.

 

[Dave Thomas]

Currently RDoc already can be extended to add support for other forms
of markup (see doc for RDoc:age), but this could be improved, as to
accept additional markup tags and modifiers in an easy way. Let's say
I used to work with java and wanted to comment a method like in the
example below.

# Method description here
#
# :arams::
# url: an absolute URL giving the base location of the image
# name: the location of the image, relative to the url argument
# ::return::
# the image at specified URL
# ::see::
# Image
#
def load_image(url, name) #:takes: String, String; #:returns: Image

It should be viable for someone to come up with an RDoc extension that
can generate the JavaDoc-like output from the comments above without
excessive digging into RDoc's source.

All neat and worthy ideas.

I'm the last person to defend RDoc--it has already served it's
purpose, and is long due for replacement.

However, one of the goals of RDoc was to not pollute the source code
with markup. I want the comments to look like comments, and not like
something put there for the convenience of a documentation tool.
That's why, for example, RDoc does automatic hyperlinking when it can--
it seems better to live with the fact it sometimes gets it wrong
rather than live with extra stuff in the code. It's also why RDoc will
produce halfway useful documentation for a source file containing no
comments at all.

Anyway, I'd suggest that your example here could equally well be written

# Takes the base location of the image and the name of the image (both
as strings)
# and returns the absolute image url. Also see the Image class.

def image_url(base, image_name)
...
end


Remember that, unlike JavaDoc, RDoc already extracts the names of
parameters from the source code. Give them meaningful names, and I'm
not sure you need to have tables that duplicate those names in the
comment block.



Dave

 

[Trans]

Those are my preliminary ideas, any feedback on those would be really
appreciated.

I agree with Dave about the overly extensive markup. But I agree with
you about file docs. I'd rather see a table of contents too.

T.

 

[Eric Hodel]

I'm a brazilian student wishing to participate in Google Summer of
Code this year, and work on something related to RDoc. Before I finish
my proposal, I'd like some community feedback on the ideas below

Hi, I'm the current maintainer of RDoc.

I have made some refactorings and minor improvements to RDoc for Ruby
1.9. If you decide to continue with this project, please make your
changes against 1.9 so they will be easy for me to merge in.

(The RDoc that ships with 1.8 is RDoc 1.0.1, the RDoc that ships with
1.9 I have changed to RDoc will be RDoc 2.)

First of all, I think documentation of any Ruby library can be split
in two different sections: Reference, which is documenting every class
and method on the library, in a way that's easy for one to access
information on a specific feature, and a more general documentation,
which is not bounded to any Ruby code directly. This part of
documentation would include an overview of the library, tutorials and
stuff like that. More "common" libs, like a String library, probably
don't really need the second part, but as I was designing a ACL plugin
for a Rails project I'm participating, I found that having a separate
section of the documentation for explaining how ACL works, system
concepts and some additional tutorials would be a nice add. RDoc sort
of currently supports through files without extension like README or
CHANGELOG, but that makes this documentation accessible only though
the Files menu.

In the past I've used the appropriate class for the location of
overview/tutorial/how-to documentation, but I agree it is not optimal.

This leads to another important aspect. I think that RDoc shouldn't
include that Files menu

I don't like the Files list either, I don't find it helpful for
navigating the documentation.

it should be structured more like a Table of
Contents. So, for me, the best layout would be having a left
navigation menu, and my ACL plugin documentation menu could look
something like that:

Introduction
System Concepts
What is an ACL?
Privileges
Selectors
Tutorials
...
Reference
[Object Browser-like structure]

This kind of layout is already implemented by Noobkit. See:
http://www.noobkit.com/show/ruby/ruby.html

My idea is not to force every RDoc generated to look like above, but
improve current RDoc system so that the layout above can be achieved
by simply using a special template. Currently, RDoc templates are Ruby
Modules which define a lot of string constants, containing
placeholders, which results in template construction and readability
being far from ideal. The already suggested idea of changing RDoc to
use ERb templates would be really appropriate here, we could really
use some inspiration from rails and use partials to render the
different elements in the page, and have a main template file that
puts everything together.

RDoc 2 has an ERb-based template system that makes it easy to convert
RDoc 1.x templates.

I think it would be OK to switch to a completely different ERb-based
template system. (For example, there is a ruby object -> Hash/Array/
Hash conversion that happens to support the old template system, but
it is not necessary for ERb. Removing this conversion would make it
easier to make more-flexible templates.)

For implementing code-unrelated documentation, one idea would be to
define a default folder (let's say, /add_doc), which, if present on
the source code root folder, would be parsed by RDoc and added to the
documentation table of contents. Each file inside this dir should call
a title directive on its first line, to define that page's title. So,
the TOC above could be generated by the following file structure:

/add_doc/intro.rdoc
/add_doc/concepts.rdoc
/add_doc/concepts/acl.rdoc
/add_doc/concepts/privileges.rdoc
(...)

Or, maybe simply an rdoc/ directory, since the rdoc command-line tool
generates documentation into doc/ by default.

Currently RDoc already can be extended to add support for other forms
of markup (see doc for RDoc:age), but this could be improved, as to
accept additional markup tags and modifiers in an easy way. Let's say
I used to work with java and wanted to comment a method like in the
example below.

# Method description here
#
# :arams::
# url: an absolute URL giving the base location of the image
# name: the location of the image, relative to the url argument
# ::return::
# the image at specified URL
# ::see::
# Image
#
def load_image(url, name) #:takes: String, String; #:returns: Image

It should be viable for someone to come up with an RDoc extension that
can generate the JavaDoc-like output from the comments above without
excessive digging into RDoc's source.

Those are my preliminary ideas, any feedback on those would be really
appreciated.

I am not sure how easy it is to add additional directives (on the def
line) to RDoc, I haven't looked. Adding additional markup inside a
comment is much easier and (I think) could be achieved with a mixin
module.

However, I agree with Dave here. I find documentation easier to write
if it reads like a sentence. This is an example of some documentation
I wrote:

class Gem::Requirement

##
# Constructs a Requirement from +requirements+ which can be a
String, a
# Gem::Version, or an Array of those. See parse for details on the
# formatting of requirement strings.

def initialize(requirements)
end

(RDoc will automatically link "parse" to the parse method's
documentation. I only bothered to mark up "requirements" to draw the
eye.)

 

[Ken Bloom]

I'm a brazilian student wishing to participate in Google Summer of Code
this year, and work on something related to RDoc. Before I finish my
proposal, I'd like some community feedback on the ideas below

First of all, I think documentation of any Ruby library can be split in
two different sections: Reference, which is documenting every class and
method on the library, in a way that's easy for one to access
information on a specific feature, and a more general documentation,
which is not bounded to any Ruby code directly. This part of
documentation would include an overview of the library, tutorials and
stuff like that. More "common" libs, like a String library, probably
don't really need the second part, but as I was designing a ACL plugin
for a Rails project I'm participating, I found that having a separate
section of the documentation for explaining how ACL works, system
concepts and some additional tutorials would be a nice add. RDoc sort of
currently supports through files without extension like README or
CHANGELOG, but that makes this documentation accessible only though the
Files menu.

This leads to another important aspect. I think that RDoc shouldn't
include that Files menu, it should be structured more like a Table of
Contents. So, for me, the best layout would be having a left navigation
menu, and my ACL plugin documentation menu could look something like
that:

Introduction
System Concepts
What is an ACL?
Privileges
Selectors
Tutorials
...
Reference
[Object Browser-like structure]

This kind of layout is already implemented by Noobkit. See:
http://www.noobkit.com/show/ruby/ruby.html

Perhaps you could look at my rdoc wishlist at http://
blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/295589 and consider
something along those lines -- separate the backend from the frontend,
and integrate rubygems into the system.

--Ken

 

[s.ross]

Perhaps you could look at my rdoc wishlist at http://
blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/295589 and
consider
something along those lines -- separate the backend from the frontend,
and integrate rubygems into the system.

--Ken

Ken--

I don't have a rails app that does what you describe but I did put one
up that does some browsing of the ruby gems. It's at http://gemmy.calicowebdev.com
I, too, see some ways to improve the availability of information and
one of the things I'd like to figure out is how to get the Ruby
documentation in a guess-searchable state like the online dox for many
other development platforms are. What I mean by that is that it would
be great to be able to look somewhere other than Google for
authoritative information about Ruby.

To the OP--

Great project idea. I agree with Dave Thomas about commenting. Looking
at too much markup in the code reminds me of JavaDoc and PHPDoc, but
more to the point, it produces too much visual noise in what is
otherwise an expressive code-writing environment.

 

[Jeremy McAnally]

Is this new RDoc on Github or somewhere that I can get to? I'd be
interested in looking at how the ERb stuff works to see if I can come
up with a simple system for plugins/markup extensions.

--Jeremy

I'm a brazilian student wishing to participate in Google Summer of
Code this year, and work on something related to RDoc. Before I finish
my proposal, I'd like some community feedback on the ideas below

Hi, I'm the current maintainer of RDoc.

I have made some refactorings and minor improvements to RDoc for Ruby
1.9. If you decide to continue with this project, please make your
changes against 1.9 so they will be easy for me to merge in.

(The RDoc that ships with 1.8 is RDoc 1.0.1, the RDoc that ships with
1.9 I have changed to RDoc will be RDoc 2.)

First of all, I think documentation of any Ruby library can be split
in two different sections: Reference, which is documenting every class
and method on the library, in a way that's easy for one to access
information on a specific feature, and a more general documentation,
which is not bounded to any Ruby code directly. This part of
documentation would include an overview of the library, tutorials and
stuff like that. More "common" libs, like a String library, probably
don't really need the second part, but as I was designing a ACL plugin
for a Rails project I'm participating, I found that having a separate
section of the documentation for explaining how ACL works, system
concepts and some additional tutorials would be a nice add. RDoc sort
of currently supports through files without extension like README or
CHANGELOG, but that makes this documentation accessible only though
the Files menu.

In the past I've used the appropriate class for the location of
overview/tutorial/how-to documentation, but I agree it is not optimal.

This leads to another important aspect. I think that RDoc shouldn't
include that Files menu

I don't like the Files list either, I don't find it helpful for
navigating the documentation.

it should be structured more like a Table of
Contents. So, for me, the best layout would be having a left
navigation menu, and my ACL plugin documentation menu could look
something like that:

Introduction
System Concepts
What is an ACL?
Privileges
Selectors
Tutorials
...
Reference
[Object Browser-like structure]

This kind of layout is already implemented by Noobkit. See:
http://www.noobkit.com/show/ruby/ruby.html

My idea is not to force every RDoc generated to look like above, but
improve current RDoc system so that the layout above can be achieved
by simply using a special template. Currently, RDoc templates are Ruby
Modules which define a lot of string constants, containing
placeholders, which results in template construction and readability
being far from ideal. The already suggested idea of changing RDoc to
use ERb templates would be really appropriate here, we could really
use some inspiration from rails and use partials to render the
different elements in the page, and have a main template file that
puts everything together.

RDoc 2 has an ERb-based template system that makes it easy to convert
RDoc 1.x templates.

I think it would be OK to switch to a completely different ERb-based
template system. (For example, there is a ruby object -> Hash/Array/
Hash conversion that happens to support the old template system, but
it is not necessary for ERb. Removing this conversion would make it
easier to make more-flexible templates.)

For implementing code-unrelated documentation, one idea would be to
define a default folder (let's say, /add_doc), which, if present on
the source code root folder, would be parsed by RDoc and added to the
documentation table of contents. Each file inside this dir should call
a title directive on its first line, to define that page's title. So,
the TOC above could be generated by the following file structure:

/add_doc/intro.rdoc
/add_doc/concepts.rdoc
/add_doc/concepts/acl.rdoc
/add_doc/concepts/privileges.rdoc
(...)

Or, maybe simply an rdoc/ directory, since the rdoc command-line tool
generates documentation into doc/ by default.

Currently RDoc already can be extended to add support for other forms
of markup (see doc for RDoc:age), but this could be improved, as to
accept additional markup tags and modifiers in an easy way. Let's say
I used to work with java and wanted to comment a method like in the
example below.

# Method description here
#
# :arams::
# url: an absolute URL giving the base location of the image
# name: the location of the image, relative to the url argument
# ::return::
# the image at specified URL
# ::see::
# Image
#
def load_image(url, name) #:takes: String, String; #:returns: Image

It should be viable for someone to come up with an RDoc extension that
can generate the JavaDoc-like output from the comments above without
excessive digging into RDoc's source.

Those are my preliminary ideas, any feedback on those would be really
appreciated.

I am not sure how easy it is to add additional directives (on the def
line) to RDoc, I haven't looked. Adding additional markup inside a
comment is much easier and (I think) could be achieved with a mixin
module.

However, I agree with Dave here. I find documentation easier to write
if it reads like a sentence. This is an example of some documentation
I wrote:

class Gem::Requirement

##
# Constructs a Requirement from +requirements+ which can be a
String, a
# Gem::Version, or an Array of those. See parse for details on the
# formatting of requirement strings.

def initialize(requirements)
end

(RDoc will automatically link "parse" to the parse method's
documentation. I only bothered to mark up "requirements" to draw the
eye.)

--
http://jeremymcanally.com/
http://entp.com

Read my books:
Ruby in Practice (http://manning.com/mcanally/)
My free Ruby e-book (http://humblelittlerubybook.com/)

Or, my blogs:
http://mrneighborly.com
http://rubyinpractice.com

 

[Eric Hodel]

Is this new RDoc on Github or somewhere that I can get to? I'd be
interested in looking at how the ERb stuff works to see if I can come
up with a simple system for plugins/markup extensions.

You can get it out of Ruby trunk. I'll try to make a release for
testing in the next week.

 

[Jeremy McAnally]

Great!

I'd written an RDoc alternative called DocR that used ERb, but I'm
glad you switched to ERb. Saves me a lot of effort and maintenance
crap.

I'm looking to make a small hook or DSL or something for extensions to
the RDoc markup (see what Merb is doing with their markup or maybe
YARD or whatever) to let people add interesting things (for example a
public API might want to add the automatic appending of resource URLs
to the docs or something). I'll grab it out of trunk and look at it
tonight.

--Jeremy

You can get it out of Ruby trunk. I'll try to make a release for
testing in the next week.

--
http://jeremymcanally.com/
http://entp.com

Read my books:
Ruby in Practice (http://manning.com/mcanally/)
My free Ruby e-book (http://humblelittlerubybook.com/)

Or, my blogs:
http://mrneighborly.com
http://rubyinpractice.com

 

[Eric Hodel]

Please don't top-post.

I'd written an RDoc alternative called DocR that used ERb, but I'm
glad you switched to ERb. Saves me a lot of effort and maintenance
crap.

I'm looking to make a small hook or DSL or something for extensions to
the RDoc markup (see what Merb is doing with their markup or maybe
YARD or whatever) to let people add interesting things (for example a
public API might want to add the automatic appending of resource URLs
to the docs or something). I'll grab it out of trunk and look at it
tonight.

It's easy to add "specials" to RDoc markup, depending upon how
complicated you want them to be. I wrote a small wiki using RDoc 2's
markup here:

http://blog.segment7.net/articles/2008/02/14/an-rdoc-wiki

A WikiWord is handled like this:

class WikiHtml < RDoc::Markup::ToHtml
def initialize
super

@markup.add_special(/\b(([A-Z][a-z]+){2,})\b/, :WIKIWORD)
end

def handle_special_WIKIWORD(special)
"<a href=#{special.text}>#{special.text}</a>"
end
end

And you can use WikiHtml to convert text like so:

wiki_html = WikiHtml.new
html = wiki_html.convert <<-EOF
= Some heading

WikiWord links to some other page
EOF