Tried Ruby (or, "what Python *really* needs" or "perldoc!")

[john_sips_tea]

Just tried Ruby over the past two days. I won't bore you
with the reasons I didn't like it, however one thing really
struck me about it that I think we (the Python community)
can learn from.

Ruby has ... an issue with docs. That is to say, there are
almost none. Well, actually, there are some. For example,
the "PickAxe" book (google it), and "Why's" Poignant Guide.
But there's a disturbing lack of *built-in* docs for Ruby.
Now, the situation is getting better. In fact, it's getting
better very quickly. Let me explain.

IMO, Perl has docs nailed. I learned Perl before coming
to Python, and I can tell you that their docs kick butt.
I believe the reason why is (besides Larry's excellent
and entertaining writing) because of perldoc. Here's how
it works: they write special doc directives into their .pl
files that the interpreter ignores, but that the perldoc
command processes into a manpage for you. This is a *bit*
like what pydoc does, only pydoc just regurgitates docstrings,
while perldoc formats special directives into headings, code
listings, bulleted lists, etc.

Now, the *real* magic of perldoc is that it makes writing
small self-contained little manpage style docs easy as pie.
What this means is that really good doc writers can sit down
and quickly produce something that can be readily incorporated
into the perl-doc distribution. *That's* the magic. That's
why I think they've organically grown such an amazing crop
of great docs. Anyone can quickly and easily write docs that
the community can then filter to find the best ones.

Back to Ruby's docs. Their situation is getting better. This
is because they're picking up on Perl's doc success and doing
the same sort of thing. Only, I think their solution may be
clunkier because it requires a separate program (rdoc) to
process the doc directives, and then you actually read the
docs with a program called ri (not sure). Regardless of the
minute details, it looks like their docs are starting to get
better, rapidly.

Here's what I think is currently happening with Python
(please correct me if I'm wrong): There's the docutils project
http://docutils.sourceforge.net/ which, AFAICT, is working on a
number of complicated things, one of which is to have a way to
allow you to put reStructuredText markup (aka "reST")
( http://docutils.sourceforge.net/rst.html ) into your docstrings
(and presumably have the pydoc command understand it). There's
more info in PEP 287 http://www.python.org/doc/peps/pep-0287/ .

I certainly don't understand all that Docutils is trying to
do. All I'm convinced of is, the real magic is in being able to
quickly write a .py file containing marked up docstrings and
have the pydoc command be able to render it as something that
looks like a man page in my terminal window. If it can later
also produce html, that's great too. But *that's* the essense,
IMO, of what will foster community involvement in making the
docs great, as well as allowing folks to more easily document
their own modules and contribute integrated docs to all the
great modules already available.

It looks like we have the tools do this *right now*. We've got
the markup (reST), we've got the tools to churn that into a
manpage or html (part of docutils?), and we've got the pydoc
command. I think the current hangup is that the docutils guys
(being smart and ambitious folks) want to get some sort of
inter-doc linking thing working so you can refer from one doc
page to another (?). I don't think perldoc has (or needs) that
feature... maybe we could put that on the "would be nice to
have down the road" list, and get a quick common-sense
docstring-reST pydoc setup working for us much sooner? I
don't know for sure, but my guess is "yes".

Eventually, maybe the tutorial, language reference, library ref,
etc., could even all make it into this format, with the html
versions generated from the .py/docstring/reST sources. That might
make it easier for the community to contribute to them (if their
respective maintainers are interested in that).

Please note, I certainly don't want to step on the doc-sig folks'
toes here -- but rather to generate more interest in what they're
doing, and to help make Python be even better in an area that I
see it struggling.

What do you folks think?

Yes, I'm trying to make time to look at the docutils code and the
pydoc command to see what's involved. Unfortunately, my spare
time is vanishingly close to zero right now.

Related link:
Doc-SIG http://www.python.org/community/sigs/current/doc-sig/

Thanks,
---John

 

[Diez B. Roggisch]

Please note, I certainly don't want to step on the doc-sig folks'

toes here -- but rather to generate more interest in what they're
doing, and to help make Python be even better in an area that I
see it struggling.

What do you folks think?

Yes, I'm trying to make time to look at the docutils code and the
pydoc command to see what's involved. Unfortunately, my spare
time is vanishingly close to zero right now.

You heard of epydoc? http://epydoc.sourceforge.net/

It pretty much does what you say I think - and for my personal projects I
use it. Maybe we an adopt it as standard-tool.

Diez

 

[john_sips_tea]

Thanks Diez! Epydoc looks great.

Can we use epytext to generate output suitable for a manpage?

Do you prefer epytext or reST?

 

[Diez B. Roggisch]

Thanks Diez! Epydoc looks great.

Can we use epytext to generate output suitable for a manpage?

Don't know, never tried that.

Do you prefer epytext or reST?

So far epytext suited my needs.

Diez

 

[john_sips_tea]

So far epytext suited my needs.

I like it too.

Ok, now I'm starting to get excited.

 

[Colin J. Williams]

Don't know, never tried that.

The answer appear to be No, but there is a man.py file which indicates
that some work was done on it.

Below is the epydoc commandline guide.

Colin W.

epydoc [OPTIONS] MODULES...

MODULES... The Python modules to document.
--html Generate HTML output (default).
--latex Generate LaTeX output.
--pdf Generate pdf output, via LaTeX.
--check Run documentation completeness checks.
-o DIR, --output DIR The output directory.
-n NAME, --name NAME The documented project's name.
-u URL, --url URL The documented project's url.
-t PAGE, --top PAGE The top page for the HTML documentation.
-c SHEET, --css SHEET CSS stylesheet for HTML files.
--private-css SHEET CSS stylesheet for private objects.
--inheritance STYLE The format for showing inherited objects.
-V, --version Print the version of epydoc.
-h, -?, --help, --usage Display this usage message.
-h TOPIC, --help TOPIC Display information about TOPIC (docformat,
css, inheritance, usage, or version).

 

[john_sips_tea]

Ok. I'm going to try and make something happen. Give me a day or so.


---John

 

[Nick Craig-Wood]

IMO, Perl has docs nailed. I learned Perl before coming
to Python, and I can tell you that their docs kick butt.
I believe the reason why is (besides Larry's excellent
and entertaining writing) because of perldoc. Here's how
it works: they write special doc directives into their .pl
files that the interpreter ignores, but that the perldoc
command processes into a manpage for you. This is a *bit*
like what pydoc does, only pydoc just regurgitates docstrings,
while perldoc formats special directives into headings, code
listings, bulleted lists, etc.

As another perl refugee I agree with you 100% here.

"perldoc perltoc" then "perldoc xxxx" will find you anything in perl.

Its frustrating that pydoc is only half (or less) of the docs.

However having bedded into python for a few years, I now just reach
for my web browser for the global module index instead of pydoc. Its
not as good as perldoc as it doesn't cover all the modules I've got
installed, but its very good documentation. Some modules have good
pydoc-umentation but not all of them. Code examples tend to be
missing.

I think a major problem with our way of thinking about
perldoc/pydoc/man pages is that it is a) unix centric, and b) possibly
a bit old fashioned! a) and b) apply to perl certainly, but I don't
think they apply to python in the same way.

I'd love to have a unified documentation system where *all* the
documentation for *all* installed modules was available to pydoc *and*
the web browser and *all* this documentation was in .py files.

(Putting the code together with the documentation is essential in my
opinion and experience - if you seperate the two then the documention
will lag the code.)

PS I've used reST and perldoc. reST is easier to use for the easy
things, but gets complicated for the hard things.

 

[Terry Hancock]

On 14 Mar 2006 09:25:07 -0800

Do you prefer epytext or reST?

I personally prefer epytext primarily because it has support
for greek letters and math symbols. That could be very
useful in documenting certain kinds of software. OTOH, I
haven't had much occasion to use that feature (once upon a
time almost all the software I wrote was scientific, but it
seems I've pretty much given it up -- not that I
particularly planned to, but it's turned out that way).

I was under the impression that pydoc already interpreted
restructured text notation, but maybe I was wrong. I don't
like pydoc because it doesn't make useable static
documentation sets -- that's where epydoc (which I suppose
stands for "extended pydoc") shines.

Although it's great to have "manpage" type help, I
personally find HTML documentation much easier to read and
browse. Pydoc can do this if you are willing to use it as a
server, but it doesn't do so well at making an in-package
website, which is what I usually want to do.

There's also happydoc, which was better than either at
discovering documentation, but stagnated somewhere between
2.x and 3.x with bugs that make it fairly unusable. What
would be cool is if some of happydoc's unique features were
ported to epydoc (such as getting information from comments
as well as docstrings).

 

[aaronwmail-usenet]

I agree that more progress is needed on the Python documentation
front. For example if you look at the "codecs" module documentation
there is no hint of what a codec is anywhere that I can see. Also
the distinction between an "encoder" and a "decoder" is not explained.
Even though I've used it many times and understand it, I still find
myself using the interactive interpreter to make sure I'm sending the
bytes in the right direction...

Perhaps some faqwiz/wiki-like tool to allow the broader community to
propose documentation enhancements would be useful?

-- Aaron Watters

===

"I know what it's like to put food on my family." G.W.Bush

 

[john_sips_tea]

Well, we've already got a wiki, of course: http://wiki.python.org/moin/

Regarding the docs for the module you're asking about, the way it's
supposed to work is (I think), you're supposed to checkout the Python
source, add your docs to the docstrings of that module, then either
commit your changes back in (if you have access privileges) or else
send a patch to the right list (python-dev?) so your patch can be
integrated.

It seems to me that it's best to write your docstring as epytext. That
way, for now, it just looks like regular plain text (though neatly
formatted) when viewed with the pydoc tool...

 

[aaronwmail-usenet]

hmmm. Interesting about the wiki.
It's unusable in my version of IE. Javascript error
on almost every keystroke !

http://wiki.python.org/moin/

It works in Firefox, which I have, of course, but
still...

And the patch procedure you described requires
a higher degree of motivation (and free time) than
most potential contributors might have on offer, imo.

-- Aaron Watters

===
"I weep for you," the walrus said, I deeply sympathize.
With sobs and tears he sorted out those of the largest size.
Holding his pocket handkercheif before his streaming eyes.
from "The Walrus and the Carpenter", Lewis Carroll

 

[john_sips_tea]

And the patch procedure you described requires

a higher degree of motivation (and free time) than
most potential contributors might have on offer, imo.

Another option is to simply email the author/maintainer
for a given module your modifications to their module.

cd ~/dev/python/modified_modules
cp /path/to/module_needs_docs.py .
vim ./module_needs_docs.py # add docs

 

[Paul Boddie]

(e-mail address removed) wrote:

[Quoting Aaron Watters - *the* Aaron Watters?!]

The patch procedure described seemed to involve mailing python-dev,
which I thought was not the accepted practice. Even making patch
descriptions in SourceForge won't apparently get patches straight into
Python, although there may be fewer restrictions on submitting
documentation patches.

Another option is to simply email the author/maintainer
for a given module your modifications to their module.

cd ~/dev/python/modified_modules
cp /path/to/module_needs_docs.py .
vim ./module_needs_docs.py # add docs

True, but I'm not convinced this is going to work when the modules are
in the standard library. Anyway, there were going to be some
improvements to the processes, last time I looked, and various editable
resources did spring up to meet the need for documentation edits - none
of them official as far as I know, though.

Paul

P.S. On the documentation-from-sources question, I've had some success
with epydoc. Some documentation systems (even those for
statically-typed languages) have you describe parameters in
mind-numbing detail, but all I would expect from a documentation tool
is that it highlight mentions of the parameter names in my docstrings.
Having some understanding of the nasty issues around deducing types, I
wouldn't expect a tool to fill in the gaps about parameter types: good
documentation for dynamic languages like Python should be able to
describe what kind of objects should be passed in, and type deduction
could actually give quite the wrong impression about acceptable
arguments to functions, although interface descriptions could be
helpful.

 

[Steve]

I have found the Python sidebar VERY helpful:
http://projects.edgewall.com/python-sidebar/

 

[msoulier]

While epydoc is nice, I'll point out that one thing that Unix people
like myself really like is to be able to check docs on a remote server
that we're logged into via a terminal session. The help() function in
the interpreter is great for this, although it seems that python eggs
broke it.

Please don't leave the terminal behind. Perl provides docs in all
formats, since if you can write POD (plain old documentation), you can
run perldoc, or run pod2html, or pod2man, etc.

I'll also point out that a lot of these complex solutions are ignoring
the KISS principle. POD takes about 5 minutes to learn, tops.

 

[msoulier]

I'd love to have a unified documentation system where *all* the

documentation for *all* installed modules was available to pydoc *and*
the web browser and *all* this documentation was in .py files.

Seconded!

Mike

 

[msoulier]

I have found the Python sidebar VERY helpful:

Personally, I can't use local docs on my desktop as they may not be the
same version of the docs for the Python distro running on the server
that I'm deploying on. I usually go to python.org and use the wayback
machine to look at the old docs for the release that I'm on.

But, if Python would match Perl for docs available on the command-line,
then I'd have it all at my fingertips. I simply don't understand why
this is not being done. When I'm coding in C, I use the manpages on the
remote host so that I know the docs are correct for my target. Why
can't I do that in Python? It's yet another thing that my Perl-using
coworkers point out as a Python weakness.

Mike

 

[Paul Boddie]

But, if Python would match Perl for docs available on the command-line,
then I'd have it all at my fingertips. I simply don't understand why
this is not being done. When I'm coding in C, I use the manpages on the
remote host so that I know the docs are correct for my target. Why
can't I do that in Python?

What about pydoc?

% pydoc os

[Clear screen]

Help on module os:

NAME
os - OS routines for Mac, DOS, NT, or Posix depending on what
system we're on.

FILE
/usr/lib/python2.4/os.py

MODULE DOCS
http://www.python.org/doc/current/lib/module-os.html

DESCRIPTION

[...]

It's yet another thing that my Perl-using coworkers point out as a Python weakness.

In my experience, Perl-using coworkers tend to dredge up all sorts of
"weaknesses", although many of them are no more than cheap shots: the
name, whitespace, and so on. Sure, Python's documentation system could
be better, but let's consider the tools we already have first.

Paul

 

[john_sips_tea]

[snip]
But, if Python would match Perl for docs available on the command-line,
then I'd have it all at my fingertips. I simply don't understand why
this is not being done. [snip]

Mike

Ok, well, here's my attempt to begin to make that happen:

http://www.simisen.com/jmg/cpd/

This "release" is as alpha as alpha gets. It's so alpha it
actually loops back around to zeta -- but it's a start, and I
think it's exactly what the Python community needs.

The command to read the docs works like pydoc but is
called dennis (extra points for getting the obscure Monty
Python reference before reading the web page). The dennis
command is only a few lines of Python held together by bailing
wire, so it will need lots of work, but it lets us read the docs
so it's ok for now. You're invited to fix it up and send me an
improved version.

Please have a look and let me know what you think. If people
start writing modules (that is, documentation) and sending
them to me, I'll probably be forced to start a mailing list and
actually put this stuff in CVS/SVN/bzr.

Have at it!

---John