Python documentation too difficult for beginners

[jk]

Hi,

I've been coding in PHP and Java for years, and their documentation is
concise, well structured and easy to scan.

Others have mentioned this apparently for years (see:
http://stackoverflow.com/questions/4046166/easy-to-navigate-online-python-reference-manual/4070851
and http://www.russellbeattie.com/blog/python-library-docs-still-suck
and http://xahlee.org/perl-python/xlali_skami_cukta.html).

Compare for instance the differences in ease of use, and speed of use,
of these:

http://docs.python.org/library/functions.html#open
http://uk.php.net/manual/en/function.fopen.php

The former is difficult to find (try searching for 'open' in the
search box and see what you get). It is simply a collection of
paragraphs without strong enough contrast to differentiate the
different parts - parameters, parameter values, return types,
exceptions and related functions. It is slow to read and doesn't allow
easy visual scanning.

The latter has clearly delineated, standardised content areas for each
of these without excessive text. It uses tables which are easy to scan
for possible opening modes and allows users to contribute their own
examples.

Sadly, the use of restructured text by python doesn't allow a new
document generator to be written - all existing documentation would
need updating with docblocks or something similar.

Has anyone else struggled while trying to learn the language? The
whole documentation system seems geared towards people who already
know what they're looking for and is close to useless for beginners.
I'm not the only one who finds google an easier way to find
documentation about python.

Is there much chance that the Python maintainers will change their
documentation system to make it more like Java or PHP? How would I go
about trying to make that happen?

 

[jk]

This (http://epydoc.sourceforge.net/stdlib/) is what I'm talking
about.

Why aren't the official docs like this, and why has it taken me 2 days
of searching? All this needs is a search engine behind it and it'd be
perfect.

 

[Tim Wintle]

This (http://epydoc.sourceforge.net/stdlib/) is what I'm talking
about. Aaaarrrgggghhhh

Why aren't the official docs like this,

Because not everyone likes documentation like that. Personally I far
prefer the existing documentation to the JavaDoc-style link you sent.

and why has it taken me 2 days of searching?
All this needs is a search engine behind it and it'd be
perfect.

Personally I use Google, e.g.

"list site:docs.python.org"

to bring up documentation about the list type.

 

[Tim Golden]

This (http://epydoc.sourceforge.net/stdlib/) is what I'm talking
about.

Why aren't the official docs like this, and why has it taken me 2 days
of searching? All this needs is a search engine behind it and it'd be
perfect.

I'm glad you find the epydoc format useful. And I'm glad that various
people have taken the trouble to produce documentation for Python in
various formats that suit them. But why do you imagine that the core
Python documentation -- developed and maintained by a group of people
who clearly have some idea what they're doing -- should change to a
format which happens to suit you?

The Python documentation source and the source code of Python itself
are all freely available. Any initiative by you or by others to
produce alternative, possibly searchable and commentable, versions of
them would I'm sure be welcomed by many. But not everyone finds, eg,
the PHP style of user annotation helpful. Not everyone likes epydoc
output: I don't myself.

In short, please feel free to contribute directly to the core
documentation effort, or to produce alternatives yourself and to
advertise them here or elsewhere within the Python community. But
please don't come along and say "Why aren't the Python docs like
<this other thing> which happens to suit me better?"

TJG

 

[Martin P. Hellwig]

On 11/02/10 10:42, jk wrote:

Is there much chance that the Python maintainers will change their
documentation system to make it more like Java or PHP? How would I go
about trying to make that happen?

I am by no means an authority however since you ask it here I feel
compelled to give you my opinion

In general I would think that more documentation is always welcome, if
you feel like you can make a contribution, excellent, please do!

However, I found that the documentation available was enough for me, and
I didn't even have to go to the googles for that.

Typing help(thing_i_want_info_of) in the interpreter gives me precise
consistent information for what I need to do with whatever it is I am
doing and yes that is largely a replication of what is mentioned on the
site itself (well more the other way around actually).

In the odd cases this doesn't help me, I google for examples.
If that doesn't help I look at the modules __file__ and open that module
to read the source.

And when I started 10 odd years ago with Python it was my first language
with no prior confusion of other languages, since then I extended my
knowledge with C and assembler but on a day to day basis I still use Python.

 

[Antoine Pitrou]

This (http://epydoc.sourceforge.net/stdlib/) is what I'm talking
about.

Why aren't the official docs like this, and why has it taken me 2 days
of searching?

What's wrong with this:
http://docs.python.org/library/
?

If you have specific ideas for improvements, you can open issues at
http://bugs.python.org.

Thank you

Antoine.

 

[jk]

But why do you imagine that the core
Python documentation -- developed and maintained by a group of people
who clearly have some idea what they're doing -- should change to a
format which happens to suit you?

It's not just me who's found the current documentation frustrating.
And sure, the developers know how to code, but they probably can't see
the project with the eyes of a beginner any more.

Making a change to how code is documented to allow more javadoc-style
documentation to be produced could help people migrate from a java
background and ease the learning curve for them, leading to wider
adoption of the language. It wouldn't necessarily mean that the
current documentation style would need to change either.

In short, please feel free to contribute directly to the core
documentation effort, or to produce alternatives yourself

I may well do that.

@Tim Wintle

Personally I use Google, e.g.
"list site:docs.python.org"
to bring up documentation about the list type.

Surely you shouldn't have to go to google though? Or the interpreter?
Maybe it's just what you're used to, but I'd expect the language's web
site to provide enough of a reference in itself, while using google
for examples.

 

[Steven D'Aprano]

Hi,

I've been coding in PHP and Java for years, and their documentation is
concise, well structured and easy to scan.

Well, that's one opinion.

Others have mentioned this apparently for years (see:
http://stackoverflow.com/questions/4046166/easy-to-navigate-online- python-reference-manual/4070851
and http://www.russellbeattie.com/blog/python-library-docs-still-suck
and http://xahlee.org/perl-python/xlali_skami_cukta.html).

Compare for instance the differences in ease of use, and speed of use,
of these:

http://docs.python.org/library/functions.html#open
http://uk.php.net/manual/en/function.fopen.php

The former is difficult to find (try searching for 'open' in the search
box and see what you get).

A fair point -- the built-in open comes up as hit #30, whereas searching
for open in the PHP page brings up fopen as hit #1. But the PHP search
also brings up many, many hits -- ten pages worth.

But in any case, the Python search functionality could be smarter. If I
had a complaint about the docs, that would be it. Fortunately, I have
google

It is simply a collection of paragraphs
without strong enough contrast to differentiate the different parts -
parameters, parameter values, return types, exceptions and related
functions. It is slow to read and doesn't allow easy visual scanning.

It's *nine* paragraphs, three of which are one-liners, the longest of
which is eight lines. If you have trouble reading that, well, you have a
problem. The PHP docs for fopen are FIFTY-EIGHT paragraphs.

Okay, okay, I was unfair. I counted section headings as separate
paragraphs. A more reasonable count is... twenty-six paragraphs, tables,
sections and subsections. Plus *dozens* of user-contributed recipes, bug
reports, tricks, tips and comments. And you call this concise???

Reading the docs, I'd say that PHP needs all this extra documentation
because it's so much more complicated. fopen has all this implicit magic
behaviour that needs documenting -- it will try to guess a scheme from
the file name, if it can't guess the scheme it will guess that it's a
local file, and the behaviour depends on various globals. In comparison,
Python's open is very simple: it only opens files. No wonder Python's
docs are simpler.

The PHP docs felt it necessary to give a warning *three times*, one after
the other, about using binary mode when opening files. Apparently once
was not enough.

The Description subsection of the PHP fopen doc says:

fopen() binds a named resource, specified by filename, to a stream.

What's a stream? So I read, and read, and read, and eventually, almost at
the bottom of the official docs, I find the section "Return Values":

Returns a file pointer resource on success, or FALSE on error.

Great! Now, what's a file pointer resource, and how does it differ from a
stream? No idea.

Contrast that with the Python docs. In the *very first sentence*, it says:

Open a file, returning an object of the file type described in
section File Objects.

with both "file" and "File Objects" being hyperlinks to the appropriate
part of the docs. I think I'll stick with the Python style, thank you
very much.

The latter has clearly delineated, standardised content areas for each
of these without excessive text. It uses tables which are easy to scan
for possible opening modes and allows users to contribute their own
examples.

There has been discussion on python-dev about user contributed examples.
The pros are that users can add tricks and tips. The cons are that,
without constant attention, the user-contributed content will grow old
and stale, or worse, be filled with nonsense.

However, there is a Python wiki. It doesn't get anywhere near as much
love as it deserves, and (I think) the consensus was that the official
Python docs should stay official, but link to the wiki for user-
contributed content. This hasn't happened yet.

http://wiki.python.org/moin/

Sadly, the use of restructured text by python doesn't allow a new
document generator to be written - all existing documentation would need
updating with docblocks or something similar.

Has anyone else struggled while trying to learn the language? The whole
documentation system seems geared towards people who already know what
they're looking for and is close to useless for beginners. I'm not the
only one who finds google an easier way to find documentation about
python.

Why do you think this is a bad thing? The Python docs are the reference
manual, not a tutorial. Quite frankly, if I were a PHP developer, I'd be
pretty annoyed at having to read this in the docs for fopen:

If you use the wrong line ending characters when writing your
files, you might find that other applications that open those
files will "look funny".

Gosh, really? Thanks for the tip, Captain Obvious.

It's always difficult to know how much information is too much. The PHP
docs seem to take an "everything including the kitchen sink" approach.
Given that approach, it makes sense to divide everything into
subsections, one page per function. But with Python's minimalist
approach, it would just be annoying. Compare the four lines of:

http://docs.python.org/library/functions.html#id

with this re-write in the PHP fashion:


=====
id
=====
(Python 1.x, Python 2.x, Python 3.x)

id -- id of an object


Description
-----------

id(object)

id returns the numeric "identity" of an object, which is guaranteed to be
unique and constant for this object during its lifetime.

Note: two objects with non-overlapping lifetimes may have the same id()
value.

Note: CPython implementation detail: This is the address of the object.


Parameters
----------

* object

Any object.

Note: all data in Python are objects, even ints and strings.

Note: there are no undefined objects in Python. If you call
id(variable) on an unbound variable name, Python will raise an
exception.

Return values
-------------

Returns an integer or long integer object representing the ID of the
argument.


Errors/exceptions
-----------------

If the argument to id() is a named variable rather than a literal, and
that name is not bound to any object, then a NameError will be raised.
Otherwise every call to id() must succeed.

Note: if the call to id() is inside a function, the exception may be a
subclass of NameError such as UnboundLocalError.

Note: literals are not guaranteed to always refer to the same object.


Changelog
---------

0.9 First version added (I think).


Examples
--------

id(x)
id(alist[1])
id(instance.attribute)
id(module.name.attribute['key'].method(arg1, arg2).seq[2])


Notes
-----

If you're still reading, I admire your persistence.


See also
--------

Python's object model
Exceptions

Is there much chance that the Python maintainers will change their
documentation system to make it more like Java or PHP? How would I go
about trying to make that happen?

Unlikely. You could raise the issue on the python-dev list, or see if
there is a SIG mailing list specifically for the docs.

Frankly, I think that the best thing you could do is start a fork of the
docs and see if you get any interest from people. If you do, then you can
go back to python-dev with proof that there is a genuine popular desire
for more structured, PHP-style documentation.

 

[jk]

It's always difficult to know how much information is too much. The PHP
docs seem to take an "everything including the kitchen sink" approach.
Given that approach, it makes sense to divide everything into
subsections, one page per function. But with Python's minimalist
approach, it would just be annoying. Compare the four lines of:

http://docs.python.org/library/functions.html#id

with this re-write in the PHP fashion:

=====
id
=====
(Python 1.x, Python 2.x, Python 3.x)

id -- id of an object

Description
-----------

id(object)

id returns the numeric "identity" of an object, which is guaranteed to be
unique and constant for this object during its lifetime.

Note: two objects with non-overlapping lifetimes may have the same id()
value.

Note: CPython implementation detail: This is the address of the object.

Parameters
----------

* object

  Any object.

  Note: all data in Python are objects, even ints and strings.

  Note: there are no undefined objects in Python. If you call
  id(variable) on an unbound variable name, Python will raise an
  exception.

Return values
-------------

Returns an integer or long integer object representing the ID of the
argument.

Errors/exceptions
-----------------

If the argument to id() is a named variable rather than a literal, and
that name is not bound to any object, then a NameError will be raised.
Otherwise every call to id() must succeed.

Note: if the call to id() is inside a function, the exception may be a
subclass of NameError such as UnboundLocalError.

Note: literals are not guaranteed to always refer to the same object.

Changelog
---------

  0.9  First version added (I think).

Examples
--------

   id(x)
   id(alist[1])
   id(instance.attribute)
   id(module.name.attribute['key'].method(arg1, arg2).seq[2])

Notes
-----

   If you're still reading, I admire your persistence.

See also
--------

   Python's object model
   Exceptions

Steven

You're right in that the python docs in this case are less lines, but
that's one of the problems. It doesn't mention anywhere the extra
detail you've added regarding exceptions thrown. That's the kind of
thing that probably comes through experience or some kind of
convention which isn't obvious to beginners. Having things split into
sections - parameters, return types, exceptions, etc - lets me find
what I'm looking for quickly.

As for the 9 paragraphs statement, there's a usability book that
applies here - it's called "Don't make me think". I shouldn't have to
go through freeform text to find what I'm looking for when a list
would make that information easier to find. And splitting the docs
into sections would allow me to skip to what I'm looking for. I really
would be much happier with your example documentation.

I think the key difference is that I don't want to have to *read* the
python docs - I want to be able to scan for what I'm looking for and
find it easily. That makes me productive.

 

[Paul Rudin]

A fair point -- the built-in open comes up as hit #30, whereas searching
for open in the PHP page brings up fopen as hit #1. But the PHP search
also brings up many, many hits -- ten pages worth.

OTOH googling for "python open" gives you the correct (for 2.7) page as
hit #1 - although you then have to use your browser's "find" facilty to
actually get to the description of the function in question.

 

[Bruno Desthuilliers]

jk a écrit :

Hi,

I've been coding in PHP and Java for years, and their documentation is
concise, well structured and easy to scan.

Others have mentioned this apparently for years (see:
http://stackoverflow.com/questions/4046166/easy-to-navigate-online-python-reference-manual/4070851
and http://www.russellbeattie.com/blog/python-library-docs-still-suck
and http://xahlee.org/perl-python/xlali_skami_cukta.html).

Totally unrelated, but the last example is nothing but a reference -
xahlee is one of the worst internet troll ever.

Compare for instance the differences in ease of use, and speed of use,
of these:

http://docs.python.org/library/functions.html#open
http://uk.php.net/manual/en/function.fopen.php

Sorry but as far as I'm concerned, PHP doc sucks big time, and I find
Javadoc-style stuff close to useless.


(snip)

Has anyone else struggled while trying to learn the language?

Not as far as I'm concerned. I found Python the easiest language to
learn right from the beginning. Not to say the doc couldn't be improved,
or that alternate documentations could help, but I never had any problem
with it.

 

[Terry Reedy]

Compare for instance the differences in ease of use, and speed of use,
of these:

http://docs.python.org/library/functions.html#open
http://uk.php.net/manual/en/function.fopen.php

The former is difficult to find (try searching for 'open' in the
search box and see what you get).

duh. 'open' is a common word and if you make an unstructured search for
it in all text, you should get a lot of hits.

The Python docs have both a Global Module Index (which I use constantly)
and a General Index of functions (methods), classes, and terms. Learn to
use them. If you look in the [o] section for 'open', the first entry is
"open() (built-in function)" -- just what you were looking for. There
are also about 30 more nicely labelled entries for 'open' in various
modules.

> It is simply a collection of
paragraphs without strong enough contrast to differentiate the
different parts - parameters, parameter values, return types,
exceptions and related functions. It is slow to read and doesn't allow
easy visual scanning.

It is possible that this particular entry could be improved.

Is there much chance that the Python maintainers will change their
documentation system to make it more like Java or PHP?

There are plans to make doc feedback from users easier.

 

[Tim Harig]

As for the 9 paragraphs statement, there's a usability book that
applies here - it's called "Don't make me think". I shouldn't have to

Anything that promotes a lack of thinking sends up red flags in my head.
We want to recruit smart people who think, not idiots.

go through freeform text to find what I'm looking for when a list
would make that information easier to find. And splitting the docs
into sections would allow me to skip to what I'm looking for. I really
would be much happier with your example documentation.

ctrl-f will bring up a search dialog in most graphical browsers. '/' will
in many others. With some practice, your fingers will be able to find
something far faster then your eyes can see it happen.

There is a religious war in the GNU community between info page as
documentation versus the traditional manual format. The manual format
contains all of the information on one page that can be easily searched
whereas the info pages are split into sections that must be viewed
individually. With the man pages, you can almost always find what you want
with a quick search through the document. Info pages are much harder to
use because you have to try and figure out which section the author decided
to place the information that you are looking for. The information may be
stored several levels deep, which means that it can be a deep productivity
hit if you don't guess the proper location on the first try.

I think the key difference is that I don't want to have to *read* the
python docs - I want to be able to scan for what I'm looking for and
find it easily. That makes me productive.

The real question is what do you want to gain by your posts here. You
should already know that most groups are, by their very nature, slow to
make changes to the status quo. The problem tends to be exasperated in
open source projects where any changes mean that people have to donate
their time to make anything happen. You will in general find two things to
be true:

1. Since they are dontating their time, you will find that people tend to
scratch their own iches first.

2. People who do take the time to contribute to open source projects are
people of action. They don't tend to be appreciative of those who
constantly generate feature requests but have no inclination to do
any of the work themselves. They do appreciate other people of
action who are interested in making the project better.

Therefore, if you truly want changes in the documentation, I suggest that,
rather then whining to the group, you make some of the changes yourself.
When you are finished, you can post a link to your alternate documentation
to this group. If you documentation is truly better then the existing
documentation, you will not have to say another word. People within the
community will rally around it and promote it. If it gains wide enough
support, then there will be a movement to use it to supplant the existing
documentation. It is the difference between whining from the sidelines and
actively participating in the game.

 

[Ian]

You're right in that the python docs in this case are less lines, but
that's one of the problems. It doesn't mention anywhere the extra
detail you've added regarding exceptions thrown. That's the kind of
thing that probably comes through experience or some kind of
convention which isn't obvious to beginners. Having things split into
sections - parameters, return types, exceptions, etc - lets me find
what I'm looking for quickly.

It doesn't mention it because those exceptions don't actually have
anything to do with the id() function. They're just what happens any
time an unbound variable name is evaluated, in any context. The exact
same thing could be said about any Python function in existence that
takes at least one argument.

Cheers,
Ian

 

[Kee Nethery]

Hi jk,

I totally agree. But you will get nowhere.

A few weeks back I complained that
http://docs.python.org/reference/executionmodel.html#naming-and-binding
was more than a little opaque - and was not understood by Python noobs such as myself.

I was invited to rewrite it and submit an improved version.

In this world of moderated wikis one would think that noobs such as myself could enhance the docs when we find something confusing in the docs. <insert snarky comment here>

Kee

 

[Teemu Likonen]

The manual format contains all of the information on one page that can
be easily searched whereas the info pages are split into sections that
must be viewed individually. With the man pages, you can almost always
find what you want with a quick search through the document. Info
pages are much harder to use because you have to try and figure out
which section the author decided to place the information that you are
looking for.

There is also the problem that people are less familiar with info
browsers than the usual "less" pager which is used by "man" command.

With the text terminal info browser called "info" as well as Emacs' info
browser you can use command "s" (stands for "search"). It prompts for a
regexp pattern to search in the whole document, including subsections
etc.

 

[John Nagle]

OTOH googling for "python open" gives you the correct (for 2.7) page as
hit #1 - although you then have to use your browser's "find" facilty to
actually get to the description of the function in question.

Right. Google does a far better job of organizing Python's
documentation than the Python community does. I don't even try
looking up anything starting at Python.org; I always start
with a Google search. Even though Python.org's search is
powered by Google, it's inferior to a general search.

Compare:

http://www.google.com/search?domains=www.python.org&sitesearch=www.python.org&q=open

http://www.google.com/search?q=Python+open



John Nagle

 

[Tim Harig]

There is also the problem that people are less familiar with info
browsers than the usual "less" pager which is used by "man" command.

I thoroughly agree. The default info viewers are quite possibly the most
counterintuitive programs I have ever encountered. I never did bother to
learn how to use them. I instead installed the more intuitive pinfo
program.

With the text terminal info browser called "info" as well as Emacs' info
browser you can use command "s" (stands for "search"). It prompts for a
regexp pattern to search in the whole document, including subsections
etc.

Right, pinfo offers this as well; but, then you have to figure out where in
the nodes that the search has taken you and how to navigate from that node
to find additional information that you may need. I have, in general, come
to think of info pages as a failed experiment and I know very few people
who actually prefer them over the simpler man pages.

 

[Tim Harig]

Right, pinfo offers this as well; but, then you have to figure out where in
the nodes that the search has taken you and how to navigate from that node
to find additional information that you may need. I have, in general, come
to think of info pages as a failed experiment and I know very few people
who actually prefer them over the simpler man pages.

I should add two more things here:

1. Another confusing aspect of the info pages is that you often have to
know what package a command came from or you don't get the
information that you are looking for.

2. Series of man pages can be used in a way that seem like they have a
structure as they can effectively link to other pages. If I open
one of the ncurses man pages in pinfo, I can follow what seem like
links to other man pages. I can open the main curses page and I
effectively get an index to all of the other curses functions.

 

[John McMonagle]

Right. Google does a far better job of organizing Python's
documentation than the Python community does. I don't even try
looking up anything starting at Python.org; I always start
with a Google search. Even though Python.org's search is
powered by Google, it's inferior to a general search.

Compare:

http://www.google.com/search?domains=www.python.org&sitesearch=www.python.org&q=open


http://www.google.com/search?q=Python+open

Even better:


http://www.google.com/search?sitesearch=docs.python.org&q=open


Regards,

John McMonagle