Python docs disappointing

[greg]

Such a reorg is not a simple matter
of moving a file from here to there. It will require a lot
moving about of sections and a lot of word-smithing to glue
them back together again in a coherent way.

Concerning this particular issue, not everyone would
agree that the doc sections in question are misplaced.

In Python the distinction between builtin and library
types is fairly blurry. There are advantages to having
them all documented in one place -- when looking for
info about a type, you don't have to try to guess
whether it's considered built-in or not to know which
manual to look in.

 

[Steven D'Aprano]

But that's the problem. Such a reorg is not a simple matter of moving a
file from here to there. It will require a lot moving about of sections
and a lot of word-smithing to glue them back together again in a
coherent way.

That's no different from *any* major refactoring. The exact same problem
exists for code as well as documentation. It's a solved problem for code,
and it's a solved problem for documentation.

In some order:

Consensus that there is a problem that needs solving;
Volunteer(s) to do the work;
Somebody to take responsibility to check the changes in.


Sometimes, in the face of apathy or disagreement (which may very well be
justified), you have to at least partly solve the problem *before* people
will agree that it needed to be solved. Welcome to the real world.

A tracker issue, even one that was fairly specific about how things
should be reorganized but which did not provide all the needed changes
(a large patch) would almost certainly be ignored or rejected.

Yes it would. Most patches are ignored, because the dev team are
overworked, and if they don't see the need for a patch, they won't
approve it.

But
providing a large patch raises two questions. How likely is it to be be
accepted? (Something anyone would want to know before investing the
time.) And how to actually do the work needed to generate it (it could
be a very large amount of work for an individual and I don't think it
can be broken down into small independent issues.) How is one to test
the waters as to acceptability, or solicit help, if one simply submits a
tracker issue?

No, submitting a tracker issue is a necessary but not sufficient step.
Without a tracker issue, you're very unlikely to have people agree to
replace the existing docs with your docs, although a PEP would probably
do it. (A PEP is significantly more effort than raising a tracker issue.)

You also have to get attention from those with check-in privileges. If
they approve your patches, you're done. If they don't approve them, or
fail to respond, you can try convincing them, or taking it to the Python-
Dev list and request somebody review your patches.

If you have no patches, perhaps because you're unwilling to invest the
effort without a guarantee that it will be treated seriously, then you
need to get some sort of consensus among the relevant people that the
problem is worth solving.

Guess what? Sometimes others will disagree with you. What you see as the
Worst. Docs. EVAR. may be seen as perfectly adequate, even excellent, by
others. If this is the case, remember, Python isn't your product, you
don't have veto over what goes into it. Feel free to publish your
alternatives. Write a book. Go home and cry into your beer over it.
Whatever. Sometimes you win, and sometimes you don't.

This is the same process whether dealing with code or documentation.

As luminous a Pythoneer as Fredrik Lundh set up a documentation wiki to
collect user contributions but it has faded away. If he couldn't pull
it off then I'd say most other attempts without the backing of
python.org are almost certainly doomed to failure.

Perhaps that's a good indication that a Python wiki *doesn't* fulfill a
need in the community, and that despite what a few squeaky wheels think,
the docs *are* doing a good job?

As long as every "the docs
sux" complaint is met here with the standard responses that I've
mentioned,

But they aren't met with such a so-called "standard response".

rather than, "yes, they do have some problems, what do you
think we should do about them?",

We know that there are problems. We've said repeatedly that corrections
and patches are welcome. We've repeatedly told you how to communicate
your answer to the question of what should be done. None of this is good
enough for you. I don't know what else you expect.

Eh? I have a computer filled with software that I put no money or
effort into, yet there it is.

Ah yes, I'm sorry, I missed one other alternative: sit around and wait
for somebody else to put in the money and/or effort.

That some of us choose to
invest it somewhere other than Python does not deprive of of our right
to point out problems in Python when we note them.

Of course not. But it does mean that you won't be taken seriously, and
you have no right to be taken seriously.

 

[Hendrik van Rooyen]

You want community input into the docs, but you're not willing to give
that input except to bitch and moan and sook that the tracker is no good.

wtf does the verb "sook" mean?

I find:

sook
  /sʊk/ Show Spelled Pronunciation [sook] Show IPA
Use sook in a Sentence
–noun
1. Australia and New Zealand. a timid, cowardly person, esp. a young person;
crybaby.
–interjection
2. Midland U.S. (used to summon cows from the pasture).
Origin:
1890–95; prob. from earlier sense “calf reared by hand,†perh. suck(-calf),
with sp. repr. N England, Scots pron. of suck (but earliest cited pron. of
sook is (soÌ…oÌ…k))

Sometimes English drives me crazy.

- Hendrik

 

[Steven D'Aprano]

You want community input into the docs, but you're not willing to give
that input except to bitch and moan and sook that the tracker is no
good.

wtf does the verb "sook" mean?

I find:

sook
  /sʊk/ Show Spelled Pronunciation [sook] Show IPA
Use sook in a Sentence
–noun
1. Australia and New Zealand. a timid, cowardly person, esp. a young
person; crybaby.

That's the one. It's also used as a verb.

 

[Paul Boddie]

We know that there are problems. We've said repeatedly that corrections
and patches are welcome. We've repeatedly told you how to communicate
your answer to the question of what should be done. None of this is good
enough for you. I don't know what else you expect.

Maybe the problem is that although everyone welcomes contributions and
changes (or says that they do), the mechanisms remain largely beyond
criticism. Consequently, one sees occasional laments about there not
being enough people contributing to Python core development and soul-
searching about the reasons this might be so. If it were insisted that
changes to, say, Wikipedia were to be proposed by submitting a patch
or report for perusal by the editors and for future inclusion in some
version of the project, the whole project would most likely be a
shadow of its current self, and ambitions of large-scale collaborative
editing in general would still be ridiculed.

A free-for-all isn't likely to be the best solution for more actively
edited Python documentation, but Wiki solutions undeniably provide a
superior "fast path" for edits by trusted users to be incorporated and
published in accessible end-user documentation. Almost every analysis
of the current (and previous) documentation mechanisms has identified
the editorial workflow as a bottleneck and then proceeded to replicate
such a bottleneck in any proposed solution. I'm starting to believe
that there's a certain snobbery about Wiki solutions which lead many
people to develop all sorts of short-term, arcane solutions under the
illusion that something super-special and customised is necessary and
that they have to start virtually from scratch in order to cater to
the ultra-special needs of the target audience; by the time they're
done, no-one's interested any more, except to propose the next legacy
system in the making.

[...]

Of course not. But it does mean that you won't be taken seriously, and
you have no right to be taken seriously.

That's an absurd position that has soured the reputation of numerous
projects. When someone spends the time to write a bug report, they are
often investing as much time and effort in something that they are
able to in any productive sense. I make a habit of submitting bug
reports to software distributions, typically so that the people who
are responsible for the components involved can investigate the
problem effectively. When the maintainers just close such reports or
mark them with a number of different labels which mostly indicate that
they consider those reports not worth their time, it sends the message
that they consider their time to be vastly more important than their
users, even though their users might have set aside an hour of their
potentially busy schedule which might have meant sacrificing something
else that should have taken higher priority (like time for sleeping,
in my own personal experience). Thus, I've had the impression with
some projects that I should be maintaining all sorts of stuff - the
bootloader, the kernel, various desktop applications, Mozilla - all so
that stuff actually gets fixed and that I'm not perceived as just
another user whose bug reports aren't welcome. I don't find this
reasonable at all when in many cases there *are* people getting paid
to do these jobs.

The Python core developers seem more attentive than in various other
projects, but please let us not establish the "delicate genius"
mentality that has infested other projects to the point that any
criticism is automatically labelled as ungrateful whining by people
who supposedly don't use the software, have an axe to grind, and who
are apparent simpletons who don't understand the revolutionary vision
of the project leadership. If you think throwing away goodwill is an
acceptable way of silencing complaints, please take a look at just
about any article about KDE 4 that permits reader comments to see how
much goodwill can be lost and what effect that has on a project's
reputation.

Paul

 

[Steven D'Aprano]

Maybe the problem is that although everyone welcomes contributions and
changes (or says that they do), the mechanisms remain largely beyond
criticism. Consequently, one sees occasional laments about there not
being enough people contributing to Python core development and soul-
searching about the reasons this might be so. If it were insisted that
changes to, say, Wikipedia were to be proposed by submitting a patch or
report for perusal by the editors and for future inclusion in some
version of the project, the whole project would most likely be a shadow
of its current self, and ambitions of large-scale collaborative editing
in general would still be ridiculed.

If Python had the tens of thousands of users, and hundreds of trusted
(for some definition of trusted) editors, then Python could run using the
same model as Wikipedia. The Wikipedia model is great, and I contribute
to it myself.

But Wikipedia gets its users from the entire population of web-users,
because there's something of interest to everyone in Wikipedia.
Interested in movies? There are Wikipedia pages for you to contribute to.
Interested in medicine? There are pages you can help with. Interested in
the history and development of the mechanical pencil? There's probably
even a page for you. And if there isn't, you can create one.

With tens of millions of web users, it's no surprise that Wikipedia can
attract thousands of editors. But this does not apply to Python, which
starts from a comparatively tiny population, primarily those interested
in Python. Have a look at the Wikipedia page for Python. The Talk Page
has comments from no more than *eight* people. The History stats suggest
that, over seven years, only sixty-nine people have made more than a
single edit to the page, most of them having made just two edits. Just 36
people have made more than two edits, and some of those are bots. Only
one user, Lulu of the Lotus-Eaters (David Mertz), has made more than 100
edits.

A free-for-all isn't likely to be the best solution for more actively
edited Python documentation, but Wiki solutions undeniably provide a
superior "fast path" for edits by trusted users to be incorporated and
published in accessible end-user documentation.

And the Python time-machine strikes again:

http://wiki.python.org/moin/

That's an absurd position that has soured the reputation of numerous
projects. When someone spends the time to write a bug report, they are
often investing as much time and effort in something that they are able
to in any productive sense.

Firstly, in context, I wasn't talking to somebody who had made bug
reports. I was talking to somebody whose only contribution, as near as I
can tell, was to loudly complain that there are flaws in the Python
documentation and insist that somebody else should fix them just the way
he wants them fixed -- without being willing to even explain how he wants
them fixed. Possibly the developers are expected to intuit from first
principles what he wants.

Secondly, the world is full of complainers who won't lift a finger to
help but demand others help them. It may be unfair to tar everybody with
the same brush, but life is to short and time to valuable to avoid making
judgements as to who to pay attention to. Those who invest a lot of
effort into providing patches get listened to closely; so do those who
make good quality detailed bug reports. Those who just say "It's broken,
fix it" don't. Sometimes that will mean that someone with genuinely good
ideas will be ignored, but that's the price one pays for avoiding being
drowned by a chorus of trivial, contradictory, vague and insubstantial
complaints.

If the Python Dev team paid attention to every post here claiming that
Python "has a bug" when the bug was actually in the complainant's own
code, we'd probably still be running Python 1.5.

I make a habit of submitting bug reports to
software distributions, typically so that the people who are responsible
for the components involved can investigate the problem effectively.
When the maintainers just close such reports or mark them with a number
of different labels which mostly indicate that they consider those
reports not worth their time, it sends the message that they consider
their time to be vastly more important than their users, even though
their users might have set aside an hour of their potentially busy
schedule which might have meant sacrificing something else that should
have taken higher priority (like time for sleeping, in my own personal
experience).

Oh dear me. You mean that they don't agree that YOUR time is more
important than theirs??? What horrible people they must be, to expect you
to sacrifice some of your sleep time just so *they* can get some sleep
themselves!!! Who do they think they are???

 

[Antoine Pitrou]

A free-for-all isn't likely to be the best solution for more actively
edited Python documentation, but Wiki solutions undeniably provide a
superior "fast path" for edits by trusted users to be incorporated and
published in accessible end-user documentation.
Agreed.

I'm starting to believe
that there's a certain snobbery about Wiki solutions which lead many
people to develop all sorts of short-term, arcane solutions under the
illusion that something super-special and customised is necessary and
that they have to start virtually from scratch in order to cater to
the ultra-special needs of the target audience; by the time they're
done, no-one's interested any more, except to propose the next legacy
system in the making.

Not sure why you think it's snobbery... There are certain tacit expectations
regarding the docs:
- that they are versioned with the source tree (because, often, changes in
documentation will be synchronized with changes in behaviour / functionality,
because we must maintain documentation for several versions at once, because you
want to use the same kind of merging that is used between different branches)
- that they can be used offline, rebuilt in different formats, etc.
- that you don't need a Web server (even locally) to navigate through them
- that proposed changes are to be reviewed by maintainers (core developers)
before they get actually committed

I'm not sure of any existing wiki system which fits the bill. So, while I agree
that the current situation can present a high threshold for occasional doc-only
contributions, there doesn't seem to be a simple solution to improve things.

 

[Paul Boddie]

With tens of millions of web users, it's no surprise that Wikipedia can
attract thousands of editors. But this does not apply to Python, which
starts from a comparatively tiny population, primarily those interested
in Python. Have a look at the Wikipedia page for Python.

What does the Python entry on Wikipedia have to do with editing the
Python documentation in a Wiki? Once everyone has agreed that the
description of Python on Wikipedia is reasonable, there's not much
point in editing it, is there? In contrast, there's a continuous
stream of people who don't think Python's documentation is that great.

[...]

And the Python time-machine strikes again:

http://wiki.python.org/moin/

And I suggested that the complainants use it as a starting point.

[...]

Oh dear me. You mean that they don't agree that YOUR time is more
important than theirs??? What horrible people they must be, to expect you
to sacrifice some of your sleep time just so *they* can get some sleep
themselves!!! Who do they think they are???

That's quite an attempt to make my position more extreme than it
actually is. I get people asking me to improve my own software, you
know, and even if I don't have the time or inclination to do what they
ask, I do spend time discussing it with them. Such people, including
myself when I'm on the other side of the fence, appreciate more than
just a brush-off and no: they don't insist that their own time be
valued above anyone else's (as you would have me misrepresented); they
just ask that their own efforts aren't treated as having no value
because they're not part of the "elite" development team. You get
various projects doing soul-searching about embracing the efforts of
non-programmers, and the first port of call on that particular voyage
is to not treat them like idiot consumers whose remarks can only be
considered as mere heckling while the visionaries act out their
flawless production.

Paul

P.S. The mention of "social problems" ties in with other remarks made
recently, and I've increasingly found it more trouble than has been
worthwhile to pursue Python-related matters of late. When one tries to
encourage people to participate in improving various things, which
usually means the community having to accept a degree of criticism,
people claim that it's encouraging "undesirable" influences to point
such critics in the right direction instead of showing them the door.
When one tries to pursue such improvement matters oneself, people
always have something to say about the choice of technology or whether
they like the particular background colour being used or indeed have
an opinion, typically shallow and negative, about anything but the
task at hand, and there'll always be someone queuing up to dismantle
anything that does get done at the first opportunity. In contrast,
I've found other groups of people to be grateful for even modest
technical assistance, and I know that such people are much more likely
to get my support and input in the future than those who think that
it's some kind of advantage to have potential contributors run the
gauntlet of denial (that there are structural problems in their
project), skepticism (that newcomers can have anything to contribute),
discouragement (because any solution which doesn't validate someone's
technology preferences must be criticised) and, indeed, outright
hostility.

One can always spend one's time doing something which isn't 100%
enjoyable or 100% rewarding if one feels that the time is still being
spent on something worthwhile. I'm getting the feeling that lots of
Python-related stuff doesn't quite satisfy such criteria any more.

 

[Steven D'Aprano]

What does the Python entry on Wikipedia have to do with editing the
Python documentation in a Wiki?

Good question. I was responding to you mentioning Wikipedia as a possible
role model for the Python docs.

Once everyone has agreed that the
description of Python on Wikipedia is reasonable, there's not much point
in editing it, is there?

And once we're all fabulously wealthy, there won't be any point in anyone
working any more either!

The problem for your argument is, even if it were correct, not everyone
agrees the Python article is "reasonable" -- there were three edits made
since the 7th of this month. And before that, there was a stream of 22
edits on the 5th, and another 25 edits since the 8th of July. Obviously
the Python article is still in active development.

Some of those edits were, apparently, vandalism, which gives yet another
reason why the Wikipedia model is not necessarily the right model to
follow.

In contrast, there's a continuous stream of
people who don't think Python's documentation is that great.

And a great flood of those who think it's pretty good and gets the job
done adequately, and a trickle of those who think it's perfect just the
way it is.

It's not the people who suggest improvements to the docs that are the
problem, but the ones who insist that the docs are terrible, but aren't
willing to do anything but complain. Oh, and trolls like ... I hesitate
to mention his name in case he has a bot monitoring the list ... first
name starts with "X" followed by "ah", second name sounds like "Mee" ...
who even if they make a few good points, they're lost in a sea of insults
to others, arrogance and self-aggrandisement.

And I suggested that the complainants use it as a starting point.

Sorry, I seem to have missed that.

[...]

Oh dear me. You mean that they don't agree that YOUR time is more
important than theirs??? What horrible people they must be, to expect
you to sacrifice some of your sleep time just so *they* can get some
sleep themselves!!! Who do they think they are???

That's quite an attempt to make my position more extreme than it
actually is.

Well, you did raise the issue of the sacrifices you were making to report
these bugs. All I did was exaggerate the attitude a tad.

I get people asking me to improve my own software, you
know, and even if I don't have the time or inclination to do what they
ask, I do spend time discussing it with them. Such people, including
myself when I'm on the other side of the fence, appreciate more than
just a brush-off and no: they don't insist that their own time be valued
above anyone else's

Then you're incredibly lucky to attract a better class of end-users. In
my experience, most end-users won't even spend the effort to describe the
problem they're having beyond "it doesn't work". And they usually
misspell that.

(as you would have me misrepresented); they just ask
that their own efforts aren't treated as having no value because they're
not part of the "elite" development team. You get various projects doing
soul-searching about embracing the efforts of non-programmers, and the
first port of call on that particular voyage is to not treat them like
idiot consumers whose remarks can only be considered as mere heckling
while the visionaries act out their flawless production.

A noble vision, but wait until the idiot heckling consumers discover your
software, then we'll see how much time you're prepared to give them.

Paul

P.S. The mention of "social problems" ties in with other remarks made
recently, and I've increasingly found it more trouble than has been
worthwhile to pursue Python-related matters of late. When one tries to
encourage people to participate in improving various things, which
usually means the community having to accept a degree of criticism,
people claim that it's encouraging "undesirable" influences to point
such critics in the right direction instead of showing them the door.

Can you point me to a discussion where this has happened?

When one tries to pursue such improvement matters oneself, people always
have something to say about the choice of technology or whether they
like the particular background colour being used

You've discovered bike-shedding.

or indeed have an
opinion, typically shallow and negative, about anything but the task at
hand,

When you're agitating for change, anyone defending the status quo has
opinions which are shallow and negative. When you're happy with the
status quo, possibly even for good, rational reasons and not just because
you're a shallow-minded, ignorant, know-nothing nay-sayer, it's those
agitating for change who have shallow and negative opinions. It's such a
bother trying to determine who is right, so I prefer to just accuse the
other guy of being shallow and negative rather than try to understand his
point of view. I find it saves time in the long run.

and there'll always be someone queuing up to dismantle anything
that does get done at the first opportunity. In contrast, I've found
other groups of people to be grateful for even modest technical
assistance, and I know that such people are much more likely to get my
support and input in the future than those who think that it's some kind
of advantage to have potential contributors run the gauntlet of denial
(that there are structural problems in their project), skepticism (that
newcomers can have anything to contribute), discouragement (because any
solution which doesn't validate someone's technology preferences must be
criticised) and, indeed, outright hostility.

I'm having trouble parsing the semantics of that sentence, so please
excuse me for any misunderstandings.

There seems to be a hidden assumption in your sentence that there *are*
structural problems in the project. Now obviously no project is
"perfect", and we all know of cases where a group of people immersed in a
project will become so close to it that they can't see the rotten trees
for the wood, so to speak. But generally, the people in the project have
a very good understanding of the project's strengths and weaknesses,
because they've been living with it for a long, long time, and it's new-
comers, who don't know the tools, don't know the libraries, aren't
familiar with the conventions, and don't know the reasons for the
decisions made, who can't see the trees for the wood.

It's not experienced Python developers who write really slow Java code in
Python, it's Java developers. And I'm sure the opposite occurs too:
Python developers learning Java for the first time will struggle to write
idiomatic Java code, and will criticise *the wrong things* because
they're thinking like Python developers.

You're expecting outsiders to bring in fresh insights that the jaded,
stuck-in-a-rut locals can't see. Well, it happens. But for every time it
happens, there's a hundred outsiders who come along and say the
equivalent of "What, no GOTO? Well this sucks."

 

[Paul Boddie]

Good question. I was responding to you mentioning Wikipedia as a possible
role model for the Python docs.

Yes, but claiming that only a few people want to edit a single entry
on one site (admittedly a popular one) isn't the same as saying that
few people would edit a documentation Wiki covering numerous different
things. A bunch of people edit the existing Python Wiki now, although
there's not that much direction behind it.

[...]

It's not the people who suggest improvements to the docs that are the
problem, but the ones who insist that the docs are terrible, but aren't
willing to do anything but complain. Oh, and trolls like ... I hesitate
to mention his name in case he has a bot monitoring the list ... first
name starts with "X" followed by "ah", second name sounds like "Mee" ...
who even if they make a few good points, they're lost in a sea of insults
to others, arrogance and self-aggrandisement.

Right, but those good points are still worth taking on board. There
have been Xah Lee posts which have been relatively constructive, but
when the only responses are from people who see the name and can't be
bothered reading the message before issuing a stock "he's a troll"
response, the tone is likely to remain vulgar from that point onwards.
Xah Lee can be quite coherent and rational on comp.lang.lisp, which is
more than can be said for a number of regulars on that group.

[...]

Can you point me to a discussion where this has happened?

I won't name names as in some cases I've corresponded privately with
various people who have been perceived to be "trolls" (as you put it
above) and who have had the "don't talk to them" responses from
various regulars. Some people criticise in apparently unacceptable
ways for their own amusement, but most critics do so because they are
unaware of any better way and aren't aware of the most effective
methods to fix the issues that bother them, and this latter group is
clearly invested in finding solutions because they could quite easily
go and use something else. Certainly, I wouldn't spend my time
repeatedly enumerating the problems with a piece of technology if no-
one were interested in helping me do something about them.

You've discovered bike-shedding.


When you're agitating for change, anyone defending the status quo has
opinions which are shallow and negative. When you're happy with the
status quo, possibly even for good, rational reasons and not just because
you're a shallow-minded, ignorant, know-nothing nay-sayer, it's those
agitating for change who have shallow and negative opinions. It's such a
bother trying to determine who is right, so I prefer to just accuse the
other guy of being shallow and negative rather than try to understand his
point of view. I find it saves time in the long run.

I can expand what I've written to just about any project,
"improvement" or otherwise, where there may not be an existing
solution that anyone actually supports or is willing to use. And
still, if you give people something they could use (which is better
than effectively nothing), my experience is that in some communities
your work, however trivial, will be appreciated. But I get the
impression that in Python-related communities, it's all "Why didn't
you use XYZ?" or "What a toy!" instead.

[...]

There seems to be a hidden assumption in your sentence that there *are*
structural problems in the project.

Let me assume that maybe the barriers aren't really that bad for
Python documentation; that anyone who is really going to care about
submitting something will jump through the hoops and deliver something
that can be merged by the core developers. Even then, there's going to
be a whole class of improvements that won't get made by outsiders and
will fall on the editors to make. Now, more often than not, the people
who are already the most overworked are precisely those in the
position of reviewing and merging changes (as well as making their
own), and surely they will want to delegate some of that work, but
it's the classic pitfall of "delegation failure": they don't have the
capacity to delegate because that just means even more work for them
as a result. Shower such people with trivial change requests or bug
reports about documentation errors, and it's clear that the processes
aren't helping the editors to do more or for an increased number of
people to usefully collaborate.

I'm almost past caring about the documentation - it's useful for most
of what I want it for - because the bulk of the documentation that I
do use concerns the standard library which looks more and more
antiquated and disorganised every time I look at its contents. I don't
see any top-down remedy for this situation - Python 3 didn't even
bother with it properly - and maybe the solution will ultimately
involve a completely independent standard library developed by people
under their own steam who will then be criticised for not choosing the
most permissive licence possible and/or being technologically
backwards in some aspect of what they've done. That would, after all,
be the "Pythonic" response I'd most expect to see.

Paul

 

[Ethan Furman]

Right, but those good points are still worth taking on board.

The responsibility for communication is shared. How much to each party
varies by circumstance (employer/employee, rank, volunteer, etc.). For
myself, his posts are automatically deleted -- my time is precious to me.

~Ethan~

 

[Raymond Hettinger]

Maybe the problem is that although everyone welcomes contributions and
changes (or says that they do), the mechanisms remain largely beyond
criticism.

FWIW, I support the idea the regular docs incorporating links to
freely editable wiki pages. That will at least make it easier for
people to make changes or add notes.

That being said, I would like to add a few thoughts about the
current process. ISTM that important corrections (when the
docs are clearly in error) tend to get made right away. What
is more interesting are the doc requests that get rejected
and why:

* Many doc requests come from people just learning the language
(that makes sense because the learning process involves reading
the docs). Unfortunately, a fair number of those requests are
flat-out wrong or represent a profound misunderstanding of the
feature in question. That may be an indicator that the docs
need to be improved, but the specific suggestion can be inane.

* Some doc requests come from people who simply do not like the
feature in question. It is natural for tastes, styles, and
preferences to vary; however, we do have a firm rule that Guido's
tastes, styles, and preferences are the ones that go into the
language. So the doc writers need to try to channel Guido instead
of fighting him. So, if you think eval() is evil (I don't but many
do), we're not going to document that eval() should *never* be used.
If you hate super(), that's too bad -- the docs need to describe
what it does and how it was intended to be used -- the docs are no
place for diatribes on why inheritance is over-used and why you
think the world would be a better place without mixins or
multiple inheritance.

* Then, there is a matter of where to put a particular piece of
documentation (how many times do you repeat a concept that pervades
the language). Hashing is a good example. The docs can discuss how
some objects hash to their object id and that object ids can change
from run-to-run, but if someone gets tripped-up by the idea (hey,
my set reprs changed between runs, wtf!), they want the docs updated
in the specific place that tripped them up (i.e. you must put big
red warnings in the set documentation, and the dict documentation,
and everywhere else a hash gets used ...). The core problem is that
the docs are interrelated -- the one place you're looking for
documentation of a specific builtin or function can't contain
every concept in the language.

* Some behaviors are intentionally left unspecified. For the longest
time, Tim did not want to guarantee sort stability. This left him
free to continue to search for algorithmic improvements that did not
have stability. Later, the property was deemed so important that it
did become a guaranteed behavior. Also, some things are unspecified
to make things easier for other implementations (IronPython, PyPy,
Jython, etc.) We need to make sure that some one doesn't casually
go through the docs making doc promises that are hard to keep.

* Some things are just plain difficult to fully explain in English
and not misrepresent that actual behavior. For example, the str.split
()
docs have been continuously tweaked over the years. Even now, I think
there are corner cases that are not fully captured by the docs.
Casual
edits to str.split() docs are more likely than not to take them
farther
away from the truth.

* Then, there is the problem of talking too much. A book could be
written about super(), but that shouldn't all go into the docs for
the super builtin. Beginners often want to master all the builtins
and they try to read the doc page on builtin functions. It used to be
that you could read through the builtin descriptions in a few minutes.
Now, it takes a concerted effort to get through. It is hard to take
a sip of water from a firehose. Too much information has make a
function harder to understand.

* My biggest pet peeve are requests to fill the docs with big red
warnings. I do not want the docs to look like a mine field. The
warnings
should be reserved for a handful of security or data corruption risks.
For the most part, the docs should be matter-of-fact, explaining what
a function or method does and how it was intended to be used.

Preferred: "The value str.letters is locale dependent"

Not preferred: "Caution, the str.letters value can be adversely
affected by the locale setting (it could even change length!); use
this
only when you are certain the locale setting will not violate any of
your program invariants; consider using a string literal instead; I
hate
string.letters and think Guido was smoking crack when it was
introduced."

* Another category of rejected doc requests come from people looking
for
absolution from one of their programming bugs. It typically takes the
form of, "I made an assumption that the language did X, but it did Y
and my program didn't do what I wanted; therefore, the docs must be
to blame and they must change ...". The suggestion is "I was
suprised"
implies "the docs are hosed". The fact is that people with diffferent
backgrounds are going to have different expectations and someone is
going to get "surprised". The docs need to say what functions do, but
they don't need to be changed everytime someone writes a buggy
program.

In short, most doc requests that get rejected are requests that didn't
actually improve the documentation.

I do support links from the regular docs to an external wiki but the
main docs should continue to go through the regular process using the
tracker.


Raymond

 

[Raymond Hettinger]

[Xah Lee]

i've wrote several articles about this issue, total time spend on this
is probably more than 2 months full-time work. See:

• Python Documentation Problems
 http://xahlee.org/perl-python/python_doc_index.html

I just read you post. You did devote a substantial amount of time
to the project. Some of your criticisms are valid. Wish you had
posted patches, I think many of them would have been accepted.

Since you wrote this a few years ago, many examples have
been added to the docs and more are forthcoming.

I often receive thank you emails for 2 particular articles, which are
most frequently google searched as indicated by my weblog:

• Python Doc Problem Example: gzip
 http://xahlee.org/perl-python/python_doc_gzip.html

• Python Doc Problem Example: sort()
 http://xahlee.org/perl-python/python_doc_sort.html

• Sorting in Python and Perl
 http://xahlee.org/perl-python/sort_list.html

Some are the criticisms are valid; others seem off-base.

Here are a few thoughts on list.sort() for those who are interested:

* The key= and reversed= parameters are not intended for special
cases, leaving cmp= for the general case. They were intended to
be full replacements. In Python3.x, the cmp function is gone.

* The interaction of the key= and cmp= functions can be made to
interact (the key function is first applied to every element and
the cmp function then gets applied to the results of the key
function). This isn't a normal or intended use case, so the docs
don't delve into the subject.

* The reversed parameter does more than list.sort() followed by
list.reverse(). It also preserves stability in the event of equal
keys:

>>> sorted([(1,2), (1,3)], key=itemgetter(0), reverse=True)

[(1,2), (1,3)]

So it was not correct to say that the following are equivalent:

li.sort(lambda x, y: cmp(x[1],y[1]), reverse=True)
li.sort(lambda x, y: cmp(y[1],x[1]))

* We should link the sorted() and list.sort() docs to the
sorting how-to (with a fuller discussion on the art of sorting
including a discussion of operator.itemgetter() and
operator.attrgetter() which were designed to work with the key=
parameter.


Raymond

 

[r]

* Many doc requests come from people just learning the language
(that makes sense because the learning process involves reading
the docs).  Unfortunately, a fair number of those requests are
flat-out wrong or represent a profound misunderstanding of the
feature in question.  That may be an indicator that the docs
need to be improved...

Yes, if many people have problems with the docs then that must be a
*clue* to some underling problem in the way the docs are presented.
Whether its from misinfomation, misguidance, or just plain
misunderstanding on the reader's part that does not matter. There are
problems and we need the feedback from everybody noob-to-pro on how to
fix this conundrum. One thing that some naysayers may forget is the
fact that these noobs most likely have no idea *how*, *when*, or
*where* to voice a complaint so they just move on to the next language
or suffer with an incomplete understanding of the language and/or
proper python idioms. I would say the complaints that this list has
seen concerning docs only scratches the surface of the huge underling
issues that face us here!

So, if you think eval() is evil (I don't but many
do), we're not going to document that eval() should *never* be used.
If you hate super(), that's too bad -- the docs need to describe
what it does and how it was intended to be used -- the docs are no
place for diatribes on why inheritance is over-used and why you
think the world would be a better place without mixins or
multiple inheritance.

Eloquent and beautiful a paragraph that was Raymond. Why, because
common sense is just so damn beautiful. Keep the docs clean of
personal opinions and just give us the facts people. Who cares about
the history of OOP --Google it!-- i want to read about using Python.
Give me the nitty-gritty-down-and-dirty-facts that relate to Python
syntax and structure, and only the facts, in the most strait forward
and common sense way so i can get on to actually writing some code!

If you seek self gratification visit c.l.py and vent away, everyone
else seems to.

 

[Raymond Hettinger]

[Raymond Hettinger]

Here are a few thoughts on list.sort() for those who are interested:

After one more reading of Xah Lee's posts on the documentation for
sort,
here are couple more thoughts:

* The reason that list.sort() allows None for the cmp parameter is not
so that you can write list.sort(None). It was put there to make it
easier for people writing function definitions where the cmp function
is a possible argument:

def sort_and_chop(seq, maxlen, cmp=None):
s = seq[:]
s.sort(cmp) # needs to accept None as a possible
argument
return s[:maxlen]

* The reason for implementing the key= parameter had nothing to do
with limitations of Python's compiler. Instead, it was inspired by
the
decorate-sort-undecorate pattern.


Raymond

 

[Cousin Stanley]

FWIW, I support the idea the regular docs incorporating links
to freely editable wiki pages. That will at least make it
easier for people to make changes or add notes.

That being said, I would like to add a few thoughts about the
current process. ISTM that important corrections (when the
docs are clearly in error) tend to get made right away. What
is more interesting are the doc requests that get rejected
and why:
....
In short, most doc requests that get rejected are requests
that didn't actually improve the documentation.
....

Raymond ....

Thanks for your very eloquent and cohernt response
that in my opinion helps to explain many of the
problems, either real or perceived, with the Python
documentation ....

 

[rurpy]

That's no different from *any* major refactoring. The exact same problem
exists for code as well as documentation. It's a solved problem for code,
and it's a solved problem for documentation.

Huh? I don't buy this at all. Code refactoring doesn't change
the semantics of the program at all -- it simplifies the code
that implements behavior without changing behavior. How does
this apply to revising documentation? It may be true that some
of the same techniques applicable to maintaining code are
applicable to documentation but I need a lot more than your
assertion to believe they are as equivalent as you seem to be
claiming.

In some order:

Consensus that there is a problem that needs solving;
Volunteer(s) to do the work;
Somebody to take responsibility to check the changes in.

That's not refactoring as I understand it -- that is making a
large change. I don't see that that addresses the problem of
getting a large patch approved or the other issues I mentioned.

Sometimes, in the face of apathy or disagreement (which may very well be
justified), you have to at least partly solve the problem *before* people
will agree that it needed to be solved. Welcome to the real world.


Yes it would. Most patches are ignored, because the dev team are
overworked, and if they don't see the need for a patch, they won't
approve it.

I'm confused. If they weren't overworked, then they would
approve patches they didn't see a need for? Or are you
saying because they are overworked they fail to approve patches
that should be approved? I am not sure how either supports
the argument that the tracker is the best method of improving
the docs.

No, submitting a tracker issue is a necessary but not sufficient step.
Without a tracker issue, you're very unlikely to have people agree to
replace the existing docs with your docs, although a PEP would probably
do it. (A PEP is significantly more effort than raising a tracker issue.)

Has there ever been a PEP for a doc change? Are you making
a serious suggestion?

You also have to get attention from those with check-in privileges. If
they approve your patches, you're done. If they don't approve them, or
fail to respond, you can try convincing them, or taking it to the Python-
Dev list and request somebody review your patches.

If you have no patches, perhaps because you're unwilling to invest the
effort without a guarantee that it will be treated seriously, then you
need to get some sort of consensus among the relevant people that the
problem is worth solving.

You are again describing the current process, not the issue
of whether the current process is the optimal one or not.

And while I would never expect a guarantee that a particular
submission would be accepted, I think I would need, if not a
guarantee, at least a high expectation that a submission would
be treated seriously before I invested a large amount of work
in it.

Guess what? Sometimes others will disagree with you. What you see as the
Worst. Docs. EVAR. may be seen as perfectly adequate, even excellent, by
others. If this is the case, remember, Python isn't your product, you
don't have veto over what goes into it. Feel free to publish your
alternatives. Write a book. Go home and cry into your beer over it.
Whatever. Sometimes you win, and sometimes you don't.

Right, this is obvious. But the other side of that coin
is that if people who see problems with the docs perceive
that those who control access to the docs have no interest in
changing them, then the first group will likely post their
complaints on this newsgroup. If you don't like that then you
have the same choices you listed above. Of course if the docs
really are that good, there will relatively few complaints and
the vast majority of readers will recognize the few complaints
as invalid and ignore them, so if you are right that the docs
are good enough, there should be no problem.

This is the same process whether dealing with code or documentation.


Perhaps that's a good indication that a Python wiki *doesn't* fulfill a
need in the community, and that despite what a few squeaky wheels think,
the docs *are* doing a good job?

That fails to explain the repeated criticisms of the docs here
and in blogs. Of course this argument is not likely to convince
you since there is no reliable quantitative support for the
claim that there are more complaints about the docs than other
aspects of Python -- it relies on each individual's powers of
observation. But you also have to explain away very specific
criticisms of the docs such as Xah Lee's as invalid or
exceptions.

But they aren't met with such a so-called "standard response".

I just looked through the first 70 or so messages in this thread
and in this case I have agree with you, most of the responses were
not what I called "standard responses". There was one guy, a
Steven D'Aprano early on that trotted out the, "it's free software,
fix it if you don't like it" line, and the troll epithet was used
a time or two, but the vast majority were in fact reasonable and
thoughtful responses. I will try to find some examples from
other past threads.

We know that there are problems. We've said repeatedly that corrections
and patches are welcome. We've repeatedly told you how to communicate
your answer to the question of what should be done. None of this is good
enough for you. I don't know what else you expect.

You have been told repeatedly why your insistence that the
tracker must be the only channel, is wrong. I don't understand
why you can't understand that.

(Generally only those in authority, bosses, parents, police,
and the like, "tell" others what a situation is and have a
right to demand that the subject accept it without question.
I think you could find a more respectful and less authoritarian
way of phrasing what you said above since you are not in
a position of authority over me.)

And who exactly is this "we" that you are a member of?

Ah yes, I'm sorry, I missed one other alternative: sit around and wait
for somebody else to put in the money and/or effort.


Of course not. But it does mean that you won't be taken seriously, and
you have no right to be taken seriously.

On usenet no one has a right to anything. You can take my
posts however you wish. I've tried to make reasonable arguments
to support my view (with the understanding that these are usenet
posts, not a doctoral thesis). But you (and the unnamed "we"
above) seem to feel that public complaints or discussion about
better alternatives to the current doc improvement process are
somehow out-of-bounds. I'm sorry, I disagree.

 

[Terry Reedy]

FWIW, I support the idea the regular docs incorporating links to
freely editable wiki pages. That will at least make it easier for
people to make changes or add notes.

That being said, I would like to add a few thoughts about the
current process.

The summary that follows accords with my experiences and observations.
Perhaps you could add it to the Wiki.

ISTM that important corrections (when the

docs are clearly in error) tend to get made right away.

Many within a day. I have seen this with both some of my suggestions and
others.

What

is more interesting are the doc requests that get rejected
and why:

One of my suggestions was postponed pending discussion on PyDev of the
underlying issue, which I have not initiated yet. Another Georg accepted
in principle but awaits me taking the time to write a fleshed out
replacement text.

* Many doc requests come from people just learning the language
(that makes sense because the learning process involves reading
the docs). Unfortunately, a fair number of those requests are
flat-out wrong or represent a profound misunderstanding of the
feature in question. That may be an indicator that the docs
need to be improved, but the specific suggestion can be inane.

* Some doc requests come from people who simply do not like the
feature in question. It is natural for tastes, styles, and
preferences to vary; however, we do have a firm rule that Guido's
tastes, styles, and preferences are the ones that go into the
language. So the doc writers need to try to channel Guido instead
of fighting him. So, if you think eval() is evil (I don't but many
do), we're not going to document that eval() should *never* be used.
If you hate super(), that's too bad -- the docs need to describe
what it does and how it was intended to be used -- the docs are no
place for diatribes on why inheritance is over-used and why you
think the world would be a better place without mixins or
multiple inheritance.

* Then, there is a matter of where to put a particular piece of
documentation (how many times do you repeat a concept that pervades
the language).

I doubt anyone can *really* understand the problem of organizing a
book's worth in technical information into coherent order until they
have actually tried to do so.

Hashing is a good example. The docs can discuss how
some objects hash to their object id and that object ids can change
from run-to-run, but if someone gets tripped-up by the idea (hey,
my set reprs changed between runs, wtf!), they want the docs updated
in the specific place that tripped them up (i.e. you must put big
red warnings in the set documentation, and the dict documentation,
and everywhere else a hash gets used ...). The core problem is that
the docs are interrelated -- the one place you're looking for
documentation of a specific builtin or function can't contain
every concept in the language.

* Some behaviors are intentionally left unspecified. For the longest
time, Tim did not want to guarantee sort stability. This left him
free to continue to search for algorithmic improvements that did not
have stability. Later, the property was deemed so important that it
did become a guaranteed behavior. Also, some things are unspecified
to make things easier for other implementations (IronPython, PyPy,
Jython, etc.) We need to make sure that some one doesn't casually
go through the docs making doc promises that are hard to keep.

* Some things are just plain difficult to fully explain in English
and not misrepresent that actual behavior. For example, the str.split
()
docs have been continuously tweaked over the years. Even now, I think
there are corner cases that are not fully captured by the docs.
Casual
edits to str.split() docs are more likely than not to take them
farther
away from the truth.

* Then, there is the problem of talking too much. A book could be
written about super(), but that shouldn't all go into the docs for
the super builtin. Beginners often want to master all the builtins
and they try to read the doc page on builtin functions. It used to be
that you could read through the builtin descriptions in a few minutes.
Now, it takes a concerted effort to get through. It is hard to take
a sip of water from a firehose. Too much information has make a
function harder to understand.

* My biggest pet peeve are requests to fill the docs with big red
warnings. I do not want the docs to look like a mine field. The
warnings
should be reserved for a handful of security or data corruption risks.
For the most part, the docs should be matter-of-fact, explaining what
a function or method does and how it was intended to be used.

Preferred: "The value str.letters is locale dependent"

Not preferred: "Caution, the str.letters value can be adversely
affected by the locale setting (it could even change length!); use
this
only when you are certain the locale setting will not violate any of
your program invariants; consider using a string literal instead; I
hate
string.letters and think Guido was smoking crack when it was
introduced."

* Another category of rejected doc requests come from people looking
for
absolution from one of their programming bugs. It typically takes the
form of, "I made an assumption that the language did X, but it did Y
and my program didn't do what I wanted; therefore, the docs must be
to blame and they must change ...". The suggestion is "I was
suprised"
implies "the docs are hosed". The fact is that people with diffferent
backgrounds are going to have different expectations and someone is
going to get "surprised". The docs need to say what functions do, but
they don't need to be changed everytime someone writes a buggy
program.

In short, most doc requests that get rejected are requests that didn't
actually improve the documentation.

I do support links from the regular docs to an external wiki but the
main docs should continue to go through the regular process using the
tracker.

Agreed

Terry Jan Reedy

 

[Terry Reedy]

Right, but those good points are still worth taking on board. There
have been Xah Lee posts which have been relatively constructive,

The last time that he did do so that I read, I responded rationally like
I would with any other normal post. He responded with foul insults. End
of discussion.

> when the only responses are from people who see the name and can't be
bothered reading the message before issuing a stock "he's a troll"

False premise.

response, the tone is likely to remain vulgar from that point onwards.

He was vulgar even when I did ignore the name and did bother to read and
write (not 'issue') a specific response. He has also 'blown'
opportunities on the py-dev list.

tjr

 

[rurpy]

FWIW, I support the idea the regular docs incorporating links to
freely editable wiki pages. That will at least make it easier for
people to make changes or add notes.

That being said, I would like to add a few thoughts about the
current process. ISTM that important corrections (when the
docs are clearly in error) tend to get made right away. What
is more interesting are the doc requests that get rejected
and why:

[...snip interesting categorization of "bad" doc enhancement
suggestions...]

In short, most doc requests that get rejected are requests that didn't
actually improve the documentation.

I do support links from the regular docs to an external wiki but the
main docs should continue to go through the regular process using the
tracker.

What is the purpose of such a wiki?

1. To provide hopefully useful adjunct information for each
doc page?

2. To provide a source of input for improving the next version
of the docs?

In general I have a low opinion of wikis. Nearly all the ones
I've seen are trash heaps of out-dated and wrong information,
atrocious writing, and comments that look like the middle of
some arcane usenet thread. They lack organization making it
very difficult to find the needles of good information in the
haystack of junk. The only one that seems to work is Wikipedia
and it "works" because it has a very heavy-weight process
controlling contributions. (And even then, my biggest complaint
with Wikipedia is the poor writing quality in a lot of articles.)

I'm not trying to be negative, only pointing out that to serve
either of the goals I asked about above, a wiki will probably
require a considerable management effort that should be faced
up front. To simply set up a wiki and "wait for it to
self-organize" (as I here read once) is a pipe-dream.

How will it be managed? Anything written stays? If not what
standards are used to remove junk entries? Wrong entries?
Questionable entries? Who will do this? Will they act based
on their own judgment or some documented formal standards?
What happens to comments when a new doc version is released?
What happens when entries become obsolete? What happens when
comments are disputed?

If the goal is (2) what kind of wiki contributions are being
solicited ("I don't understand what this sentence means"? a
complete rewrite of a section?) and how is that communicated?
Who will take a long string of wiki comments constituting a
discussion and condense them into something that can be
submitted as a doc patch? (This I suspect is where the
work is, where someone with good writing skills is needed.)
Will this process be sufficient to raise the quality of the
docs significantly, or are other steps also needed such as
an attitude shift in what the core committers consider
acceptable?

If the goal is (1) above, it could potentially have a
deleterious effect on the docs in that it will be easier to
respond to requests to expand some description with, "there's
no need since that is covered in (or belongs in) the wiki
comments".

All the above not withstanding, I too think a wiki is worth
trying. But without doing a lot more than just "setting up
a wiki", I sadly believe even a python.org supported wiki
is doomed to failure.