rotor replacement

[phr]

There isn't. The interface might be beautifully designed, and you might
claim it is, and I would *still* require that the module gets field
testing before being incorporated into Python.

Yes, of course, that's completely reasonable.

If other people start attesting that the module is beatifully
designed, and should be included in the Python core - *then* it is
worth looking into inclusion.

You said something more than that. You said it's not worth even
thinking about inclusion until the finished module is actually in
distribution outside the core. That it's not appropriate for the
distro maintainers to look at the spec and the reference (pure Python)
implementatation and say "yes, we want this, go write the C version
and we'll include it after it's had some testing". I think that's
silly. (Anyway I don't think "beautifully designed" needs to be a
requirement, and I don't claim to have done a beautiful design. "Gets
the job done straightforwardly" is all that's needed and is all that
I've aimed for).

When the inestimable Alexbot went to O'Reilly to pitch "Python in a
Nutshell", I'm sure they didn't tell him "go write the whole book from
cover to cover, circulate it for at least a year and get it thoroughly
reviewed, and if enough people recommend it, then and only then will
we begin to think about distributing it ourselves". More normally,
he'd have given them a proposal with an outline and maybe a sample
chapter and they used that to decide to give him a go-ahead. Usually
at that point, they'd even also pay him some money in advance to do
the rest of the work with, and promise to pay him a cancellation fee
if they changed their mind about the book.

I certainly never asked for anything like that from the Python guys
about the crypto module. I did ask for, and receive, a "this looks
good, we want it". I later heard that they backed off because of
legal concerns, so I put the task aside.

Similarly, look at how the whole PEP process works. There are lots of
times when a PEP has been accepted before any working code is
distributed.

To take your view to an extreme, no project should even have a task
list of desired features that people are invited to implement.
There's been numerous times when I've written something for the GNU
project because it's been on a list like that, and it's been accepted
after checking that it does what it's supposed to.

Not if I have a say in it. *Any* new module should see out-of-the-core
distribution first (unless there is BDFL pronouncement to include it,
of course).

This really is a matter of development process, not of technical
quality.

That's bizarre and abnormal as a development process. What kind of
development process in industry doesn't decide whether to include a
feature, until after the feature is completely implemented at a
production scale? They figure out what the features should be, and
write them into a specification. The specification defines what
features will be included. Then they implement the features. (I'm
talking specificially about straightforward, well-defined features,
not blue-sky or experimental concepts).

You see, they decide in advance what features they're going to
include, because development consumes resources and they don't want to
burn resources to develop something unless they're pretty sure they're
going to use the result once it's done. (Of course it doesn't always
work out that way. Everyone has worked on projects that have gotten
cancelled before delivery. However that by definition means that
something has gone wrong. No engineer likes it when that happens.)

You seem to have the attitude that since volunteer development effort
doesn't consume actual PSF funds, the volunteer effort is worth
nothing and can be expended arbitrarily. The volunteers may not feel
that way.

I personally would not say that, although I can imagine that some people
do say that, and I would also defend an inclusion, and push compliance
the BXA requirements so we can legally export Python out of the U.S.A.

The BXA notification on the US side is pretty easy to handle, as you
know. If you look at those Python-dev messages, the concern was about
importing Python-with-crypto into countries like Holland, not about
exporting it from the USA. Those concerns are not necessarily
something to sneeze at. Salman Rushdie had to go into hiding for
years, after publishing a book in England whose contents were illegal
in Iran. So, I can't get too upset with the Python maintainers for
deciding to stay away from such political messes, even though I don't
feel such strong concern myself.

In either case, the user would best use the pre-compiled binary that
somebody else provided for the platform.

Who's the "somebody else"? The module author might also not have had
a compiler for that platform. Are you suggesting that he maintain his
own distribution of multiple binaries, duplicating the machinery of
the Python distro for managing multi-platform builds? Do you do that
with your own modules, and still say that it's easy? If you don't,
then how does the end user know where to get the module for his
platform?

No, I'm merely repeating myself, and rephrasing each time.
I have to, because apparently you don't see what my requirement is.

Unless your requirement is different than what you say it is, I do see
what it is, and I'm saying it's better to do what normal projects do
and what Python has done in the past. That is, it's perfectly ok to
decide to do something and then do it, rather than insisting,
bizarrely, that it always be the other way around.

The original source code of the SHA-1 implementation is the NIST code
(Gutmann, then Hollerbach), so I guess that had hundreds of users before
the module was contributed to Python.

Not terribly relevant, the NIST code was not a Python module.

The module itself (including the API) was written by Greg Stein and
Andrew Kuchling. I believe (without being able to verify) that they
distributed this module for quite some time, before contributing it
to Python. We would have to ask them how many users they had until
they felt confident to contribute the code.

I think that question isn't the right one. We need to ask how many
users the sha module was required to have, before Greg and Andrew
could have reasonable confidence that the sha module would go into the
core once it was tested enough and shown to be reliable. I suspect
they were able to have that confidence long before testing was
complete, and maybe before implementation even started. That is, they
were likely able to decide that the core obviously needed a sha
module, before setting off to write and test one for that purpose.

If Andrew is reading this, maybe he can comment.

 

[John J. Lee]

The way I understand it, that 7-qubit computer was based on embedding
the qubits on atoms in a large molecule, then running the computation

Oh, you mean that particular kind, OK. Doesn't apply to QC in
general.

The problem is maintaining enough coherence through the whole
calculation that the results aren't turned into garbage. In any
physically realizeable experiment, a certain amount of decoherence
will creep in at every step. So you need to add additional qubits for
error correction, but then those qubits complicate the calculation and
add more decoherence, so you need even more error correcting qubits.

Yes, that's much more interesting, dunno what the current state of
play is.

[...]

I'm not any kind of expert in this stuff but have had some
conversations with people who are into it, and the above is what they
told me, as of a few years ago. I probably have it all somewhat garbled.

Me too


John

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

That it's not appropriate for the
distro maintainers to look at the spec and the reference (pure Python)
implementatation and say "yes, we want this, go write the C version
and we'll include it after it's had some testing".

I know that I'm not going to give a blanket promise to include some
code in the future. I can give blanket promises to *review* such
code. Actually, assuming I find the time, I will review *any*
code that is contributed.

When the inestimable Alexbot went to O'Reilly to pitch "Python in a
Nutshell", I'm sure they didn't tell him "go write the whole book from
cover to cover, circulate it for at least a year and get it thoroughly
reviewed, and if enough people recommend it, then and only then will
we begin to think about distributing it ourselves".

Sure. That is because O'Reilly is willing to take a financial risk
of failure of a book product. I'm not willing to take similar risks
for Python code (risk of having to redesign the interface, fix bugs
in the implementation, or have the submitter run away after the
contribution).

Similarly, look at how the whole PEP process works. There are lots of
times when a PEP has been accepted before any working code is
distributed.

Indeed. For new language features, it is more difficult to try them out
in advance than for new library API. As a result, flaws of the new
feature are often only fully understood years after the new feature
first gets released. For an example, consider the buffer interface.

To take your view to an extreme, no project should even have a task
list of desired features that people are invited to implement.

Taking it to the extreme misses the point. I'm asking for field testing
for new modules - not for all changes.

That's bizarre and abnormal as a development process. What kind of
development process in industry doesn't decide whether to include a
feature, until after the feature is completely implemented at a
production scale?

Any high-quality standardization process. Standards in IETF and OMG
are only accepted after implementations have been available.

You seem to have the attitude that since volunteer development effort
doesn't consume actual PSF funds, the volunteer effort is worth
nothing and can be expended arbitrarily. The volunteers may not feel
that way.

The volunteers are free to work on whatever they please. If you chose
not to write an AES module - that's fine with me. Still, people do
contribute to Python, and they do so without asking for permission
first. Typically, they develop the code because it solves their own
needs - then it doesn't really matter whether it also solves the
needs of others.

Who's the "somebody else"?

Some user. I find that users contribute binaries for all kinds of
platforms for the code I publish. This is how open source works.

Do you do that
with your own modules, and still say that it's easy?

I publish or link to binaries of my code that others have created,
and find no problems in doing so.

Unless your requirement is different than what you say it is, I do see
what it is, and I'm saying it's better to do what normal projects do
and what Python has done in the past. That is, it's perfectly ok to
decide to do something and then do it, rather than insisting,
bizarrely, that it always be the other way around.

No, it's three steps
1. decide that you want to do it
2. do it
3. decide whether you are pleased with the result, and only
use it if you are

IOW, there should not be a blanket guarantee to use it after
step 1.

I think that question isn't the right one. We need to ask how many
users the sha module was required to have, before Greg and Andrew
could have reasonable confidence that the sha module would go into the
core once it was tested enough and shown to be reliable.

They did not have *any* guarantee until they asked. I guess when they
asked it was accepted immediately.

I suspect
they were able to have that confidence long before testing was
complete, and maybe before implementation even started.

I'm pretty certain that you are wrong with that assumption.
Again, we would have to ask - but I would not be surprised if
AMK started implementing the module without even *considering*
a later inclusion in the Python core at that time. He has done
so on many occasions (include PyXML, which I inherited from
him).

Regards,
Martin

 

[Jeremy Bowers]

That is sometimes true, but not always. Sometimes the best way is
obvious.

With all due respect, you aren't arguing very well. Your posts are, like
this quote, more reactionary than thought-out.

You need to stop arguing for the sake of argument, read more carefully,
and think more about your global goals in this conversation than your
local ones. I'm 99% certain that you've already gotten all the answers
you're going to get and all the answers you need in this thread, but
you're so busy arguing the local points like this, you haven't noticed.

(Either that, or it's the "you haven't answered me because you haven't
given the exact answer I want" bit, which I find quite tiresome; there's
another list where there are these three teenagers that don't even realize
they've defined "answer" that way and keep bitching that they aren't being
"answered" when the "answer" is, basically, "it's nice that you feel that
way." But I digress.)

I don't need your permission to do that. What I'm looking for is an
indication that it's worth my while, before I spend the effort.

The policy has been laid out, multiple times, by multiple people now. The
answer is, you are not going to get any such indication that will satisfy
you. Note that I am not a Python contributor of any kind. Also note that I
figured this out about two days ago. You can wheedle these guys all you
want, but they are too experienced for you to extract a promise from them.

 

[phr]

I know that I'm not going to give a blanket promise to include some
code in the future. I can give blanket promises to *review* such
code. Actually, assuming I find the time, I will review *any*
code that is contributed.

I don't see why you can't make up your mind enough to issue simple
statements like "the Python lib should have a module that does
so-and-so, and it should meet such-and-such requirements, so if
someone submits one that meets the requirements and passes code review
and testing and doesn't have unexpected issues or otherwise fail to
meet reasonable expectations, we'll use it".

Sure. That is because O'Reilly is willing to take a financial risk
of failure of a book product. I'm not willing to take similar risks
for Python code (risk of having to redesign the interface,

Again, we're talking about straightforward modules whose basic
interface needs are obvious. And interfaces in stdlib do get extended
from version to version all the time, if users turn out to need
additional features beyond the obvious basics.

fix bugs in the implementation,

Obviously there must be testing and review before inclusion.
Acceptance is contingent on the module passing tests and review.

or have the submitter run away after the contribution).

There is no way to know in advance whether that's going to happen. A
lot of work on the ANSI X9 crypto standards came to a screeching halt
a few years ago when one of the more prolific contributors tripped
over his vacuum cleaner cord, fell down the stairs, and was killed.
So if you have to be absolutely sure that the submitter will always be
around, you can never accept anything. I think you mostly have to go
by how maintainable the code looks and how much maintainance you think
it will actually need and how many people you think are around who can
take care of it when needed. And I do believe that experienced
programmers are capable of making reasonable judgements about those
questions, so they should not refuse to ever make such judgements.

Indeed. For new language features, it is more difficult to try them out
in advance than for new library API.

I don't see why that should be. Nothing stops anyone from
implementing and testing a language feature before standardizing it.
It will get even easier with PyPy, which is perhaps a reason to ban
further language changes until PyPy is deployed.

Taking it to the extreme misses the point. I'm asking for field
testing for new modules - not for all changes.

"Field testing" to most people means testing that a specific
implementation is reliable enough for inclusion. It is the final step
in a normal process of declaring a feature ready for deployment, not
the initial step:

1) think about whether you want the feature
2) decide you want it
3) implement
4) field test (this naturally recognizes the possibility of
reversing step 2, if something unexpectedly goes wrong in testing
that's not easily repaired, but step 2 declares the basic
intention for what should happen after a successful test).
5) deploy

You wanted much more than for step 4 to always happen before step 5,
which is reasonable. You claimed step 4 should always happen before
step 1, which is silly.

Any high-quality standardization process. Standards in IETF and OMG
are only accepted after implementations have been available.

I don't know what OMG is, but there is no IETF requirement that any
implementations be available in any particular language. There are
also plenty of instances where the IETF decides that it wants
something to be standardized (e.g. IPSEC) so it invites a working
group to develop a specification. The WG members then spend a lot of
time in meetings and discussions reaching a consensus on what the spec
should say. They are willing to spend that time because the IETF has
already given them reasonable expectation that their end result will
actually be used. The IETF doesn't say "go develop a complete
standard and implementation and put it in the field for a year before
we [the IETF] will even think about whether we want to standardize
it". They are capable of announcing in advance that they want to
standardize something. I don't see why the Python folks have to be
incapable of ever doing the same.

For the module we're discussing, there is already a published working
implementation, written in Python. It implements every feature of the
API and has been integrated into various applications/demos and tested
with them. And, there was reasonable consensus on clpy and on the
python-crypto list that the API did the right things. That is all
that the IETF normally requires ("rough consensus and working code").
The Python implementation is unsuitable for the stdlib because it's
too slow to use in serious production apps and so it needs to be
rewritten in C, not because anything is missing or untested about it.
(There's also an unpublished hybrid Python/C implementation that's
tolerably fast, but unsuitable for contribution because it depends on
external packages that can't go in the Python distro). But the IETF
would not care about that. All they care is that useable
implementations exist.

Finally, the IETF generally only specifies protocols and doesn't care
about either the implementation specifics or the API (they require
implementations to exist only in order to prove that the protocol is
usable). The protocols here would be the FIPS operating modes (ECB,
CBC, CFB, etc). Those are already standardized as FIPS and have test
vectors published by NIST. The Python implementation computes all the
published test vectors correctly and interoperates with various
non-Python programs that use those same modes. So again, there is
already enough available to satisfy an IETF-like standardization
process. Your IETF example does not support your stance.

The volunteers are free to work on whatever they please. If you chose
not to write an AES module - that's fine with me.

However, the result of my not writing an AES module is that Python
doesn't have an AES module. Because as far as I can tell, while
numerous people want to use such a module, nobody except me has
expressed willingness to write and contribute one. If someone else
did it, I would be overjoyed, and I'd happily use whatever they wrote,
if it was useable at all, dropping any plans to write my own. (And
frankly, the only feature such a module absolutely needs to provide is
simple ECB mode block encryption and decryption. The other modes are
helpful for good performance and convenience but can be done as pure
Python functions that call the ECB mode primitive, with tolerable
speed for lots of useful apps. It's just the block cipher primitive
that's painfully slow in Python).

Still, people do contribute to Python, and they do so without asking
for permission first.

They have not done that for an AES module. That is a fact on the
ground. I don't have any burning desire to be the author of a Python
AES module. I just think Python should have one that I and other app
writers can rely on, so I've been willing to volunteer to write and
contribute it, since I'm qualified and nobody else has stepped
forward. But there appears to currently be a policy in force saying
"there will be no crypto module in the Python stdlib regardless of its
technical quality". Under that policy, there's no point in my writing
one to contribute it.

Typically, they develop the code because it solves their own needs -
then it doesn't really matter whether it also solves the needs of others.

In fact that's been done with AES and DES numerous times, and the
resulting modules do meet my needs for my own end-use (I'm currently
using a C library wrapped with SWIG). What's missing is a module in
the stdlib that lets me ship pure-Python crypto apps that other end
users can run without installing C code. And, as we discussed
already, the modules people have written for their own needs (at least
so far) aren't really technically suitable for the stdlib, because a
good general purpose stdlib module has somewhat different
characteristics than a module written to support a specific
application, which is what people write to solve their own needs.

Note that the same thing is true for most other types of library
modules besides crypto (i.e. a general purpose module suitable for the
stdlib tends to be different from what someone writes to solve their
own needs). So either most stdlib modules were either not written
purely to solve the author's needs and then submitted as an
afterthought as you describe, or else the modules in the stdlib
typically don't have good general-purpose designs. (In fact the
latter is often true, which suggests that the practice of contributing
modules originally written only to solve an individual need often
results in lousy stdlib modules, and a policy change towards
encouraging designing intentionally for the stdlib is probably a good
thing. To paraphrase RMS, a good stdlib needs to come from a vision
and a plan, not just from scratching a bunch of different itches).

Some user. I find that users contribute binaries for all kinds of
platforms for the code I publish. This is how open source works.


I publish or link to binaries of my code that others have created,
and find no problems in doing so.

And you seriously believe that the result is as painless for end users
as having a module in the stdlib that apps can just import and call
without the end-user having to first locate, download and install a
3rd party module? Do you think the Python philosophy of "batteries
included" really is a) meaningless, b) worthwhile in the past but now
obsolete, c) a mistake to begin with, d) sort of worthwhile but not
really important, e) something else? That's not a rhetorical
question, I'm really having a hard time figuring out what you think
"batteries included" is supposed to mean. I personally consider it
meaningful and extremely important, or else I'd be using Scheme
instead of Python.

No, it's three steps
1. decide that you want to do it
2. do it
3. decide whether you are pleased with the result, and only
use it if you are

IOW, there should not be a blanket guarantee to use it after step 1.

But, it's completely normal to say before step 1 that "if the result
of step 2 does so-and-so, then I'll be pleased in step 3", stating
some clear requirements and sticking to them. People do that in real
life all the time, including in standardization processes, and often
with ironclad guarantees (called "contracts").

It's pathetic if the Python maintainers are so indecisive as to be
incapable to ever saying (even -without- making ironclad guarantees),
that "yeah, we really ought to have an XYZ module and we'll use one if
someone submits it and it does so-and-so", giving an informal
assurance to someone thinking of doing an implementation that they're
probably not wasting their time. That's often what allows step 1 to
happen.

They did not have *any* guarantee until they asked. I guess when they
asked it was accepted immediately.

And who do you think they asked, hmm? When I asked Guido about
submitting a crypto module, he told me that he defers all technical
crypto issues to Andrew. So I think Andrew had a reasonable
expectation of what would happen when he submitted it. You and
Frederik seem to think there's something inappropriate or
self-inflated about wanting that expectation before committing to do a
pile of work that's primarily for other people's benefit. I think
your stated attitude is completely bizarre, that you can't really
believe anything so silly, so you're really just acting bureaucratic,
looking for excuses to say no instead of yes to worthwhile proposals.

Again, we would have to ask - but I would not be surprised if
AMK started implementing the [sha] module without even *considering*
a later inclusion in the Python core at that time. He has done
so on many occasions (include PyXML, which I inherited from him).

I would be surprised. That need for an sha module was completely
obvious, what it needed to do was completely obvious, and the
requirements and implementation have nothing like the subtlety of PyXML.

 

[Skip Montanaro]

phr> I don't see why you can't make up your mind enough to issue simple
phr> statements like "the Python lib should have a module that does
phr> so-and-so, and it should meet such-and-such requirements, so if
phr> someone submits one that meets the requirements and passes code
phr> review and testing and doesn't have unexpected issues or otherwise
phr> fail to meet reasonable expectations, we'll use it".

Because good requirements specification is difficult and testing improves
the breed. Better to have the major API changes and bugs taken care of, and
to have its popularity demonstrated *before* it gets into the Python
distribution. The best way to do something isn't always obvious at the
outset. If multiple solutions exist in the community, everyone benefits.
The best survive. The worst don't. If one implementation is prematurely
chosen for inclusion in the Python distribution, it stifles the vetting
process. Finally, what if, saints be preserved, your whizbang new module is
included in the core distribution and it's just not as popular as was first
thought? It just winds up as a weight around the maintainers' necks. Once
included in the distribution it's difficult to remove. Even rexec and
Bastion, which are both known to be inadequate from a security standpoint,
are still in the distribution.

So, feel free to implement whatever it is you propose. Register it with
PyPI, announce it on comp.lang.python.announce, etc. Support it for awhile.
Run it through a couple revisions to fix API warts and bugs. When it's the
category king and there is substantial community support for inclusion, it
will be considered.

Python is popular in part because of its fairly conservative development
strategy. That goes for the libraries as well as the language itself.

Skip

 

[phr]

Because good requirements specification is difficult and testing improves
the breed. Better to have the major API changes and bugs taken care of, and
to have its popularity demonstrated *before* it gets into the Python
distribution. The best way to do something isn't always obvious at the
outset.

That is sometimes true, but not always. Sometimes the best way is
obvious.

If multiple solutions exist in the community, everyone benefits.
The best survive. The worst don't.

What's needed here "good enough", not necessarily "the best
conceivable", a level of perfection which is perhaps unattainable.

If one implementation is prematurely chosen for inclusion in the
Python distribution, it stifles the vetting process.

There have been exactly zero alternative implementations proposed.

Finally, what if, saints be preserved, your whizbang new module is

It is not a whizbang module. It is a stripped-down, basic
implementation of a well-accepted set of standards that are being used
in thousands of other applications in other languages.

included in the core distribution and it's just not as popular as
was first thought?

There is demand for it. Look at how this thread started: some crypto
user noticed that rotor was gone and wanted to know what to do instead.

The issue of whether there's enough desire for a crypto module to
warrant including one in the stdlib was decided a long time ago. The
proof of that somebody decided to accept the rotor module into the
distro.

That decision, that enough users want crypto to make having a stdlib
module worthwhile, has never been reversed as far as I know. Rather,
the specific module (rotor) was withdrawn because it turned out not to
do the job on technical grounds. And the decision not to replace it
was based on new legal and political concerns that outweighed the
technical side, not on thinking that there wouldn't be enough users.

It just winds up as a weight around the maintainers' necks. Once
included in the distribution it's difficult to remove.

The rotor module is gone (and good riddance). That's how this thread
started, remember? It shows that bogus modules can be removed.

Have you ever used a crypto library in a serious way? Which ones have
you used? The python-crypto folks (including myself) have used quite
a few of them, and after a while one realizes that the usable ones all
do pretty much the same stuff. The one we're talking about does about
the same stuff that the rest of them do. It does the same stuff
listed in PEP 272, although it's organized a little bit differently in
order to make supplying multiple cipher primitives more convenient,
which the application that drove PEP 272 apparently didn't need to do.
Andrew (the author of PEP 272) has said that the different
organization is fine. The different organization is similar to what's
done internally in OpenSSL, in Peter Gutmann's cryptlib, in GnuPG, and
various other successful programs. It's not radical or experimental.
It's old hat and low risk. It provides functionality that's missing
from the stdlib.

Even rexec and Bastion, which are both known to be inadequate from a
security standpoint, are still in the distribution.

They should be removed pronto, just as rotor was removed. Some other
ones should be removed too, like SmartCookie.

So, feel free to implement whatever it is you propose.

I don't need your permission to do that. What I'm looking for is an
indication that it's worth my while, before I spend the effort.

Register it with PyPI, announce it on comp.lang.python.announce, etc.

I don't know how to do that, but the folks who care about it are
mostly on the python-crypto mailing list, and the module has been
discussed at length there and has been well-received.

Support it for awhile. Run it through a couple revisions to fix API
warts and bugs.

I already did that with the Python implementation, and did in fact
revise the API somewhat based on reviewer feedback. The C
implementation would have the exact same API that the reviewers seem
to be happy with.

When it's the category king and there is substantial community
support for inclusion,

It's already the category king, because there are precisely zero other
entrants in the category. If someone else wants to do one and can get
it into the stdlib, I will gladly use theirs and stop worrying about
mine.

it will be considered.

Not according to how I currently interpret Guido's posts to python-dev
about it. I read those as saying that no crypto module will be
considered for inclusion whether or not it's the category king,
because any such module might conflict with crypto regulations in some
countries. No matter how much community support or technical
perfection it has, it's just won't be included, period. So tell me
again what to do after writing and releasing a C module. There's just
no reason to write one, if the module can't go into the stdlib. The
stdlib has some special requirements but there are existing crypto
modules that I'm using and that work just fine outside of the stdlib,
even though they don't meet the stdlib's special needs. The only
reason to write a special module just to meet the stdlib's special
needs is so that the stdlib can use it.

Python is popular in part because of its fairly conservative development
strategy. That goes for the libraries as well as the language itself.

Tell me again how rotor got into the distribution.

 

[phr]

The policy has been laid out, multiple times, by multiple people now. The
answer is, you are not going to get any such indication that will satisfy
you.

Actually I already got an indication that satisfied me, from Guido and
Andrew, although it was later withdrawn for nontechnical reasons (i.e.
legal restrictions) that I don't feel entitled to argue against very
much. I do, however, believe that what Frederik and Martin are saying
is bogus on technical grounds, and furthermore, they're just being
busybodies, since they've had no involvement at all in what's happened
already with that module, and their picture of the Python development
process doesn't have much resemblance to how the current Python distro
got to be the way it is.

Note that I am not a Python contributor of any kind. Also note that I
figured this out about two days ago. You can wheedle these guys all you
want,

I have not asked them for anything, since they are not in a position
to give it.

but they are too experienced for you to extract a promise from them.

Actually, one of them already gave a promise that he couldn't keep
even if he wanted to. He said that if I wrote a crypto module and got
it tested enough, it would be considered for inclusion (i.e. based on
its technical merits). But, he doesn't get to decide that. Current
policy per per Guido seems to be that because of the legal stuff,
there will be no crypto module in the stdlib regardless of its merits.

 

[Skip Montanaro]

Finally, what if, saints be preserved, your whizbang new module is

phr> It is not a whizbang module. It is a stripped-down, basic
phr> implementation of a well-accepted set of standards that are being
phr> used in thousands of other applications in other languages.

Then there should be a library already out there already. All you should
need to do is wrap it (manually, with SWIG, whatever).

phr> There is demand for it. Look at how this thread started: some
phr> crypto user noticed that rotor was gone and wanted to know what to
phr> do instead.

Yes, and putting rotor back would be the wrong thing to do.

phr> The issue of whether there's enough desire for a crypto module to
phr> warrant including one in the stdlib was decided a long time ago.
phr> The proof of that somebody decided to accept the rotor module into
phr> the distro.

No, rotor was added in Python's early days (July 1992). Times have changed.

As long as we are discussing cryptography, what's wrong with m2crypto?

http://sandbox.rulemaker.net/ngps/m2/

Why not incorporate it into the standard distribution?

Or, what about Andrew Kuchling's crypto toolkit?

http://www.amk.ca/python/code/crypto.html

I believe both have been around awhile. If crypto-in-the-core is really
what's needed why not see if one of them is ready to go?

phr> The rotor module is gone (and good riddance). That's how this
phr> thread started, remember? It shows that bogus modules can be
phr> removed.

Yeah, but it was there for over 10 years.

phr> Have you ever used a crypto library in a serious way?

Nope, never directly. Don't make this about me. I'm interested in the
Python development process and how you'd like to turn it on its head.

phr> It's already the category king, because there are precisely zero
phr> other entrants in the category.

See my above references. Note, I don't use crypto at all, yet I was aware
of both of these (no Googling required). My guess would be they are
substantially more mature than your proposed module.

phr> I read those as saying that no crypto module will be considered for
phr> inclusion whether or not it's the category king, because any such
phr> module might conflict with crypto regulations in some countries.

That may be a problem, sure. I'm not sure how the discussion here changes
that. That's just life as we know it.

phr> So tell me again what to do after writing and releasing a C module.
phr> There's just no reason to write one, if the module can't go into
phr> the stdlib.

Why in the heck is inclusion in the standard library a requirement for you
to write this thing? If it's useful to you, write it and get on with your
life.

phr> Tell me again how rotor got into the distribution.

Okay. It was 1992. Bill Clinton had recently been elected president. It
was pretty much pre-WWW as we know it. Definitely pre-comp.lang.python and
pre-Google (heck, pre-Yahoo, pre-Win98 and pre-Mac OSX as well). Pre-string
methods. Pre-triple-quoted strings. Pre-spammers. Pre-DSL. Pre-lots of
stuff. There was an Emacs python-mode and <wink>s, both thanks to Tim.
People opined about Python's performance, just as they do today. Python's
version number was around 0.9.4, definitel < 1.0. Guido was the only person
with direct repository access. Including something in the distribution was
probably the only convenient way to make new modules available. If nothing
else, the rotor module (along with anything else included in the
distribution back then) may have been a good exercise in and demonstration
of writing extension modules, so it probably served a useful non-crypto
purpose. Python's user community was probably a few hundred people. Guido
likely had no thoughts of world domination with the little language that
could. Times have changed. You seem think there was a PEP process and
distutils and Wikis. I suspect some of the algorithms one might include in
a robust crypto toolkit today weren't even invented in 1992.

So throw away the rotor crutch and put your money where your mouth is.

Skip

 

[Lucas Raab]

As long as we are discussing cryptography, what's wrong with m2crypto?

http://sandbox.rulemaker.net/ngps/m2/

Why not incorporate it into the standard distribution?

Or, what about Andrew Kuchling's crypto toolkit?

http://www.amk.ca/python/code/crypto.html

<snip>

Umm, is it just me or did we just discuss the legal issues of that??

 

[Skip Montanaro]

As long as we are discussing cryptography, what's wrong with m2crypto?
Lucas> Umm, is it just me or did we just discuss the legal issues of
Lucas> that??

You may have. Whether or not there are legal issues with them is of little
relevance to the point I was making. Anything Paul writes would probably
have the same legal entanglements. I was simply pointing out that maybe,
just maybe, there are already suitable candidates from a technical
standpoint and that he doesn't need to write anything.

Skip

 

[phr]

phr> It is not a whizbang module. It is a stripped-down, basic
phr> implementation of a well-accepted set of standards that are being
phr> used in thousands of other applications in other languages.

Then there should be a library already out there already. All you
should need to do is wrap it (manually, with SWIG, whatever).

I'm currently using something wrapped with SWIG, but my understanding
is that core modules are not supposed to depend on SWIG. So a core
module will likely use some existing primitives wrapped by hand. That
is what I've offered to do, along with providing a somewhat more
Pythonic interface (though still a straightforward one) than directly
wrapping a C library intended for use from C applications.

phr> There is demand for it. Look at how this thread started: some
phr> crypto user noticed that rotor was gone and wanted to know what to
phr> do instead.

Yes, and putting rotor back would be the wrong thing to do.

Correct. The right thing is to replace rotor with something
reasonable that follows standards.

phr> The issue of whether there's enough desire for a crypto module to
phr> warrant including one in the stdlib was decided a long time ago.
phr> The proof of that somebody decided to accept the rotor module into
phr> the distro.

No, rotor was added in Python's early days (July 1992). Times have changed.

I don't see that. There's surely even more demand for crypto now than
there was in 1992.

As long as we are discussing cryptography, what's wrong with m2crypto?

http://sandbox.rulemaker.net/ngps/m2/

It's a good package but it's pretty heavyweight. It depends on both
SWIG and OpenSSL. I think it's still under development--there's an
occasional flurry of messages about it on python-crypto, but I haven't
been following it closely. I'd have a hard time making a case for
accepting it into the core given the difficulty I'm having making the
case for something as simple as a block cipher wrapper.
m2crypto+OpenSSL is at least 100 times as much code as the module I've
proposed. I think the Python lib should someday have its own
pure-Python SSL/TLS implementation sort of like the one Java has. But
if m2crypto went into the lib, I'd use it.

Why not incorporate it into the standard distribution?

I don't have the authority to incorporate anything into the standard
distribution. All I can do is offer to contribute stuff that I write,
and let the distro maintainers decide whether to incorporate it.

I don't have the authority to offer m2crypto, since I'm not the author.
Only the authors can do that. They haven't done so, as far as I know.

Or, what about Andrew Kuchling's crypto toolkit?

http://www.amk.ca/python/code/crypto.html

This is perhaps more suitable than m2crypto but as far as I know,
Andrew hasn't offered to contribute it. Whatever his reasons are, I
have to respect them. I think it has more stuff than a core crypto
module really needs (e.g. numerous semi-obsolete algorithms that
aren't in widespread use so aren't needed for interoperability) but
the extra stuff doesn't really get in the way. If it were in the
core, it would completely fulfill my desires and I would be
transported with ecstacy. But I've never seen any indication that
it's headed there.

I believe both have been around awhile. If crypto-in-the-core is
really what's needed why not see if one of them is ready to go?

I don't think m2crypto is the answer, though maybe I'm wrong. And if
Andrew's package is the answer, he would have submitted it already.

phr> Have you ever used a crypto library in a serious way?

Nope, never directly. Don't make this about me. I'm interested in the
Python development process and how you'd like to turn it on its head.

Either you're the one turning the actual process on its head, or else
it's already on its head and needs to be turned rightside up. Either
way, the existing process has certainly been a total failure so far at
producing good crypto support in the lib.

phr> It's already the category king, because there are precisely zero
phr> other entrants in the category.

See my above references. Note, I don't use crypto at all, yet I was aware
of both of these (no Googling required).

The authors have not offered to contribute them, so they're not in the
category. The category consists of crypto modules that have actually
been offered. As I keep saying, I'd love it if someone else offered
one. I'm not eager for this headache. I just saw that somebody
really ought to do it, and nobody was doing it, so I decided I was
elected.

My guess would be they are substantially more mature than your
proposed module.

This may sound a little strange, since my module is not yet written,
but if we take maturity to mean lower future maintenance needs, then I
wouldn't say m2crypto is precisely more mature. m2crypto uses SWIG
and is intimately connected to an immensely complex package (OpenSSL)
anpd has to track new OpenSSL and SWIG releases. OpenSSL has all
kinds of OS dependencies, has high performance math subsystems written
in assembly code for various different cpu's, etc. So my guess is
that m2crypto has a longer development curve and will always need
significant amounts of maintenance. My module is more like a basic
portable math library that computes square roots, cosines, etc. and is
OS and CPU independent. Once written and tested, it's intended to be
pretty solid and not need much attention.

Andrew's package is along the same lines as mine, except Andrew's is
already done, so Andrew's is definitely more mature. I like to think
that my module's API incorporates some lessons from Andrew's and
improves on it, but that's not that important in practice.

phr> I read those as saying that no crypto module will be
phr> considered for inclusion whether or not it's the category
phr> king, because any such module might conflict with crypto
phr> regulations in some countries.

That may be a problem, sure. I'm not sure how the discussion here changes
that. That's just life as we know it.

It doesn't change it. However, it does change my level of interest in
writing a module, as is reasonable since the only important
characteristics of my module that are different from existing ones is
that mine is designed specifically to be suitable for the core and has
been offered for distribution in the core. Outside the core, the
existing modules work ok so I use them.

Why in the heck is inclusion in the standard library a requirement
for you to write this thing?

There's no other reason to write it. Outside the stdlib, there are
other modules that are already useable, like the ones you mentioned,
so I use those.

If it's useful to you, write it and get on with your life.

It's not worth doing unless it's intended for the stdlib. How many
times do I have to explain this?

phr> Tell me again how rotor got into the distribution.

Python's user community was probably a few hundred people. Guido
likely had no thoughts of world domination with the little language that
could. Times have changed.

If Python is aiming for world domination, it needs to be able to
compete with Java in security features. Java ships with the whole
JSSE system and a much fancier low level crypto architecture than the
one we've been discussing. So I think the Python folks should figure
out how to incorporate crypto without getting crushed under the iron
boot (or at least the wooden shoe) of the Dutch government or whatever
it is that they're worried about.

You seem think there was a PEP process and distutils and Wikis. I
suspect some of the algorithms one might include in a robust crypto
toolkit today weren't even invented in 1992.

AES wasn't invented then. DES became a US federal standard around 1977.

So throw away the rotor crutch and put your money where your mouth is.

If you mean write yet another crypto module knowing ahead of time that
it won't go into the stdlib, that would be more like throwing my money
down the drain.

 

[phr]

Lucas> Umm, is it just me or did we just discuss the legal issues of
Lucas> that??

You may have. Whether or not there are legal issues with them is of little
relevance to the point I was making. Anything Paul writes would probably
have the same legal entanglements.

Andrew's toolkit does incorporate some patented algorithms, but those
aren't critical and could be removed from a stdlib version.

I was simply pointing out that maybe, just maybe, there are already
suitable candidates from a technical standpoint and that he doesn't
need to write anything.

There really don't appear to be any that are both technically
suitable, and that the authors are willing to contribute to PSF. I'd
be delighted to be wrong. Of course that still leaves the legal
issue.

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

I don't see why you can't make up your mind enough to issue simple
statements like "the Python lib should have a module that does
so-and-so

I can say that assuming I know what so-and-so is. For the specific
case of AES, I would say "I don't think the Python lib necessarily
needs to have an AES module, but I would not object if it had one"
(the latter part in consideration of consequences that inclusion
of crypto code might have).

and it should meet such-and-such requirements

I can only say such things if I know such-and-such in detail
to specify requirements. For the specific case of AES, I don't
know enough about it to specify requirements. I will have to
trust others (and by that, I mean *multiple* others)

so if
someone submits one that meets the requirements and passes code review
and testing and doesn't have unexpected issues or otherwise fail to
meet reasonable expectations, we'll use it".

Because I cannot specify requirements, I cannot make such a promise.

In addition, for any new module, there is one primary requirement
for acceptance that cannot be fulfilled in code quality: the
contributor should promise to support the module in a foreseeable
future (i.e. a couple of years).

Again, we're talking about straightforward modules whose basic
interface needs are obvious.

You are talking about such a thing. I don't know enough about
the functionality to specify what an obvious interface is, or
to recognize one if I see it.

I don't know what OMG is, but there is no IETF requirement that any
implementations be available in any particular language.

See RFC 2026, section 4.1.2. Two independent implementations
are required for the document to advance to draft (!) standard.

However, the result of my not writing an AES module is that Python
doesn't have an AES module.

That's not true. PyCrypto does have AES support.

But, it's completely normal to say before step 1 that "if the result
of step 2 does so-and-so, then I'll be pleased in step 3",

That's what I'm saying: If you distribute the module to users for
a year, and users express interest and support for your choice of
API, I'll support inclusion of the module. "do it" involves more than
just writing the code.

You and
Frederik seem to think there's something inappropriate or
self-inflated about wanting that expectation before committing to do a
pile of work that's primarily for other people's benefit.

It's very easy. If you are primarily do it for other people's
benefit, and if you don't find any satisfaction in the process
of doing it - THEN DON'T. I really mean that; this is how
free software works. People *volunteer* to do things. If they
don't volunteer - that's perfectly fine.

I think
your stated attitude is completely bizarre, that you can't really
believe anything so silly, so you're really just acting bureaucratic,
looking for excuses to say no instead of yes to worthwhile proposals.

As I said above - for the specific feature in question, I don't
care enough for the feature itself. Python will be just as useful
to me with the feature as it is without.

What I do care about is the effort that I will need to continue
maintaining Python. I don't want to have to maintain an ill-designed,
buggy module with no active maintainer, and I don't want to tell
people that I had to rip the module out just because it doesn't
work at all.

Regards,
Martin

 

[Paul Rubin]

I can say that assuming I know what so-and-so is. For the specific
case of AES, I would say "I don't think the Python lib necessarily
needs to have an AES module, but I would not object if it had one"

Well, ok, you're changing your tune a little bit now, and getting more
reasonable. Before, you were making blanket statements of any modules
written for the Python stdlib. Now you're limiting it to AES and basing
it on some AES-specific things.

(the latter part in consideration of consequences that inclusion
of crypto code might have).

I would say the first thing I'd request would be your sense of how
plausible it is that the current no-crypto policy might be relaxed.
And I think this is about totally up to Guido, since (reading between
the lines of those python-dev messages) he's concerned about the Dutch
crypto restrictions, which makes some sense because he's Dutch and
still has to deal with the Dutch government from time to time, while
you or I might not care what the Dutch government thinks.

I was about to mention that Rop Gongrijp has been operating a crypto
company in Holland (www.nah6.com) but I now see that it's moved to
Germany, and I wonder if those Dutch restrictions might be the reason.
I don't know any details though.

I can only say such things if I know such-and-such in detail
to specify requirements. For the specific case of AES, I don't
know enough about it to specify requirements. I will have to
trust others (and by that, I mean *multiple* others)

OK. If it helps, I can tell you that the Python crypto crowd hangs
out on a mailing list called python-crypto and has discussed this
module at some length.

Because I cannot specify requirements, I cannot make such a promise.

Again OK. I had thought from earlier discussions that you were a
crypto developer and familiar with this stuff; no problem if you're
not. However, in that case, you probably wouldn't be using the module
if it got included. I think the best policy if you don't feel
comfortable supporting including some module (whether it's crypto or
something else) that you're not personally going to use, is don't
support inclusion, but also don't obstruct it. If there's some other
stdlib maintainers who are knowledgeable about that kind of module and
have an opinion about it, then go along with them unless there's a
clear reason not to. Like, if I were a stdlib maintainer and somebody
submitted a fancy multigrid PDE solver, that's outside my expertise
and I'd go along with whatever Tim recommended.

In addition, for any new module, there is one primary requirement
for acceptance that cannot be fulfilled in code quality: the
contributor should promise to support the module in a foreseeable
future (i.e. a couple of years).

That's not too unreasonable, though I don't think I've heard it
mentioned as a requirement before.

You are talking about such a thing. I don't know enough about the
functionality to specify what an obvious interface is, or to
recognize one if I see it.

The thing to do then is just defer the whole issue to someone on the
core team who uses that type of function. Only if no one on the team
has an opinion do you start having to look for stuff like wide use of
the module in the outside community.

See RFC 2026, section 4.1.2. Two independent implementations
are required for the document to advance to draft (!) standard.

That says nothing about the implementation languages. Both might be
in Python, or one might be in VHDL and the other in Ada, or whatever.

This particular module we're discussing is a straightforward
implementation of AES, DES, and FIPS 81, for which there are thousands
of implementations in both hardware and software. For the specific
Python API, there's a Python implementation and the proposed new
module is a C implementation that does the exact same thing only
faster. So that's two implementations, though not independent in the
IETF sense, because they'd both be written by the same person.

I've never heard of any requirement before that there be two separate
implementations of every Python stdlib module, by the way. That would
be silly.

That's not true. PyCrypto does have AES support.

That's not in any Python distro that I know of. I'm talking about
modules in the stdlib, not external modules. "Batteries included",
you know.

That's what I'm saying: If you distribute the module to users for
a year, and users express interest and support for your choice of
API, I'll support inclusion of the module. "do it" involves more than
just writing the code.

Well, that's nice of you to offer to support inclusion of a module
that you're not likely to use yourself and whose functionality you
don't really understand (I'm not being sarcastic). If you're going to
offer such support it makes sense to impose an unusually high standard
for offering it. I myself would probably never support including any
such module no matter how long it was distributed, but rather would
just defer the whole question to people experienced with such modules
and trust their sense of what the acceptance criteria should be for a
specific module. That is, I'd abstain from any decision.

It's very easy. If you are primarily do it for other people's
benefit, and if you don't find any satisfaction in the process
of doing it - THEN DON'T. I really mean that; this is how
free software works. People *volunteer* to do things. If they
don't volunteer - that's perfectly fine.

That doesn't map well onto the real world. Say I'm a professional
cook and I have a good recipe for spam, eggs, sausage, and spam. I
write the PyCon organizers and ask if they'd like me to cook up a big
batch to serve for breakfast for PyCon attendees (this is as a
volunteer, all ingredients provided at my own expense, etc).
Reasonable answers might be:

1) No, we don't want to serve breakfast at PyCon, we thought about
it but decided against it. => OK, fine, I don't cook it.

2) Hey, that sounds good--cook enough for 500 people and bring it to
the site before the conference starts, and we'll put it on the
menu unless it's below expectations. => OK, fine, I start
cooking. Nobody has made an ironclad promise, but the organizers
have told me their basic intentions, which is good enough for me.

3) Hmm, that recipe sounds interesting, why don't you send one or
two portions to the organizers, we'll try them and let you know.
=> OK, this is basically a prototype implementation, like the
pure-Python version of the AES module, that's fine for testing
but not fast enough to serve (e.g.) 500 web connections. I
send the small batch and wait for an answer before deciding to
start the much bigger job of making a 500-person batch.

Your answer is more like "make enough for 500 people and bring it to
the conference site and only then will we even THINK about whether to
serve it for breakfast. Serving it at PyCon shouldn't matter to you,
what are you, some kind of egotist? You like cooking, so if you make
the 500 portions and then the organizers decline them, you can always
hand them out in the street. If you don't find any satisfaction in
that, THEN DON'T DO IT."

If the cook's goal is not just to feed Python users in the street, but
to actually improve PyCon itself by providing for PyCon to serve
breakfast to attendees, I hope you can see why he might not think much
of your answer.

As I said above - for the specific feature in question, I don't
care enough for the feature itself. Python will be just as useful
to me with the feature as it is without.

I'm sure that's true of lots of modules. If you're the sole maintainer
of the Python distro, then you have to evaluate every module that anyone
submits. But since you're part of a reasonable sized group, I think
it's best to examine the modules you care about, but don't feel that
you have to have your hands in everything.

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

Well, ok, you're changing your tune a little bit now, and getting more
reasonable. Before, you were making blanket statements of any modules
written for the Python stdlib. Now you're limiting it to AES and basing
it on some AES-specific things.

And I still stand by those blanket statements. Any new module (i.e.
specific code) should see real users for some time before it can
be incorporated into Python.

The question whether I would like to see a certain *feature*
in Python (whether as a separate module, a language feature, or
otherwise) is an entirely different question. That I would not
object to an AES code in general doesn't imply that I will
agree to any AES module that somebody contributes.

I would say the first thing I'd request would be your sense of how
plausible it is that the current no-crypto policy might be relaxed.

There is no such policy in force. Different people have different
opinions, and those opinions change over time. I cannot speak for
others.

Again OK. I had thought from earlier discussions that you were a
crypto developer and familiar with this stuff; no problem if you're
not. However, in that case, you probably wouldn't be using the module
if it got included. I think the best policy if you don't feel
comfortable supporting including some module (whether it's crypto or
something else) that you're not personally going to use, is don't
support inclusion, but also don't obstruct it.

Thanks for the advise, but I'll have my own policies. I have included
several modules in the past which I'm not personally using.

That's not too unreasonable, though I don't think I've heard it
mentioned as a requirement before.

See PEP 2.

I've never heard of any requirement before that there be two separate
implementations of every Python stdlib module, by the way. That would
be silly.

And I did not suggest such a thing. You said you never heard about
a process which requires an implementation before deciding that the
proposed feature is good, and I said that, for an example, IETF even
requires *two* implementations before deciding that the feature is
good.

That's not in any Python distro that I know of.

Sure, but "Python" is more than the Python core.

I myself would probably never support including any
such module no matter how long it was distributed, but rather would
just defer the whole question to people experienced with such modules
and trust their sense of what the acceptance criteria should be for a
specific module. That is, I'd abstain from any decision.

With that attitude, the patches tracker on SF would grow unbounded,
because we *continuously* get patches that no core developer has
any personal interest in. I'll try to look at *all* patches, whenever
I can, and in the process, I have to learn about technologies which
I've never used before (primarily operating systems, but also
APIs and protocols).

Rejecting all these patches would be unfair to the contributors.

Your answer is more like "make enough for 500 people and bring it to
the conference site and only then will we even THINK about whether to
serve it for breakfast.

I cannot see these as the same thing.

I'm sure that's true of lots of modules. If you're the sole maintainer
of the Python distro, then you have to evaluate every module that anyone
submits. But since you're part of a reasonable sized group, I think
it's best to examine the modules you care about, but don't feel that
you have to have your hands in everything.

Thanks again for the advise, but this is not how Python is being
maintained.

Regards,
Martin

 

[Paul Rubin]

And I still stand by those blanket statements. Any new module (i.e.
specific code) should see real users for some time before it can
be incorporated into Python.

Let's see, the urandom module was recently released in 2.4, I think
initially at my urging. I believe it was written just for 2.4,
combining a Windows function used only (as far as I know) by its
author, and a wrapper to read /dev/urandom on Posix systems. It
didn't have any wide distribution or use before incorporation, and
didn't need it. It was simple enough that a reasonable code review
and some testing on the relevant platforms was enough to make make
sure it worked properly.

There is no such policy in force. Different people have different
opinions, and those opinions change over time. I cannot speak for others.

If you can't speak for others, how can you say there's no policy in
force? If Guido says "no crypto", is that something other than a

proposed feature is good, and I said that, for an example, IETF even
requires *two* implementations before deciding that the feature is good.

No, that's completely wrong. IETF requires two implementations before
publishing a specification--a precise description--as a draft
standard, which means agreeing on how the feature should work at the
finest level of detail. That's much different than merely deciding
that a feature is good and inviting people to work on it.

IETF often decides and announces that a feature is good long before
any such details are decided or agreed on. For example, they decided
that IP connection encryption was a good feature, and appointed a
working group a general mandate to go figure out the details (IPSEC)
and write a spec. See? IETF announced in advance that IPSEC was a
good feature, and the working group members knew of that decision
before they committed the resources needed to write the draft. And
when they started, there were zero actual IPSEC implementations,
because nobody knew what precise characteristics IPSEC would
eventually have. The implementations got written in the process of
developing the spec and presenting it as a standard.

The two-implementation policy means there has to be two
implementations before the draft is finally declared complete, not
before the IETF can announce that it wants a standard to be written
and intends to use whatever the working group comes up with once the
draft and two implementations are done. That is what happens in any
normal professional process. Nobody commits resources to develop
something (other than for experimentation) unless they think the end
result will actually be used. Are you really having trouble
understanding this?

Sure, but "Python" is more than the Python core.

Tell me again what "batteries included" means. I personally interpret
"included" as meaning "included". That means when I install Python,
the module is right there in the distro. So if it's not in the
distro, Python itself doesn't have it. "Batteries included" is an
expression of Pythonic philosophy that says: putting components that
end-users need in the distro is superior to making the users get them
from somewhere else separately. I realize that the difference might
not matter to you, but it does matter to a heck of a lot of other
users.

With that attitude, the patches tracker on SF would grow unbounded,
because we *continuously* get patches that no core developer has
any personal interest in.

Obviously those require a different type of consideration. I'm
talking about patches where there's a core developer with an interest.
Let's say the cmath module doesn't exist and someone submits it. It
computes functions like cos(z) for complex arguments. If Tim, the
numerics expert, then says "this is worth having and I've examined the
code and it looks good, let's put it in", I'd say that takes care of
it, and I think that's what typically happens with real modules. If
nobody on the core team has an informed opinion, then it has to be
decided some slower and more formal way.

 

[Terry Hancock]

ah, you're the guy who flamed me and called me names because PIL didn't
fit some hypothetical GNU-inspired definition of "open source software." it's
always sad when people have to attack those who don't share their religion,
but it's not that much I can do about that.

That is a complete misrepresentation of the post in question. It was in
no manner of interpretation a "flame". You simply took offense out of thin
air. And there's not much *I* can do about *that*.

I figured that my intent to contribute to your project was sufficient to
indicate my good intentions.

which means that you have no idea how easy or hard it is to install PIL today
(hint: things have changed)

Oh? In which version?

Are you saying that your installation procedure now honors the "--with-jpeg=" and
"--with-png=" flags to configure? Because although you had these flags, in the last
version I installed from source, they did not work -- it was still necessary to edit
the Makefile.

That was either 1.1.3 or 1.1.4, and I'm pretty sure it was the latter.

Now if you've actually fixed this, I look forward to trying it out the next
time I have to do a complete from-source installation. I'll say "thank you",
just in case.

Cheers,
Terry

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

Let's see, the urandom module was recently released in 2.4, I think
initially at my urging.

There is no urandom module in Python 2.4.

If you can't speak for others, how can you say there's no policy in
force?

I should say I'm not aware of a policy.

If Guido says "no crypto", is that something other than a
policy? And are you speaking for him when you say that? <wink>

If he had said such a thing in general, it would be a policy
(although it best would be documented somewhere). I don't think
he said that, in general, and with the intent of stating a policy.

That's much different than merely deciding
that a feature is good and inviting people to work on it.

We don't normally invite people to work on anything. People
pick the things they work on themselves.

IETF often decides and announces that a feature is good long before
any such details are decided or agreed on. For example, they decided
that IP connection encryption was a good feature, and appointed a
working group a general mandate to go figure out the details (IPSEC)
and write a spec. See?

No. They decided that spam machinery in DNS would be a good thing,
people started to work on it, and then they decided that it is not
such a good thing, after all, because it causes too many problems.

So the decision "this is a good thing" is no guarantee for
"if it is done, we will publish it as a standard". They might
start a process, people might get nearly through it, and then
the process stucks or is given up entirely. Happens all the time.

I realize that the difference might
not matter to you, but it does matter to a heck of a lot of other
users.

It does matter for me, yet Python is still more than the core.

You might be ignoring that, but it surely is more to the many
developers which create Python libraries and distribute them
themselves, see

http://www.python.org/pypi

THIS IS ALL PYTHON.

Obviously those require a different type of consideration. I'm
talking about patches where there's a core developer with an interest.

I though you were talking about the AES module...

Regards,
Martin

 

[Paul Rubin]

There is no urandom module in Python 2.4.

Oops, sorry, it's in the os module:

http://docs.python.org/lib/os-miscfunc.html

The difference is simply a matter of the packaging. It's important
functionality and it was added after an amount of review and testing
that was deemed to be reasonable. Unless you're saying that if I
wanted to add AES to the string module (so you could say
'spam and sausage'.aes_encrypt('swordfish banana')) instead of writing a
separate module, then we wouldn't need this discussion. (Putting it
in the string module is not a serious proposal but it wouldn't be
completely insane.) What matters is the code complexity, not whether
something is in a separate module or not.

If he had said such a thing in general, it would be a policy
(although it best would be documented somewhere). I don't think
he said that, in general, and with the intent of stating a policy.

Well, if he indicates that it's not a policy and that the question is
still open, then I could see getting interested again in writing an
AES module. At the moment I continue to see his python-dev post as
quite discouraging.

We don't normally invite people to work on anything. People pick the
things they work on themselves.

Not true. For example, you once invited me to work on an ancillary
message feature for the socket module (SF bug 814689), and so it's
been on my want-to-do-one-of-these-days list since then. I think it's
reasonable for me to have taken your message there as an expression of
interest, sufficient to get me to want to work on it. So it's bogus
to say the Python developers should avoid expressing interest in
something that hasn't already been written.

No. They decided that spam machinery in DNS would be a good thing,
people started to work on it, and then they decided that it is not
such a good thing, after all, because it causes too many problems.

There's no contradiction. IETF decided something based on some
reasonable beliefs that later turned out to be wrong, so they had to
undo the original decision because the original expectations couldn't
be met. That happens sometimes. Everyone has worked on projects that
ended up failing and getting cancelled for reasons like that. There
are few ironclad guarantees of anything, only reasonable beliefs. But
real-world organizations are not afraid to start projects based on
reasonable beliefs regardless. Otherwise, few things would ever get
done.

It does matter for me, yet Python is still more than the core.

You might be ignoring that, but it surely is more to the many
developers which create Python libraries and distribute them
themselves, see

http://www.python.org/pypi
THIS IS ALL PYTHON.

No. Those are programs people have written in Python or as Python
extensions. I've written programs like that myself. I don't consider
them to be part of Python. They are separate programs. If one of
them gets accepted into the distro (as this thread is about), then it
becomes part of Python. See <http://www.python.org/doc/faq/general.html>:

1.1.6 How do I obtain a copy of the Python source?

The latest Python source distribution is always available from
python.org, at http://www.python.org/download/. The latest development
sources can be obtained via anonymous CVS from SourceForge, at
http://www.sourceforge.net/projects/python.

Python is what you get from following the download instructions and
installing the result. In <http://www.python.org/doc/faq/installed.html>,
where it asks "why is Python installed on my machine?", that is what
they are referring to by "Python".

I though you were talking about the AES module...

No, when this started you were talking about modules in general.