Python to use a non open source bug tracker?

[David Goodger]

The current request is: "please, readers of python-dev, setup a team of 6-10
people to handle roundup or we'll go to a non-free software for bug
tracking". This is something which I cannot cope with, and I'm *speaking*
up against. Were the request lowered to something more reasonable, I'd be
willing to *act*.

No, the announcement stated the situation in a very different way.

Asking for a group of maintainers to commit to an essential piece of
infrastructure is perfectly reasonable. Brett didn't ask for 6-10 full
time developer/sysadmins. He asked for typical commitment, which is up
to a few hours per week. The initial work will probably be significant,
but will undoubtedly taper off over time.

Go back to the original announcement:

"""
After evaluating the trackers on several points (issue creation,
querying, etc.), we reached a tie between JIRA and Roundup in terms of
pure tracker features.
"""

JIRA gets a leg up because of the hosting and administration also being
offered. But...

"""
If enough people step forward we will notify python-dev that Roundup
should be considered the recommendation of the committee and graciously
turn down Atlassian's offer.
"""

That is a perfectly reasonable offer. Put up or shut up.

And besides the only thing I'm really sniping the PSF against is about
*ever* having thought of non-FLOSS software. This is something I *really* do
not accept. ... I just
disagree with their initial requirements (and I have not raised this point
before because, believe me if you can, I really thought it was obvious and
implicit).

That just shows that you were being naïve. The initial requirements
were published openly and clearly.

I do respect the fact
that the PSF committee did a thorough and correct evaluation:

Yes, they did, and you should be thanking them instead of complaining.

If you feel so strongly, please volunteer.

-- David Goodger

 

[Fredrik Lundh]

No, actually switching trackers can be one big pain in the ass. You
probably aren't aware of how hard it's been for the Python development team
(I think Martin v. Loewis, mostly) to get tracker data out of SF.

http://effbot.org/zone/sandbox-sourceforge.htm

</F>

 

[=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=]

The community, and I am impressed you do not want to understand the "why".

How would "the community" actually reject it? By not using it? Well,
that won't happen: They use the current SF tracker, despite it being
non-free software.

It is an extremely bad picture for an open source flag like Python to go to a
vendor for such an easy requirement as a bug database.

You fail to recognize that Python is *already* using a non-free software
for bug tracking, as do thousands of other projects. So from that point
of view, the status wouldn't change.

Regards,
Martin

 

[skip]

Fredrik> http://effbot.org/zone/sandbox-sourceforge.htm

Thanks for the memory jog. Before you wrote your scraper/downloader I seem
to recall there were several only-semi-successful attempts to get SF to
themselves dump the tracker data for us. That was what I was referring to.

Skip

 

[=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=]

In fact, are you absolutely positive that you need so much effort to
maintain an existing bugtracker installation? I know for sure that GCC's
Bugzilla installation is pretty much on its own; Daniel Berlin does some
maintainance every once in a while (upgrading when new versions are out,
applying or writing some patches for most requested features in the
community, or sutff like that), but it's surely not his job, not even
part-time.

Daniel Berlin has put a tremendous amount of work into it. I know,
because I set up the first bug tracker for gcc (using GNATS), and
have been followed the several years of pondering fairly closely.
It was quite some work to set up GNATS, and it was even more work
to setup bugzilla.

For Python, we don't have any person similar to Daniel Berlin
(actually, we have several who *could* have done similar work,
but none that ever volunteered to do it). Don't underestimate
the work of somebody else.

I know that I'm currently putting more time into maintaining the
Subversion installation than I'd like to, despite this being a really
small amount of work per month, and despite others also having been
introduced to the infrastructure necessary for administration.
When I go on vacation, people effectively have to wait until
I return before they can get their write access enabled. I
sometimes defer dealing with admin requests for a few days, and
people start complaining.

Regards,
Martin

 

[=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=]

Frankly, I don't give a damn about the language the application is coded in

That's probably one of the reasons why you aren't a member of the Python
Software Foundation. Its mission includes to publicize, promote the
adoption of, and facilitate the ongoing development of Python-related
technology and educational resources. So the tracker being written in
Python is quite of importance.

Regards,
Martin

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

I'm not on the infrastructure list either. But I wonder why it is
"Roundup or else non-python COTS"? I gave up on Roundup a while ago
due to too many crashes. I'm now using Trac:

a) Open Source
b) Python
c) Adequate functionality (for me at least)

http://trac.edgewall.org/

I'm not trying to "sell" Trac, but I would like to know what drove the
developers away from it.

IMO, the biggest concern was that it didn't really "scale". I.e. if you
have a thousand open and several thousand closed reports in the
database, you need excellent support for queries, sorting, and alike.
Trac doesn't quite have the same power that the other tools do.
For example, to save a query/report, you actually have to write it in
SQL. For a complex report (with multiple groups), you cannot change
the order of items returned on the page (for a simple search, clicking
on the headings changes the order).

To see what I mean, please try to find out how many open reports
for the module "report system" are currently entered in the tracker
at

http://trac.edgewall.org/report

How many of these are for 0.9.x?

There is also searching, which doesn't give a tabular view of all
tickets, but instead gives a Web search result page (similar
to what a search engine produces). This makes it difficult to scan
systematically for, say, all titles.

Regards,
Martin

 

[Paul Rubin]

You fail to recognize that Python is *already* using a non-free software
for bug tracking, as do thousands of other projects.

I don't think that reflects an explicit decision. SF started out as
free software and the software became nonfree after people were
already using it.

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

No, actually switching trackers can be one big pain in the ass. You
probably aren't aware of how hard it's been for the Python development team
(I think Martin v. Loewis, mostly) to get tracker data out of SF. An
explicit requirement was that any tool chosen as a SF replacement be able to
easily export its database to avoid this sort of "lock-in" in the future.

While I put quite some effort into getting data out of SF, it was
Fredrik Lundh who eventually implemented the solution that we'll use:
screen scraping. My efforts to get data out of SF "officially"
were futile.

Regards,
Martin

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

Out of interest, here are some figures:

KDE: 12983 bugs and 11656 wishes
GNOME: 23624 reports
Python: 7159 bugs, 3843 patches, 477 feature requests

The Python figures are totals, whereas I can't be sure whether the KDE
and GNOME figures merely refer to the open issues. Nevertheless, Python
isn't going to be pushing the envelope.

Right: machine power isn't really an issue (although the snappiness
of response does contribute to usability: the various installations
showed noticable differences in response time).

The issue of scalability is really how a large number of reports
can be managed. How can a submitter easily find out whether a report
for some issue already exists, and how can a maintainer easily find
out what needs most attention (where each developer might apply
her own priority system)? For a small number of issues, you can
scan through a list. If the list contains more than 20 entries,
scanning through it becomes tedious.

Regards,
Martin

 

[Giovanni Bajo]

That's probably one of the reasons why you aren't a member of the
Python Software Foundation. Its mission includes to publicize,
promote the
adoption of, and facilitate the ongoing development of Python-related
technology and educational resources. So the tracker being written in
Python is quite of importance.

So we have a problem between the PSF and the "PSF infrastructure committee",
since the latter did not put "being written in Python" has a requirement for
the tracker.

 

[Giovanni Bajo]

I don't think that reflects an explicit decision. SF started out as
free software and the software became nonfree after people were
already using it.

Moreover, this looked like a very good chance to have this nuisance sorted out.
Too bad some people don't value free software enough.

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

I don't think that reflects an explicit decision. SF started out as
free software and the software became nonfree after people were
already using it.

That, in principle, could happen to any other free software as well.
What is critical here is that SF *hosted* the installation. If we would
use a tracker that is free software, yet hosted it elsewhere, the same
thing could happen: the hoster could make modifications to it which
are non-free. Not even the GPL could protect from this case: the
hoster would be required to publish source only if he publishes
binaries, but he wouldn't publish any binaries, so he wouldn't need
to release the source changes, either.

Also, even if it the software is open source and unmodified, there
still wouldn't be a guarantee that you can get the data out of it
if you want to. You *only* get the advantages of free software if
you also run it yourself. Unfortunately, there is a significant
cost associated with running the software yourself.

Despite what other people say, this *is* an issue. On python.org,
things that should get done don't, just because there is no
volunteer doing them. Hosting such a service elsewhere has the
clear advantage that you don't have to worry about most routine
maintenance jobs.

Regards,
Martin

 

[Giovanni Bajo]

I don't think Python has ever had this as a goal. Python's license
lets it be embedded in closed-source products; Windows binaries are
built using closed-source tools (MS Visual C), and on some platforms
we use a closed-source system compiler; python.org used to be a
Solaris box, and now uses SourceForge which runs on top of DB/2...

Notice that there is a different between "allowing/helping/supporting non-free
software" and "avoid reliance on non-free software". The fact that Python
license allows it to be used in non-free products falls in the former, while
the usage of Jira is part of the latter. Distributing binaries compiled with
closed-source tools is not a problem since people can still compile it with
different free compilers.

IMHO, using Jira presents risks that are manageable:
[...]

* A data export is available if we decide to switch. [...]

Out of curiosity, how is this obtained? Is this any plan to take a daily export
or so?

 

[Giovanni Bajo]

Go back to the original announcement:

"""
After evaluating the trackers on several points (issue creation,
querying, etc.), we reached a tie between JIRA and Roundup in terms of
pure tracker features.
"""

JIRA gets a leg up because of the hosting and administration also
being offered. But...

"""
If enough people step forward we will notify python-dev that Roundup
should be considered the recommendation of the committee and
graciously
turn down Atlassian's offer.
"""

That is a perfectly reasonable offer. Put up or shut up.

You're cherry picking your quotes:

"""
In order for Roundup to be considered equivalent in terms of an overall
tracker package there needs to be a sufficient number of volunteer admins
(roughly 6 - 10 people) who can help set up and maintain the Roundup
installation.
"""

This is *NOT* a perfectly reasonable offer, because you do not see 6-10 people
stepping up at the same time for almost *anything* in the open source world.

 

[Fredrik Lundh]

No, I'm not on the infrastructure list, but I know that capable people
*are*: and you know I am quite capable of donating my time to the cause,
when I have it to spare (and sometimes even when I don't).

what I was trying to say (between the lines) was that not only have
the people on that list worked hard to do the evaluation (not to mention
all the developers around the world that has worked even harder to set
up test trackers), there's also been a good community response to the
committee's call for "6-10 volunteers".

</F>

 

[Paul Rubin]

That, in principle, could happen to any other free software as well.
What is critical here is that SF *hosted* the installation. If we would
use a tracker that is free software, yet hosted it elsewhere, the same
thing could happen: the hoster could make modifications to it which
are non-free. Not even the GPL could protect from this case: the
hoster would be required to publish source only if he publishes
binaries, but he wouldn't publish any binaries, so he wouldn't need
to release the source changes, either.

True, though GPL 3 tries to address that. Most important is to figure
out the underlying attitude of the host. I realize it's the same
crufty software (or worse) as SF and therefore maybe not so attractive
on those grounds already, but did you think about migrating to
Savannah?

Also, even if it the software is open source and unmodified, there
still wouldn't be a guarantee that you can get the data out of it
if you want to. You *only* get the advantages of free software if
you also run it yourself. Unfortunately, there is a significant
cost associated with running the software yourself.

Well, if the cash is available, there's always the possibility of
using free software and paying someone to host it. Anyway, I wouldn't
have expected running a tracker to be that significant a task compared
with the rest of the web site, the mailing lists, the Subversion
server, the codebase itself, etc. etc. But Paul Boddie explained some
of the issues pretty well.

Despite what other people say, this *is* an issue. On python.org,
things that should get done don't, just because there is no
volunteer doing them. Hosting such a service elsewhere has the
clear advantage that you don't have to worry about most routine
maintenance jobs.

I have to wonder too why Jira is so sure to be more reliable than SF.

 

[David Goodger]

So we have a problem between the PSF and the "PSF infrastructure committee",
since the latter did not put "being written in Python" has a requirement for
the tracker.

There was no problem. The committee had their mandate, to find the best
candidate for a tracker for Python. The quality of the tracker was felt
to be more important than the language it was written in or the license
it uses. The committee members worked hard to fulfill their mandate,
and they deserve the gratitude and appreciation of all of us.

I know for a fact (having raised this very issue with the committee
myself) that they took "written in Python" into consideration. Not as a
requirement though -- the requirement was simply to decide on the best
tracker, regardless of language or license.

Look at the results again. Jira and RoundUp tied for functionality, but
Jira has a hosting/admin offer behind it. That's huge. But rather than
declaring Jira the outright winner, which they could have done, the
committee has allowed the community to decide the matter. If enough
admins come forward, RoundUp will win.

I read that as a big push for "written in Python".

David Goodger

 

[David Goodger]

You're cherry picking your quotes:

"""
In order for Roundup to be considered equivalent in terms of an overall
tracker package there needs to be a sufficient number of volunteer admins
(roughly 6 - 10 people) who can help set up and maintain the Roundup
installation.
"""

This is *NOT* a perfectly reasonable offer,

Yes it is:
Jira = 1st place functionality + hosting/admin provided
RoundUp = 1st place functionality

Jira wins. But the offer is to allow the community to make up the
difference. If anything, that's unfair to Jira. So why are you
complaining?

because you do not see 6-10 people
stepping up at the same time for almost *anything* in the open source world.

Then that's a failure of the open source world.

But I *do* see it happening right now. People *are* stepping up.

David Goodger

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

True, though GPL 3 tries to address that. Most important is to figure
out the underlying attitude of the host. I realize it's the same
crufty software (or worse) as SF and therefore maybe not so attractive
on those grounds already, but did you think about migrating to
Savannah?

We had a very clear procedure. We (the committee) didn't want to
manage the installation ourselves, or figure out how to set up
the software, or get the data into it. Instead, we sent out a call
to the community to come up with a demo installation for evaluation
purposes. Nobody offered to migrate the data into Savannah, so
we didn't consider it (nobody actually offered to show-case
Savannah for us, period).

There were several reasons to get off SF; it not being open
source was never a reason. Instead, ongoing complaints about
service level, and the UI were the main complaints. Savannah
is IMO worse than SF wrt. user interface, so it would have
lost in the evaluation even if a demo installation was provided.
We want to improve with that switch, not decrease usability.

I have to wonder too why Jira is so sure to be more reliable than SF.

It may change as time evolves, but at the moment, they are *pretty*
responsive to our inquiries. Atlassian (the company behind it) uses
the same infrastructure for their commercial offerings, as well;
this might mean that we get the same availability (it might also
mean that paying customers get more attention than non-paying ones
in the long term).

With any kind of partner, there is always the risk that they don't
deliver, and you always have to invest some trust in the beginning.
It was this way when Guido moved Python to SF, and indeed, SF did
a very good job for several years (IMO). They only went unreliable
when they grow beyond expectations. The same could happen to
Atlassian, of course, in which case we would have to move again.

OTOH, the same could also happen with a group of volunteers.
It's always possible that they all run away (like that distutils
is unmaintained, and PyXML is unmaintained). Volunteers are actually
unlikely to persist in their efforts over a period of 10 years,
as their lifes and priorities change over time. If you trust
such a service to a single volunteer, you might find that the
service can become very unusable very quickly. For example, the
Python Job Board was in a very bad shape for several months,
until we managed to find Peter Kropf to take it over (who
does a very good job ever since he started).

Regards,
Martin