python scalability

[Tim Mitchell]

Hi All,

I work on a desktop application that has been developed using python and
GTK (see www.leapfrog3d.com). We have around 150k lines of python code
(and 200k+ lines of C). We also have a new project manager with a C#
background who has deep concerns about the scalability of python as our
code base continues to grow and we are looking at introducing more
products. I am looking for examples of other people like us (who write
desktop apps in python) with code bases of a similar size who I can
point to (and even better talk to) to help convince him that python is
scalable to 300+ lines of code and beyond. I have looked at the python
success stories page and haven't come up with anyone quite like us.
One of my project managers questions is: "Are we the only company in the
world with this kind and size of project?"
I want to say no, but am having trouble convincing myself, let alone him.

If you are involved in this kind of thing please get in touch with me.

Thanks,
Tim

 

[Mike Hansen]

I have looked at the python

success stories page and haven't come up with anyone quite like us.
One of my project managers questions is: "Are we the only company in the
world with this kind and size of project?"
I want to say no, but am having trouble convincing myself, let alone him.

If you are involved in this kind of thing please get in touch with me.

While Sage ( http://www.sagemath.org ) is more of a library than an
application, it is approximately 350k lines of Python and Cython (in
roughly a 2 to 1 ratio). I think it has scaled surprisingly well.

--Mike

 

[Bruno Desthuilliers]

Tim Mitchell a écrit :

Hi All,

I work on a desktop application that has been developed using python and
GTK (see www.leapfrog3d.com). We have around 150k lines of python code
(and 200k+ lines of C). We also have a new project manager with a C#
background who has deep concerns about the scalability of python as our
code base continues to grow and we are looking at introducing more
products. I am looking for examples of other people like us (who write
desktop apps in python) with code bases of a similar size who I can
point to (and even better talk to) to help convince him that python is
scalable to 300+ lines of code and beyond.

As far as I'm concerned, I'd say that if you didn't feel anything wrong
nor spot any problem so far working on a 150 klocs project, there's just
no need to worry. FWIW, a C# port would probably requires *way* much
than 150 klocs.

I have looked at the python
success stories page and haven't come up with anyone quite like us. One
of my project managers questions is: "Are we the only company in the
world with this kind and size of project?"

Doing a quick count using wc -l (so the following numbers include
comments and blank lines), Zope 2.8 is about 418 klocs, Plone 3 about
259 klocs. Which, for the full stack, makes about 677 klocs. Even if you
estimate a 50/50 ratio between real locs and comments/blank lines, this
still makes something like 338 klocs.

Is that enough ?-)

 

[Michele Simionato]

Hi All,

I work on a desktop application that has been developed using python and
GTK (seewww.leapfrog3d.com).  We have around 150k lines of python code
(and 200k+ lines of C).

We have bigger numbers than yours here (although not for a desktop
application)
and of course we have the problems of a large size application, but
they have
nothing to do with Python. The real problem are sociological, not
language-related.
Essentially, if a project takes 10+ years and 10+ people, with most of
the people
new, you have an issue, but this is independent from the language.
Python is helping
us at least because it is readable and the situation would be probably
be worse with
another language. But as I said the software development practices
used are more
important than the language in this context.

 

[Tim Mitchell]

Thanks for all the replies - they have all been helpful.

On reflection I think our problems are probably design and people related.

Cheers,
Tim

 

[Gerhard Häring]

Thanks for all the replies - they have all been helpful.

On reflection I think our problems are probably design and people related.

I strongly agree. "Scalability" is becoming a buzzword lately, which is
meaningless unless qualified what exactly is meant.

It's overused wrt technologies, programming languages, tools, etc.

But in other interesting areas people think much less about "scalability".

Will the current software development process work with a team of 5, 10,
20, 50 team members? Does it "scale" here?

Will the software development process still work with an increased
turnover rate? Does it "scale"?

Will the way of doing things still work with an increasingly aging codebase?

-- Gerhard

 

[Dan Stromberg]

Hi All,

I work on a desktop application that has been developed using python and
GTK (see www.leapfrog3d.com). We have around 150k lines of python code
(and 200k+ lines of C). We also have a new project manager with a C#
background who has deep concerns about the scalability of python as our
code base continues to grow and we are looking at introducing more
products. I am looking for examples of other people like us (who write
desktop apps in python) with code bases of a similar size who I can
point to (and even better talk to) to help convince him that python is
scalable to 300+ lines of code and beyond. I have looked at the python
success stories page and haven't come up with anyone quite like us. One
of my project managers questions is: "Are we the only company in the
world with this kind and size of project?" I want to say no, but am
having trouble convincing myself, let alone him.

If you are involved in this kind of thing please get in touch with me.

Thanks,
Tim

It sounds less like concerns about python, and more about "nobody ever
got fired for going microsoft" (except they have).

You might focus not just on large python projects, but also major
companies with substantial commitments to python.

That said, not all python implementations scale that well with
multithreading. It sounds like multithreaded apps are fine for single
CPU machines, and fine on multi-CPU boxes if the app is more I/O bound
than CPU bound - but that's all a matter of implementation, not language
definition, and some python implementations purportedly do well anyway.

It might not hurt to mention IronPython either, since it's a python from
microsoft.

 

[Paul Hankin]

I work on a desktop application that has been developed using python and
GTK (seewww.leapfrog3d.com).  We have around 150k lines of python code
(and 200k+ lines of C).  We also have a new project manager with a C#
background who has deep concerns about the scalability of python as our
code base continues to grow and we are looking at introducing more
products.  I am looking for examples of other people like us (who write
desktop apps in python) with code bases of a similar size who I can
point to (and even better talk to) to help convince him that python is
scalable to 300+ lines of code and beyond.  I have looked at the python
success stories page and haven't come up with anyone quite like us.
One of my project managers questions is: "Are we the only company in the
world with this kind and size of project?"
I want to say no, but am having trouble convincing myself, let alone him.

If you are involved in this kind of thing please get in touch with me.

Perhaps your manager is more concerned about python because he's less
familiar with it, rather than because of scalability problems? Perhaps
using python is actually a competitive advantage for your company?

Have you seen this?
http://www.paulgraham.com/pypar.html