python scalability

Discussion in 'Python' started by Tim Mitchell, Jul 10, 2008.

  1. Tim Mitchell

    Tim Mitchell Guest

    Hi All,

    I work on a desktop application that has been developed using python and
    GTK (see www.leapfrog3d.com). We have around 150k lines of python code
    (and 200k+ lines of C). We also have a new project manager with a C#
    background who has deep concerns about the scalability of python as our
    code base continues to grow and we are looking at introducing more
    products. I am looking for examples of other people like us (who write
    desktop apps in python) with code bases of a similar size who I can
    point to (and even better talk to) to help convince him that python is
    scalable to 300+ lines of code and beyond. I have looked at the python
    success stories page and haven't come up with anyone quite like us.
    One of my project managers questions is: "Are we the only company in the
    world with this kind and size of project?"
    I want to say no, but am having trouble convincing myself, let alone him.

    If you are involved in this kind of thing please get in touch with me.

    Thanks,
    Tim
    Tim Mitchell, Jul 10, 2008
    #1
    1. Advertising

  2. Tim Mitchell

    Mike Hansen Guest

    > I have looked at the python
    > success stories page and haven't come up with anyone quite like us.
    > One of my project managers questions is: "Are we the only company in the
    > world with this kind and size of project?"
    > I want to say no, but am having trouble convincing myself, let alone him.
    >
    > If you are involved in this kind of thing please get in touch with me.


    While Sage ( http://www.sagemath.org ) is more of a library than an
    application, it is approximately 350k lines of Python and Cython (in
    roughly a 2 to 1 ratio). I think it has scaled surprisingly well.

    --Mike
    Mike Hansen, Jul 10, 2008
    #2
    1. Advertising

  3. Tim Mitchell a écrit :
    > Hi All,
    >
    > I work on a desktop application that has been developed using python and
    > GTK (see www.leapfrog3d.com). We have around 150k lines of python code
    > (and 200k+ lines of C). We also have a new project manager with a C#
    > background who has deep concerns about the scalability of python as our
    > code base continues to grow and we are looking at introducing more
    > products. I am looking for examples of other people like us (who write
    > desktop apps in python) with code bases of a similar size who I can
    > point to (and even better talk to) to help convince him that python is
    > scalable to 300+ lines of code and beyond.


    As far as I'm concerned, I'd say that if you didn't feel anything wrong
    nor spot any problem so far working on a 150 klocs project, there's just
    no need to worry. FWIW, a C# port would probably requires *way* much
    than 150 klocs.

    > I have looked at the python
    > success stories page and haven't come up with anyone quite like us. One
    > of my project managers questions is: "Are we the only company in the
    > world with this kind and size of project?"


    Doing a quick count using wc -l (so the following numbers include
    comments and blank lines), Zope 2.8 is about 418 klocs, Plone 3 about
    259 klocs. Which, for the full stack, makes about 677 klocs. Even if you
    estimate a 50/50 ratio between real locs and comments/blank lines, this
    still makes something like 338 klocs.

    Is that enough ?-)
    Bruno Desthuilliers, Jul 10, 2008
    #3
  4. On Jul 10, 6:32 am, Tim Mitchell <> wrote:
    > Hi All,
    >
    > I work on a desktop application that has been developed using python and
    > GTK (seewww.leapfrog3d.com).  We have around 150k lines of python code
    > (and 200k+ lines of C).


    We have bigger numbers than yours here (although not for a desktop
    application)
    and of course we have the problems of a large size application, but
    they have
    nothing to do with Python. The real problem are sociological, not
    language-related.
    Essentially, if a project takes 10+ years and 10+ people, with most of
    the people
    new, you have an issue, but this is independent from the language.
    Python is helping
    us at least because it is readable and the situation would be probably
    be worse with
    another language. But as I said the software development practices
    used are more
    important than the language in this context.
    Michele Simionato, Jul 10, 2008
    #4
  5. Tim Mitchell

    Tim Mitchell Guest

    Thanks for all the replies - they have all been helpful.

    On reflection I think our problems are probably design and people related.

    Cheers,
    Tim

    Michele Simionato wrote:
    > On Jul 10, 6:32 am, Tim Mitchell <> wrote:
    >
    >> Hi All,
    >>
    >> I work on a desktop application that has been developed using python and
    >> GTK (seewww.leapfrog3d.com). We have around 150k lines of python code
    >> (and 200k+ lines of C).
    >>

    >
    > We have bigger numbers than yours here (although not for a desktop
    > application)
    > and of course we have the problems of a large size application, but
    > they have
    > nothing to do with Python. The real problem are sociological, not
    > language-related.
    > Essentially, if a project takes 10+ years and 10+ people, with most of
    > the people
    > new, you have an issue, but this is independent from the language.
    > Python is helping
    > us at least because it is readable and the situation would be probably
    > be worse with
    > another language. But as I said the software development practices
    > used are more
    > important than the language in this context.
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >
    Tim Mitchell, Jul 10, 2008
    #5
  6. Tim Mitchell wrote:
    > Thanks for all the replies - they have all been helpful.
    >
    > On reflection I think our problems are probably design and people related.


    I strongly agree. "Scalability" is becoming a buzzword lately, which is
    meaningless unless qualified what exactly is meant.

    It's overused wrt technologies, programming languages, tools, etc.

    But in other interesting areas people think much less about "scalability".

    Will the current software development process work with a team of 5, 10,
    20, 50 team members? Does it "scale" here?

    Will the software development process still work with an increased
    turnover rate? Does it "scale"?

    Will the way of doing things still work with an increasingly aging codebase?

    -- Gerhard
    Gerhard Häring, Jul 11, 2008
    #6
  7. On Thu, 10 Jul 2008 16:32:38 +1200, Tim Mitchell wrote:

    > Hi All,
    >
    > I work on a desktop application that has been developed using python and
    > GTK (see www.leapfrog3d.com). We have around 150k lines of python code
    > (and 200k+ lines of C). We also have a new project manager with a C#
    > background who has deep concerns about the scalability of python as our
    > code base continues to grow and we are looking at introducing more
    > products. I am looking for examples of other people like us (who write
    > desktop apps in python) with code bases of a similar size who I can
    > point to (and even better talk to) to help convince him that python is
    > scalable to 300+ lines of code and beyond. I have looked at the python
    > success stories page and haven't come up with anyone quite like us. One
    > of my project managers questions is: "Are we the only company in the
    > world with this kind and size of project?" I want to say no, but am
    > having trouble convincing myself, let alone him.
    >
    > If you are involved in this kind of thing please get in touch with me.
    >
    > Thanks,
    > Tim


    It sounds less like concerns about python, and more about "nobody ever
    got fired for going microsoft" (except they have).

    You might focus not just on large python projects, but also major
    companies with substantial commitments to python.

    That said, not all python implementations scale that well with
    multithreading. It sounds like multithreaded apps are fine for single
    CPU machines, and fine on multi-CPU boxes if the app is more I/O bound
    than CPU bound - but that's all a matter of implementation, not language
    definition, and some python implementations purportedly do well anyway.

    It might not hurt to mention IronPython either, since it's a python from
    microsoft.
    Dan Stromberg, Jul 12, 2008
    #7
  8. Tim Mitchell

    Paul Hankin Guest

    On Jul 10, 5:32 am, Tim Mitchell <> wrote:
    > I work on a desktop application that has been developed using python and
    > GTK (seewww.leapfrog3d.com).  We have around 150k lines of python code
    > (and 200k+ lines of C).  We also have a new project manager with a C#
    > background who has deep concerns about the scalability of python as our
    > code base continues to grow and we are looking at introducing more
    > products.  I am looking for examples of other people like us (who write
    > desktop apps in python) with code bases of a similar size who I can
    > point to (and even better talk to) to help convince him that python is
    > scalable to 300+ lines of code and beyond.  I have looked at the python
    > success stories page and haven't come up with anyone quite like us.
    > One of my project managers questions is: "Are we the only company in the
    > world with this kind and size of project?"
    > I want to say no, but am having trouble convincing myself, let alone him.
    >
    > If you are involved in this kind of thing please get in touch with me.


    Perhaps your manager is more concerned about python because he's less
    familiar with it, rather than because of scalability problems? Perhaps
    using python is actually a competitive advantage for your company?

    Have you seen this?
    http://www.paulgraham.com/pypar.html

    --
    Paul Hankin
    Paul Hankin, Jul 12, 2008
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Wenning Qiu
    Replies:
    7
    Views:
    518
  2. Jeff Epler
    Replies:
    3
    Views:
    330
    Afanasiy
    Jul 10, 2003
  3. Simon Wittber (Maptek)

    RE: Embedding Python, threading and scalability

    Simon Wittber (Maptek), Jul 11, 2003, in forum: Python
    Replies:
    7
    Views:
    358
  4. Alec Taylor
    Replies:
    1
    Views:
    323
  5. Replies:
    7
    Views:
    132
    Chris Angelico
    Jan 19, 2014
Loading...

Share This Page