Help me choose a C++ compiler to work with Python

Discussion in 'Python' started by rhmd, Sep 21, 2003.

  1. rhmd

    rhmd Guest

    Just found Python and I love it. What an elegant language!
    I would like to use it for various applications, but the
    mathematical calculations are way too slow (a million sines 8 seconds
    in Python vs. 2 seconds in Excel VBA), so was thinking of learning
    enough C++ to do the number crunching in C++ and integrating the C++
    routines with Python. My question:
    My question:
    Which compiler works best with Python?
    I use Windows XP and 98. Have very little experience with C++. Am
    doing statistical and other simulations that require billions of
    calculations (taking 1 to 2 hours in Excel VBA). GCC is free, MS
    C++.Net is affordable at <$100 but supposedly has problems until the
    next version (and I could only use it on my XP machine), older
    versions of MS C++ are no longer for sale. What's the most robust
    solution? I would also like to make .dll files that I can plug into
    MS Excel.
    Sorry for the vagueness of this question, but I'm sure many of you
    have experience with this and any information, opinions, and even
    prejudices about the various compilers are welcome.
    Thanks.
    rhmd, Sep 21, 2003
    #1
    1. Advertising

  2. rhmd

    Peter Hansen Guest

    rhmd wrote:
    >
    > I use Windows XP and 98. Have very little experience with C++. Am
    > doing statistical and other simulations that require billions of
    > calculations


    Have you considered and investigated Numeric Python or similar
    products? Perhaps your assumption that you have to revert to
    hand-crafted C code is premature.

    -Peter
    Peter Hansen, Sep 21, 2003
    #2
    1. Advertising

  3. rhmd

    Cy Edmunds Guest

    "rhmd" <> wrote in message
    news:...
    > Just found Python and I love it. What an elegant language!


    Yeah, me too. :)

    > I would like to use it for various applications, but the
    > mathematical calculations are way too slow (a million sines 8 seconds
    > in Python vs. 2 seconds in Excel VBA), so was thinking of learning
    > enough C++ to do the number crunching in C++ and integrating the C++
    > routines with Python.


    This one really has me squirming in my seat. C++ isn't a very good language
    for learning just a little bit of. Compared to Python I would say the
    learning curve is much steeper. It offers a lot of control over how the
    object code is generated but that also makes it dangerous for casual users.
    If you do it wrong a lot of bad things can happen, including poor
    performance -- although poor performance would perhaps be the best of those
    bad things!

    I suggest you look at various Python libraries to do whatever numerical
    processing you might need to do. They are mostly written in C anyway so
    their performance is excellent. For instance, I use SciPy for Fourier
    analysis and PIL for image processing, getting good performance on these
    numerically intensive tasks without ever leaving the ease and comfort of
    good old Python.

    My question:
    > My question:
    > Which compiler works best with Python?


    I wouldn't choose your C++ compiler for Python compatibility. How good is
    the debugger? How is the standards conformance? C++ compilers vary widely in
    these and other important criteria.

    > I use Windows XP and 98. Have very little experience with C++. Am
    > doing statistical and other simulations that require billions of
    > calculations (taking 1 to 2 hours in Excel VBA).


    Take a look at the UVS library under my sig for some C++ code which I have
    found useful for statistical simulations.

    GCC is free, MS
    > C++.Net is affordable at <$100 but supposedly has problems until the
    > next version (and I could only use it on my XP machine), older
    > versions of MS C++ are no longer for sale. What's the most robust
    > solution?


    The .NET compiler has a very good debugger and much better standard
    compliance than the old 6.0 compiler, but I haven't used lot of different
    recent vintage C++ compilers.

    I would also like to make .dll files that I can plug into
    > MS Excel.


    I generally use COM objects written in Python to connect to Excel. I am very
    pleased with that technique.

    http://starship.python.net/crew/mhammond/

    > Sorry for the vagueness of this question, but I'm sure many of you
    > have experience with this and any information, opinions, and even
    > prejudices about the various compilers are welcome.
    > Thanks.


    --
    Cy
    http://home.rochester.rr.com/cyhome/
    Cy Edmunds, Sep 21, 2003
    #3
  4. rhmd

    John J. Lee Guest

    "Cy Edmunds" <> writes:

    > "rhmd" <> wrote in message
    > news:...

    [...]
    > This one really has me squirming in my seat. C++ isn't a very good language
    > for learning just a little bit of. Compared to Python I would say the


    Agreed! But Koenig & Moo's book (Accelerated C++) is the best I've
    seen for doing that.

    [...]
    > I wouldn't choose your C++ compiler for Python compatibility. How good is


    I don't think that's unreasonable, as long as you're *aware* of all
    the other issues. Ease of use with standard Python might come out as
    most important. And on Windows, Python work with MSVC is much less of
    a pain for most things than gcc (aka mingw).

    [...]
    > The .NET compiler has a very good debugger and much better standard
    > compliance than the old 6.0 compiler, but I haven't used lot of different
    > recent vintage C++ compilers.


    What *is* the current position with MSVC 7 and Python? Anyone? I
    still haven't heard a clear story.


    John
    John J. Lee, Sep 21, 2003
    #4
  5. rhmd

    John J. Lee Guest

    (rhmd) writes:
    [...]
    > I would like to use it for various applications, but the
    > mathematical calculations are way too slow (a million sines 8 seconds
    > in Python vs. 2 seconds in Excel VBA), so was thinking of learning
    > enough C++ to do the number crunching in C++ and integrating the C++
    > routines with Python. My question:

    [...]

    Do remember that frequently only small parts of your code (even the
    number crunching part) are responsible for most of the running time.
    Writing everything in Python then porting the 'hot spots' is often the
    best strategy. Use the profiler.

    Also (apart from Numeric / numarray), remember (or discover) Pyrex and
    psyco. Probably Pyrex is more useful to you than psyco. Don't apply
    either to your whole program. Just pick the bottlenecks.

    Quite a few people here have touted O'Caml as a replacement for C++
    for this kind of thing. C is worth considering, too, especially if
    you find the parts you need to move down to a lower-level language are
    small.

    If you insist on using C++, make sure to look at Boost.Python
    (including pyste).


    John
    John J. Lee, Sep 21, 2003
    #5
  6. Peter Hansen wrote:

    > rhmd wrote:
    >>I use Windows XP and 98. Have very little experience with C++. Am
    >>doing statistical and other simulations that require billions of
    >>calculations

    >
    > Have you considered and investigated Numeric Python or similar
    > products? Perhaps your assumption that you have to revert to
    > hand-crafted C code is premature.
    >
    > -Peter


    Look at the enthought Python installation (one is available for each
    of 2.2 and 2.3) -- they not only install a slew of the packages you
    might need for calculation heavy Python use, but (to support weave,
    I believe), they also include a mingw32 compiler (minimal gnu win32)
    which you can use for C++. If you get to the point where you are
    optimization-bound on your compiler (which you should determine by
    measuring, not intuiting), you could do worse than the Intel compiler.

    -Scott David Daniels
    Scott David Daniels, Sep 22, 2003
    #6
  7. rhmd

    d2003xx Guest

    (rhmd) wrote in message news:<>...
    > Just found Python and I love it. What an elegant language!
    > I would like to use it for various applications, but the
    > mathematical calculations are way too slow (a million sines 8 seconds
    > in Python vs. 2 seconds in Excel VBA), so was thinking of learning
    > enough C++ to do the number crunching in C++ and integrating the C++
    > routines with Python. My question:
    > My question:
    > Which compiler works best with Python?
    > I use Windows XP and 98. Have very little experience with C++. Am
    > doing statistical and other simulations that require billions of
    > calculations (taking 1 to 2 hours in Excel VBA). GCC is free, MS
    > C++.Net is affordable


    The standard C/C++ compiler (no optimization) is included in .Net
    framework SDK, which is available for free in M$'s web-site.

    > at <$100 but supposedly has problems until the
    > next version (and I could only use it on my XP machine), older
    > versions of MS C++ are no longer for sale. What's the most robust
    > solution? I would also like to make .dll files that I can plug into
    > MS Excel.
    > Sorry for the vagueness of this question, but I'm sure many of you
    > have experience with this and any information, opinions, and even
    > prejudices about the various compilers are welcome.


    I guess gcc is the only choice.. If you don't mind that it takes
    forever to compile C++ source code, until version 3.4 is released (it
    will supports pre-compiled header).
    d2003xx, Sep 22, 2003
    #7
  8. rhmd

    Guest

    d2003xx wrote:
    >
    > > I use Windows XP and 98. Have very little experience with C++. Am
    > > doing statistical and other simulations that require billions of
    > > calculations (taking 1 to 2 hours in Excel VBA). GCC is free, MS
    > > C++.Net is affordable

    >


    It is equally trivial to connect Python to Delphi using ctypes.
    Delphi can easily make a DLL that looks to the outside world like
    C. I have found Delphi to be maybe a little bit slower than MS C,
    but a little faster than gcc. They are all pretty close on speed,
    so you can't really pick a winner a priori. If you are doing VBA
    now, can't you buy the entry-level version of VB for around 100
    bucks and create C-callable DLL's?

    Al
    , Sep 22, 2003
    #8
  9. rhmd

    rhmd Guest

    Thanks to all who have responded. Your information was very helpful,
    I'm very impressed with the quality and thoughtfulness of the
    responses. Below find brief excerpts of some of the responses (which
    I hope will be useful for others who search for info on this topic),
    followed by more detail on my question.

    BEGIN EXCERPTS
    Have you considered and investigated Numeric Python or similar
    products?
    Look at the enthought Python installation (one is available for each
    of 2.2 and 2.3) -- they not only install a slew of the packages you
    might need for calculation heavy Python use, but (to support weave,
    I believe), they also include a mingw32 compiler (minimal gnu win32)
    which you can use for C++. If you get to the point where you are
    optimization-bound on your compiler (which you should determine by
    measuring, not intuiting), you could do worse than the Intel compiler.
    I suggest you look at various Python libraries to do whatever
    numerical
    processing you might need to do. They are mostly written in C anyway
    so
    their performance is excellent. For instance, I use SciPy for Fourier
    analysis and PIL for image processing, getting good performance on
    these
    numerically intensive tasks without ever leaving the ease and comfort
    of
    good old Python....The .NET compiler has a very good debugger and much
    better standard compliance than the old 6.0 compiler, but I haven't
    used lot of different recent vintage C++ compilers...I generally use
    COM objects written in Python to connect to Excel. I am very pleased
    with that technique.

    Koenig & Moo's book (Accelerated C++) is the best I've
    seen for [learning C++ for a few applications]..Python work with MSVC
    is much less of a pain for most things than gcc (aka mingw).
    Writing everything in Python then porting the 'hot spots' is often
    the
    best strategy. Use the profiler.
    Also (apart from Numeric / numarray), remember (or discover) Pyrex
    and
    psyco. Probably Pyrex is more useful to you than psyco. Don't apply
    either to your whole program. Just pick the bottlenecks. If you
    insist on using C++, make sure to look at Boost.Python(including
    pyste).
    gcc ... If you don't mind that it takes
    forever to compile C++ source code, until version 3.4 is released (it
    will support pre-compiled header).
    It is equally trivial to connect Python to Delphi using ctypes.
    Delphi can easily make a DLL that looks to the outside world like
    C. I have found Delphi to be maybe a little bit slower than MS C,
    but a little faster than gcc. They are all pretty close on speed,
    so you can't really pick a winner a priori. If you are doing VBA
    now, can't you buy the entry-level version of VB for around 100
    bucks and create C-callable DLL's?
    END EXCERPTS

    MORE DETAIL ON MY QUESTION
    VB.NET was another option I considered, have heard it produces
    binary that runs "almost" as fast as that produced by C++, but am not
    sure if that is true. VB would be very convenient for me, but can it
    produce DLLs that Python can use? The reason I was considering using
    Python/C++ combination is that I use Python already (as scripting
    language for the Blender animation program) and thought I could also
    use it with Excel to overcome certain Excel limitations, such as
    Excel's ridiculous limitations on chart markers (i.e., could make
    better charts with Python-- this would not be a time bottleneck so
    Python's slowness is not an issue there), etc. I also thought that,
    despite Python's reputation for being a bit slow, it would at least be
    faster than Excel VBA interpreter, but this may not be the case. At
    any rate, there are some calculations where I need a LOT of speed,
    e.g. generating billions (with a B) of random numbers with the
    Mersenne Twister algorithm (which is available in C++), for
    statistical problems (mulitple undefined boundary conditions which can
    only be extracted with a Monte Carlo method). For this, even doubling
    or quadrupling Python binary's speed would not be enough.
    rhmd, Sep 23, 2003
    #9
  10. rhmd wrote:


    >
    > MORE DETAIL ON MY QUESTION


    > At
    > any rate, there are some calculations where I need a LOT of speed,
    > e.g. generating billions (with a B) of random numbers with the
    > Mersenne Twister algorithm


    Off the top of my head I don't know which rng RandomArray (part of Numeric)
    uses, but its speed is not that bad:

    In [20]: def rloop(n,m):
    ....: random = RA.random
    ....: size=10**m
    ....: for i in xrange(10**n):
    ....: _=random(size)
    ....:

    In [21]: timing (1,rloop,1,5) # times are in cpu seconds
    Out[21]: 0.12000000000000455

    In [22]: timing (1,rloop,2,5)
    Out[22]: 1.1999999999999957

    In [23]: timing (1,rloop,3,5)
    Out[23]: 12.469999999999999


    The scaling is pretty linear, so for a billion terms you'd need about 2
    minutes.

    hth.

    Cheers,

    f
    Fernando Perez, Sep 25, 2003
    #10
  11. Fernando Perez wrote:
    ...
    >> e.g. generating billions (with a B) of random numbers with the
    >> Mersenne Twister algorithm

    >
    > Off the top of my head I don't know which rng RandomArray (part of
    > Numeric) uses, but its speed is not that bad:

    ...
    > The scaling is pretty linear, so for a billion terms you'd need about 2
    > minutes.


    Similar orders of magnitude for the built-in random number generator,
    which IS a Mersenne Twister in Python 2.3 -- at least on my old trusty
    Athlon box (no doubt today's CPU's would easily take 3 or 4 times less):

    [alex@lancelot python2.3]$ python timeit.py \
    > -s "from random import random" \
    > -s "times=[None]*1000"
    > "for x in times: random()"

    1000 loops, best of 3: 380 usec per loop

    380 microseconds for 1,000 calls to random.random(), and here, too,
    very linear scaling, suggests about 380 seconds for a billion calls,
    i.e., about 6 or 7 minutes, on this old machine.


    Alex
    Alex Martelli, Sep 25, 2003
    #11
  12. rhmd

    rhmd Guest

    Alex Martelli <> wrote in message news:<yFBcb.119348$>...
    >
    > Similar orders of magnitude for the built-in random number generator,
    > which IS a Mersenne Twister in Python 2.3 -- at least on my old trusty
    > Athlon box (no doubt today's CPU's would easily take 3 or 4 times less):
    >
    > [alex@lancelot python2.3]$ python timeit.py \
    > > -s "from random import random" \
    > > -s "times=[None]*1000"
    > > "for x in times: random()"

    > 1000 loops, best of 3: 380 usec per loop
    >
    > 380 microseconds for 1,000 calls to random.random(), and here, too,
    > very linear scaling, suggests about 380 seconds for a billion calls,
    > i.e., about 6 or 7 minutes, on this old machine.


    Wow! I didn't realize that 2.3 had upgraded to Mersenne Twister
    (which has a period of more than 10**10000, vs. WichmannHill which has
    a period of less than 10**10 and the Excel 2000 PNRG which (to my
    shock and chagrin) had a period of about 10**7). This is awesome.
    rhmd, Sep 28, 2003
    #12
  13. rhmd

    rhmd Guest

    There was an error in my previous message Wichmann Hill has a period
    of less than 10**14 (not 10**10)

    (rhmd) wrote in message news:<>...
    > Alex Martelli <> wrote in message news:<yFBcb.119348$>...
    > >
    > > Similar orders of magnitude for the built-in random number generator,
    > > which IS a Mersenne Twister in Python 2.3 -- at least on my old trusty
    > > Athlon box (no doubt today's CPU's would easily take 3 or 4 times less):
    > >
    > > [alex@lancelot python2.3]$ python timeit.py \
    > > > -s "from random import random" \
    > > > -s "times=[None]*1000"
    > > > "for x in times: random()"

    > > 1000 loops, best of 3: 380 usec per loop
    > >
    > > 380 microseconds for 1,000 calls to random.random(), and here, too,
    > > very linear scaling, suggests about 380 seconds for a billion calls,
    > > i.e., about 6 or 7 minutes, on this old machine.

    >
    > Wow! I didn't realize that 2.3 had upgraded to Mersenne Twister
    > (which has a period of more than 10**10000, vs. WichmannHill which has
    > a period of less than 10**10 and the Excel 2000 PNRG which (to my
    > shock and chagrin) had a period of about 10**7). This is awesome.
    rhmd, Oct 1, 2003
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Kelvin Chan

    please help me to choose a C++ book

    Kelvin Chan, Apr 8, 2004, in forum: C++
    Replies:
    5
    Views:
    349
    Thomas Matthews
    Apr 8, 2004
  2. No Spam
    Replies:
    7
    Views:
    547
    Micah Cowan
    Jan 4, 2005
  3. Nirjhar Oberoi

    Need to choose a free compiler

    Nirjhar Oberoi, Nov 16, 2006, in forum: C Programming
    Replies:
    16
    Views:
    450
    Chris Hills
    Nov 30, 2006
  4. Jason Doucette
    Replies:
    15
    Views:
    446
    Jason Doucette
    Jul 27, 2007
  5. Johannes Bauer
    Replies:
    4
    Views:
    485
    Öö Tiib
    Mar 2, 2011
Loading...

Share This Page