Literate Programming

Discussion in 'Python' started by Hans Georg Schaathun, Apr 7, 2011.

  1. Has anyone found a good system for literate programming in python?

    I have been trying to use pylit/sphinx/pdflatex to generate
    technical documentation. The application is scientific/numerical
    programming, so discussing maths in maths syntax in between
    python syntax is important.

    While I like the style, there are several things which do not
    work as they should.

    One problem is that reST strips the indentation of code. When
    documentation intersperses nested blocks, it would look better
    if indentation was preserved so that the semantics of the code
    is clear in the documentation. Are there any tricks to achieve
    this in sphinx, or other systems which get it right?

    Another problem is the the syntax highlighting sometimes get
    confused. Most of the time, keywords are highlighted, but
    sometimes they don't. For instance, documentation between
    if and else in a conditional, seems to prevent sphinx from
    recognising else. I also have some problems with 'def´ not
    being recognised, where I have absolutely no clue as to why.
    Are there any solutions to this?

    Third problem, when I use automethod to get the docstring
    prettyprinted, it would be neat if the verbatim docstring
    definition did not appear as well. Any hints?

    I am not dead set on keeping pylit/sphinx, although it would
    be good to minimise the amount of doc code requiring rewriting.
    The most important thing is to get a working system where I
    could write a tutorial explaining both the python syntax and
    the maths of a problem completely unambiguously.

    TIA
    --
    :-- Hans Georg
    Hans Georg Schaathun, Apr 7, 2011
    #1
    1. Advertising

  2. Hans Georg Schaathun

    jkn Guest

    jkn, Apr 7, 2011
    #2
    1. Advertising

  3. Hans Georg Schaathun

    Robert Kern Guest

    On 4/7/11 1:09 PM, Hans Georg Schaathun wrote:
    > Has anyone found a good system for literate programming in python?
    >
    > I have been trying to use pylit/sphinx/pdflatex to generate
    > technical documentation. The application is scientific/numerical
    > programming, so discussing maths in maths syntax in between
    > python syntax is important.
    >
    > While I like the style, there are several things which do not
    > work as they should.
    >
    > One problem is that reST strips the indentation of code. When
    > documentation intersperses nested blocks, it would look better
    > if indentation was preserved so that the semantics of the code
    > is clear in the documentation. Are there any tricks to achieve
    > this in sphinx, or other systems which get it right?


    http://sphinx.pocoo.org/markup/code.html

    As far as I can tell, it just works. See here for an example:

    http://ipython.scipy.org/doc/nightly/html/interactive/reference.html

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
    Robert Kern, Apr 7, 2011
    #3
  4. On Thu, 07 Apr 2011 16:21:52 -0500, Robert Kern
    <> wrote:
    : http://sphinx.pocoo.org/markup/code.html
    :
    : As far as I can tell, it just works. See here for an example:
    :
    : http://ipython.scipy.org/doc/nightly/html/interactive/reference.html

    Maybe I did not express myself clearly. I don't have a problem with
    highlighting or indentation within a single, complete and continuous
    block of code.
    I get trouble when I insert explaining text within nested blocks,
    especially when I do it at different levels of indentation.
    In the pages you cite, I cannot find a single example which tries to
    do this.

    So something like this is fine:

    <code>
    # This is a long and complex block::

    if mytest():
    for i in myList():
    foobar(i)
    else:
    for i in yourList():
    boofar(i)
    </code>

    If I do the following, sphinx cannot keep up ...

    <code>
    # This block should be explained step by step::

    if mytest():

    # Bla, blah blah...
    #
    # ::

    for i in myList():

    # The :func:`foobar` function is an interesting choice here ... blah
    #
    # ::

    foobar(i)

    # Otherwise, myList might not be defined, so we need yours::

    else:

    # More blah...
    #
    # ::

    for i in yourList():

    # And here we go again::

    boofar(i)
    </code>

    Highlighting tends to break starting from `else', and indentation breaks
    at the second or third level.

    --
    :-- Hans Georg
    Hans Georg Schaathun, Apr 8, 2011
    #4
  5. Hans Georg Schaathun

    Jim Guest

    Jim, Apr 8, 2011
    #5
  6. Hans Georg Schaathun

    Tim Arnold Guest

    "Hans Georg Schaathun" <> wrote in message
    news:...
    > Has anyone found a good system for literate programming in python?
    >
    > I have been trying to use pylit/sphinx/pdflatex to generate
    > technical documentation. The application is scientific/numerical
    > programming, so discussing maths in maths syntax in between
    > python syntax is important.
    >

    <snip>

    Hi Hans,
    If you already know LaTeX, you might experiment with the *.dtx docstrip
    capability.
    It has some pain points if you're developing from scratch, but I use it once
    I've got a system in reasonable shape. You have full control over the
    display and you can make the code files go anywhere you like when you run
    pdflatex on your file.
    --Tim Arnold
    Tim Arnold, Apr 8, 2011
    #6
  7. On Fri, 8 Apr 2011 05:22:01 -0700 (PDT), Jim
    <> wrote:
    : On Apr 7, 2:09 pm, Hans Georg Schaathun <> wrote:
    : > Has anyone found a good system for literate programming in python?
    :
    : Are you aware of pyweb http://sourceforge.net/projects/pywebtool/ ?

    Interesting tool, but it solves only part of the problem.
    I could use it as a replacement for pylit, but I would then still
    have the problem of commenting code within a block, which is a
    reST/sphinx problem.

    Alternatively, I could use pyweb directly with LaTeX. However, then
    I would need to find or create macro packages which provide the
    features of reST directly in LaTeX. Do you know of a good candidate?

    --
    :-- Hans Georg
    Hans Georg Schaathun, Apr 8, 2011
    #7
  8. Hans Georg Schaathun

    Jim Guest

    On Apr 8, 3:21 pm, Hans Georg Schaathun <> wrote:
    > Interesting tool, but it solves only part of the problem.
    > I could use it as a replacement for pylit, but I would then still
    > have the problem of commenting code within a block, which is a
    > reST/sphinx problem.


    I'm sorry; I don't understand "commenting code within a block" but I
    wondered if it meant you were not fully familiar with the idea of the
    web-type programs. Instead of looking to typeset the comments, you
    uses chunks. Thus a <<main>> chunk may be something like (pyweb has
    somewhat different syntax)
    <<main>>
    def main(argv=None,log=None):
    <<read command line argument>>
    <<handle options>>
    <<run driver routine>>
    <<report results>>
    Before each chunk comes the description of what that chunk does.
    Something like
    This routine factors $n$ finding any factors that are powers of a
    prime number.
    <<run driver routine>>
    def driver(n,opt1,opt2):
    <<find square root>>
    <<factor>>
    is a rough idea (here $n$ is a LaTeX; you can do HTML or RST). So you
    are commenting the chunks, which can be blocks of the code.

    > Alternatively, I could use pyweb directly with LaTeX.  However, then
    > I would need to find or create macro packages which provide the
    > features of reST directly in LaTeX.  Do you know of a good candidate?


    What features?

    Jim
    Jim, Apr 9, 2011
    #8
  9. On Sat, 9 Apr 2011 03:45:55 -0700 (PDT), Jim
    <> wrote:
    : I'm sorry; I don't understand "commenting code within a block" but I
    : wondered if it meant you were not fully familiar with the idea of the
    : web-type programs.

    The idea was pretty clear from the web page you cited. The web system
    allows merging code and doc's within one file, but it does not provide
    a markup language. Hence I need to find a markup language with suitable
    markup to go with it.

    : Before each chunk comes the description of what that chunk does.

    Sure, and reST does this fine if every chunk is at the same level of
    indentation, but if I split an indented block with a comment, reST
    does not preserve the correct indentation. Similarily, if the else
    clause is in a different chunk from the corresponding if clause,
    sphinx looses track and will not highlight consistently. I posted an
    example of what I wanted to do in reply to Robert Kern's post.

    : > Alternatively, I could use pyweb directly with LaTeX.  However, then
    : > I would need to find or create macro packages which provide the
    : > features of reST directly in LaTeX.  Do you know of a good candidate?
    :
    : What features?

    Standardised and well thought-out markup for functions/methods/classes etc.,
    as well as highlighting.

    --
    :-- Hans Georg
    Hans Georg Schaathun, Apr 9, 2011
    #9
  10. On Fri, 8 Apr 2011 12:58:34 -0400, Tim Arnold
    <> wrote:
    : If you already know LaTeX, you might experiment with the *.dtx docstrip
    : capability.

    Hi. Hmmm. That's a new thought. I never thought of using docstrip
    with anything but LaTeX. It sounds like a rather primitive tool for
    handling python code, and I would expect some serious trouble getting
    sensible highlighting in vim/eclim (or most other editors for that
    matter). But I'll give it a thought. Thanks.

    : It has some pain points if you're developing from scratch, but I use it once
    : I've got a system in reasonable shape.

    Hmmm. I wonder if I am every going to reach that stage :)

    : You have full control over the
    : display and you can make the code files go anywhere you like when you run
    : pdflatex on your file.

    If you use docstrip with python, what packages do you use to highlight
    code and markup programming concepts (methods/classes/variables)?
    If I may ask ...

    --
    :-- Hans Georg
    Hans Georg Schaathun, Apr 9, 2011
    #10
  11. Hans Georg Schaathun

    Tim Arnold Guest

    "Hans Georg Schaathun" <> wrote in message
    news:...
    > On Fri, 8 Apr 2011 12:58:34 -0400, Tim Arnold
    > <> wrote:
    > : If you already know LaTeX, you might experiment with the *.dtx docstrip
    > : capability.
    >
    > Hi. Hmmm. That's a new thought. I never thought of using docstrip
    > with anything but LaTeX. It sounds like a rather primitive tool for
    > handling python code, and I would expect some serious trouble getting
    > sensible highlighting in vim/eclim (or most other editors for that
    > matter). But I'll give it a thought. Thanks.
    >
    > : It has some pain points if you're developing from scratch, but I use it
    > once
    > : I've got a system in reasonable shape.
    >
    > Hmmm. I wonder if I am every going to reach that stage :)
    >
    > : You have full control over the
    > : display and you can make the code files go anywhere you like when you
    > run
    > : pdflatex on your file.
    >
    > If you use docstrip with python, what packages do you use to highlight
    > code and markup programming concepts (methods/classes/variables)?
    > If I may ask ...
    >
    > --
    > :-- Hans Georg


    I don't use anything special, just the verbatim environment is fine for my
    purposes. But you might like the listings package which iirc has syntax
    highlighting built-in for python. ah, yes:
    http://en.wikibooks.org/wiki/LaTeX/Packages/Listings

    There's also the 'fancyvrb' package:
    http://www.ctan.org/tex-archive/macros/latex/contrib/fancyvrb

    good luck,
    --Tim
    Tim Arnold, Apr 11, 2011
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mike Maxwell

    XMLMind and Literate Programming

    Mike Maxwell, Nov 2, 2004, in forum: XML
    Replies:
    2
    Views:
    358
    Mike Maxwell
    Nov 3, 2004
  2. How to literate a const char*

    , Jul 14, 2007, in forum: C Programming
    Replies:
    2
    Views:
    318
    Christopher Benson-Manica
    Jul 15, 2007
  3. Paul Miller

    Literate programs in Python

    Paul Miller, May 13, 2008, in forum: Python
    Replies:
    6
    Views:
    291
    Marc 'BlackJack' Rintsch
    May 14, 2008
  4. Massimiliano Mirra - bard

    Literate testing?

    Massimiliano Mirra - bard, Aug 27, 2004, in forum: Ruby
    Replies:
    6
    Views:
    104
    Mauricio Fernández
    Aug 30, 2004
  5. Ruby Quiz

    [QUIZ] Literate Ruby (#102)

    Ruby Quiz, Nov 17, 2006, in forum: Ruby
    Replies:
    10
    Views:
    184
    James Edward Gray II
    Nov 22, 2006
Loading...

Share This Page