Literate Programming

[Hans Georg Schaathun]

Has anyone found a good system for literate programming in python?

I have been trying to use pylit/sphinx/pdflatex to generate
technical documentation. The application is scientific/numerical
programming, so discussing maths in maths syntax in between
python syntax is important.

While I like the style, there are several things which do not
work as they should.

One problem is that reST strips the indentation of code. When
documentation intersperses nested blocks, it would look better
if indentation was preserved so that the semantics of the code
is clear in the documentation. Are there any tricks to achieve
this in sphinx, or other systems which get it right?

Another problem is the the syntax highlighting sometimes get
confused. Most of the time, keywords are highlighted, but
sometimes they don't. For instance, documentation between
if and else in a conditional, seems to prevent sphinx from
recognising else. I also have some problems with 'def´ not
being recognised, where I have absolutely no clue as to why.
Are there any solutions to this?

Third problem, when I use automethod to get the docstring
prettyprinted, it would be neat if the verbatim docstring
definition did not appear as well. Any hints?

I am not dead set on keeping pylit/sphinx, although it would
be good to minimise the amount of doc code requiring rewriting.
The most important thing is to get a working system where I
could write a tutorial explaining both the python syntax and
the maths of a problem completely unambiguously.

TIA

 

[jkn]

Without fully answering your question ... I suggest you have a look at
Leo

http://webpages.charter.net/edreamleo/front.html

and ask your question at the (google) groups page devoted to that
editor.

http://groups.google.com/group/leo-editor

HTH
J^n

 

[Robert Kern]

Has anyone found a good system for literate programming in python?

I have been trying to use pylit/sphinx/pdflatex to generate
technical documentation. The application is scientific/numerical
programming, so discussing maths in maths syntax in between
python syntax is important.

While I like the style, there are several things which do not
work as they should.

One problem is that reST strips the indentation of code. When
documentation intersperses nested blocks, it would look better
if indentation was preserved so that the semantics of the code
is clear in the documentation. Are there any tricks to achieve
this in sphinx, or other systems which get it right?

http://sphinx.pocoo.org/markup/code.html

As far as I can tell, it just works. See here for an example:

http://ipython.scipy.org/doc/nightly/html/interactive/reference.html

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Hans Georg Schaathun]

http://sphinx.pocoo.org/markup/code.html
:
: As far as I can tell, it just works. See here for an example:
:
: http://ipython.scipy.org/doc/nightly/html/interactive/reference.html

Maybe I did not express myself clearly. I don't have a problem with
highlighting or indentation within a single, complete and continuous
block of code.
I get trouble when I insert explaining text within nested blocks,
especially when I do it at different levels of indentation.
In the pages you cite, I cannot find a single example which tries to
do this.

So something like this is fine:

<code>
# This is a long and complex block::

if mytest():
for i in myList():
foobar(i)
else:
for i in yourList():
boofar(i)
</code>

If I do the following, sphinx cannot keep up ...

<code>
# This block should be explained step by step::

if mytest():

# Bla, blah blah...
#
# ::

for i in myList():

# The :func:`foobar` function is an interesting choice here ... blah
#
# ::

foobar(i)

# Otherwise, myList might not be defined, so we need yours::

else:

# More blah...
#
# ::

for i in yourList():

# And here we go again::

boofar(i)
</code>

Highlighting tends to break starting from `else', and indentation breaks
at the second or third level.

 

[Jim]

Has anyone found a good system for literate programming in python?

Are you aware of pyweb http://sourceforge.net/projects/pywebtool/ ?

 

[Tim Arnold]

Has anyone found a good system for literate programming in python?

I have been trying to use pylit/sphinx/pdflatex to generate
technical documentation. The application is scientific/numerical
programming, so discussing maths in maths syntax in between
python syntax is important.

<snip>

Hi Hans,
If you already know LaTeX, you might experiment with the *.dtx docstrip
capability.
It has some pain points if you're developing from scratch, but I use it once
I've got a system in reasonable shape. You have full control over the
display and you can make the code files go anywhere you like when you run
pdflatex on your file.
--Tim Arnold

 

[Hans Georg Schaathun]

> Has anyone found a good system for literate programming in python?
:
: Are you aware of pyweb http://sourceforge.net/projects/pywebtool/ ?

Interesting tool, but it solves only part of the problem.
I could use it as a replacement for pylit, but I would then still
have the problem of commenting code within a block, which is a
reST/sphinx problem.

Alternatively, I could use pyweb directly with LaTeX. However, then
I would need to find or create macro packages which provide the
features of reST directly in LaTeX. Do you know of a good candidate?

 

[Jim]

Interesting tool, but it solves only part of the problem.
I could use it as a replacement for pylit, but I would then still
have the problem of commenting code within a block, which is a
reST/sphinx problem.

I'm sorry; I don't understand "commenting code within a block" but I
wondered if it meant you were not fully familiar with the idea of the
web-type programs. Instead of looking to typeset the comments, you
uses chunks. Thus a <<main>> chunk may be something like (pyweb has
somewhat different syntax)
<<main>>
def main(argv=None,log=None):
<<read command line argument>>
<<handle options>>
<<run driver routine>>
<<report results>>
Before each chunk comes the description of what that chunk does.
Something like
This routine factors $n$ finding any factors that are powers of a
prime number.
<<run driver routine>>
def driver(n,opt1,opt2):
<<find square root>>
<<factor>>
is a rough idea (here $n$ is a LaTeX; you can do HTML or RST). So you
are commenting the chunks, which can be blocks of the code.

Alternatively, I could use pyweb directly with LaTeX.  However, then
I would need to find or create macro packages which provide the
features of reST directly in LaTeX.  Do you know of a good candidate?

What features?

Jim

 

[Hans Georg Schaathun]

I'm sorry; I don't understand "commenting code within a block" but I
: wondered if it meant you were not fully familiar with the idea of the
: web-type programs.

The idea was pretty clear from the web page you cited. The web system
allows merging code and doc's within one file, but it does not provide
a markup language. Hence I need to find a markup language with suitable
markup to go with it.

: Before each chunk comes the description of what that chunk does.

Sure, and reST does this fine if every chunk is at the same level of
indentation, but if I split an indented block with a comment, reST
does not preserve the correct indentation. Similarily, if the else
clause is in a different chunk from the corresponding if clause,
sphinx looses track and will not highlight consistently. I posted an
example of what I wanted to do in reply to Robert Kern's post.

: > Alternatively, I could use pyweb directly with LaTeX.  However, then
: > I would need to find or create macro packages which provide the
: > features of reST directly in LaTeX.  Do you know of a good candidate?
:
: What features?

Standardised and well thought-out markup for functions/methods/classes etc.,
as well as highlighting.

 

[Hans Georg Schaathun]

If you already know LaTeX, you might experiment with the *.dtx docstrip
: capability.

Hi. Hmmm. That's a new thought. I never thought of using docstrip
with anything but LaTeX. It sounds like a rather primitive tool for
handling python code, and I would expect some serious trouble getting
sensible highlighting in vim/eclim (or most other editors for that
matter). But I'll give it a thought. Thanks.

: It has some pain points if you're developing from scratch, but I use it once
: I've got a system in reasonable shape.

Hmmm. I wonder if I am every going to reach that stage

: You have full control over the
: display and you can make the code files go anywhere you like when you run
: pdflatex on your file.

If you use docstrip with python, what packages do you use to highlight
code and markup programming concepts (methods/classes/variables)?
If I may ask ...

 

[Tim Arnold]

If you already know LaTeX, you might experiment with the *.dtx docstrip
: capability.

Hi. Hmmm. That's a new thought. I never thought of using docstrip
with anything but LaTeX. It sounds like a rather primitive tool for
handling python code, and I would expect some serious trouble getting
sensible highlighting in vim/eclim (or most other editors for that
matter). But I'll give it a thought. Thanks.

: It has some pain points if you're developing from scratch, but I use it
once
: I've got a system in reasonable shape.

Hmmm. I wonder if I am every going to reach that stage

: You have full control over the
: display and you can make the code files go anywhere you like when you
run
: pdflatex on your file.

If you use docstrip with python, what packages do you use to highlight
code and markup programming concepts (methods/classes/variables)?
If I may ask ...

I don't use anything special, just the verbatim environment is fine for my
purposes. But you might like the listings package which iirc has syntax
highlighting built-in for python. ah, yes:
http://en.wikibooks.org/wiki/LaTeX/Packages/Listings

There's also the 'fancyvrb' package:
http://www.ctan.org/tex-archive/macros/latex/contrib/fancyvrb

good luck,
--Tim