RedCloth 3.0.2 -- Please, oh please, let this be the one

[why the lucky stiff]

Okay, okay. The tumblers are clicking, the clouds are parting.

And, yes, the doves are lighting on the handle of my cane.

## RedCloth 3.0.2 ######
Humane Text for Ruby

RedCloth is a simple text format. You write *little* _symbols_ and
"links to":http://something.com things. Follows Textile rules and a
portion of Markdown. More Markdown is forthcoming.

RedCloth is getting better. I'm not getting stack overflow regexps
anymore. All code blocks protected from formatting. Hard breaks
working. Filter HTML now uses detailed cleaner.

If you've been using RedCloth 2 and you want to upgrade, keep in mind
that RedCloth 3 ignores newlines used to wrap paragraphs. In RedCloth
2, these newlines were turned into `br' tags. That caused confusion, so
it's no longer default. To enable hard breaks, use RedCloth#hard_breaks.

Okay, let's get this release out and tested. I desperately need to move
on to PDF, Docbook, LaTeX output.

I'm so glad this release is out. I can let up my concerned look. What
a relief.

_why

 

[Alexey Verkhovsky]

Okay, okay. The tumblers are clicking, the clouds are parting.

Guess what did you forget to include in your announcement?

--
Best regards,

Alexey Verkhovsky

Ruby Forum: http://ruby-forum.org (moderator)
RForum: http://rforum.andreas-s.net (co-author)
Instiki: http://instiki.org (maintainer)

 

[Alexey Verkhovsky]

I'm so glad this release is out. I can let up my concerned look.
What a relief.

I'm sorry, but... still no go. But looks close.

I have a regression test case for RedCloth, which is, basically,
importing all markup of rubyonrails.com/rails into Instiki (SVN trunk
has an import feature). RedCloth 3.0.2 fails to process some 20 pages,
saying: "too big quantifier in {,}: /^ {32767}\S/"

Stack trace points to redcloth.rb:937.

E.g., this page: http://wiki.rubyonrails.com/rails/show/StoryCards

There is also one page that both 3.0.2 and 2.0.11 fail to process,
namely
http://wiki.rubyonrails.com/rails/rollback/IRC Logs in Color?rev=0

There you can see the "stack trace too deep" in regexp still (but that
didn't work since some time, so not so serious).

--
Best regards,

Alexey Verkhovsky

Ruby Forum: http://ruby-forum.org (moderator)
RForum: http://rforum.andreas-s.net (co-author)
Instiki: http://instiki.org (maintainer)

 

[Bil Kleb]

## RedCloth 3.0.2 ######

RedCloth is getting better. I'm not getting stack overflow regexps
anymore.

Confirmed on my end: I just refreshed http://fun3d.larc.nasa.gov
by using 3.0.2.

Thanks!

If you've been using RedCloth 2 and you want to upgrade, keep in mind
that RedCloth 3 ignores newlines used to wrap paragraphs.

Welcome change: for the 2-series, I had written a pre-filter to
deal with the old, default behavior. Jamis: you no longer need
to have a single line for each paragraph.

I desperately need to move on to PDF, Docbook, LaTeX output.

I'm 19 tests into a (hack) LaTeX port of 3.0.1. I'll patch to 3.0.2.
Would this be helpful?

Thanks again,

 

[why the lucky stiff]

I've been wondering whether RedCloth is being stretched in too many
directions?

On the one hand the use of gsubs! and big old regexps makes Redcloth
very powerful and great for clever textile commands for writing
beautiful books like _why's, but on the other they are really hard to
get right for the 'big pages, but mostly trivial markup' of wikis.

Yeah, totally. But I think I just need more tests.

In the examples given by Alexey, RedCloth is given IRC logs. My tests
have been limited to text that can be split into paragraphs.

I had a go at re-implementing textile to html conversion as a left to
right parser using the Strscan library (code below or at
http://www2.eng.cam.ac.uk/~tamc2/TextileParser.rb). This, I think,
manages the conversions mentioned in http://hobix.com/textile/ and
doesn't stack overflow on big pages, but does not do all the amazing
extra stuff that RedCloth actually does. The downside, and the
reason I haven't pursued it is that not only does it do less, it is
also much much slower (should have been obvious in advance I guess).

Hey, that's cool! I'm going to play with this.

I've been tinkering with making a C version (SuperRedCloth) as well,
using a similar technique. I could really use some extra speed.

PS The tabs and spaces, and the order of the attribute values in html,
are different in TextileParser compared to Redcloth, so lots of the
Redcloth tests fail on this even though they have the same syntactic
meaning, others fail because TextileParser doesn't implement beyond
http://hobix.com/textile/, and others just fail of course.

Go ahead and eliminate extra newlines and leading tabs when you're
testing. It's not worthwhile to count out all the newlines in your output.

_why

 

[why the lucky stiff]

I'm 19 tests into a (hack) LaTeX port of 3.0.1. I'll patch to 3.0.2.
Would this be helpful?

Oh, come on. You're dangling free code out there?? Seriously, Bill.
Gimme.

_why

 

[Alexey Verkhovsky]

Yeah, totally. But I think I just need more tests.

In the examples given by Alexey, RedCloth is given IRC logs.

Just one of them - and that one doesn't work with RedCloth 2.0.11, either.

As for the rest of them, I looked at all those pages - and it's a simple
bug:

irb(main):005:0> r = RedCloth.new("\n")
=> "\n"
irb(main):006:0> r.to_html
RegexpError: too big quantifier in {,}: /^ {32767}\S/
from ./redcloth.rb:937:in `flush_left'
from ./redcloth.rb:932:in `clean_white_space'
from ./redcloth.rb:270:in `to_html'
from (irb):6
irb(main):007:0>

--
Best regards,

Alexey Verkhovsky

Ruby Forum: http://ruby-forum.org (moderator)
RForum: http://rforum.andreas-s.net (co-author)
Instiki: http://instiki.org (maintainer)

 

[Bil Kleb]

Oh, come on. You're dangling free code out there?? Seriously, Bill.
Gimme.

(delivered 3.0.2 to_LaTeX tests + hack offline)

Later,

 

[Tobias Luetke]

Can't wait for this. I have been meaning to hack this myself for
Hieraki becuase PDF export is the most requested feature for Hieraki
and LaTeX seems to be the best way to get there.

 

[Nicholas Van Weerdenburg]

Can't wait for this. I have been meaning to hack this myself for
Hieraki becuase PDF export is the most requested feature for Hieraki
and LaTeX seems to be the best way to get there.




--
Tobi
http://www.snowdevil.ca - Snowboards which don't suck
http://www.hieraki.org - Open source book authoring
http://blog.leetsoft.com - Technical weblog

I'm curious- I keep hearing about LaTex -> PDF, but never
XSL-FO/FOP->PDF. Is there a reason for that?

Thanks,
Nick

 

[Bil Kleb]

I'm curious- I keep hearing about LaTex -> PDF, but never
XSL-FO/FOP->PDF. Is there a reason for that?

I don't understand the later and I need math typesetting.
Am I being aggressively incompetent? Should I study
XSL-FO/FOP?

Regards,

 

[David A. Black]

Hi --

I'm curious- I keep hearing about LaTex -> PDF, but never
XSL-FO/FOP->PDF. Is there a reason for that?

I've done some work, along those lines, on a Redcloth -> DocBook
converter, mainly (but not only) with an eye to converting Hieraki
books. It is, at the moment, pretty much an in-place hack of the
to_html converter, which is actually a fairly effective way to
bootstrap it, though I'd like to finish and polish it in a really
DocBook-specific way.

Anyone else who wants to should feel free to have a go -- it's in the
RedCloth CVS repository, at
http://rubyforge.org/cgi-bin/viewcvs.cgi/redcloth/lib/rctodb.rb?cvsroot=redcloth.


David

 

[bdarcus]

I don't understand the later and I need math typesetting.
Am I being aggressively incompetent? Should I study
XSL-FO/FOP?

This is one of those semi-religious issues.

XSL-FO is a more conceptually-elegant approach to go from XML to PDF
and printed output because it's all XML-based (with everything that
goes along with that, like unicode support). LaTeX is in many ways a
hack.

However, the quality of output from FO processors does not yet really
match the quality one can get from TeX (if setup right, which means
knowing how to customize it, using different fonts and such). LaTeX
particularly excels if you do math.

One interesting development of late is that Hans Hagen--author of the
ConTeXt macro package (sort of a more modern alternative to LaTeX)--has
been working on adding FO support to ConTeXt. This may provide the best
of boths worlds: XML-based toolchain + TeX quality.

http://www.pragma-ade.com/general/magazines/mag-0008.pdf
http://www.ntg.nl/mailman/listinfo/foxet

So, I'd say you should take a look at FO (and maybe foxet) and see for
yourself if it's worth exploring further.

BTW, I author in DocBook, and use XSLT to generate XHTML and PDF. To
generate the latter I first go to LaTeX. This will likely change in the
future though as FO tools like foxet mature (and I have more time to
experiment!).

Bruce

 

[bdarcus]

I've done some work, along those lines, on a Redcloth -> DocBook
converter, mainly (but not only) with an eye to converting Hieraki
books. It is, at the moment, pretty much an in-place hack of the
to_html converter, which is actually a fairly effective way to
bootstrap it, though I'd like to finish and polish it in a really
DocBook-specific way.

Cool. If you can, please consider the following:

1) Support v4.4 of the DTD (just released). It adds a new
citation-related element called biblioref that is important for
citations (I'm a scholar, and have been talking to the author of
PyTextile about adding support in Textile for this sort of markup).

http://netapps.muohio.edu/blogs/darcusb/darcusb/archives/2005/02/01/textile-citations-and-favelets

2) Design it in such a way that it's easy to move to v5, which is the
nextgeneration RELAX NG-based version currently in development. The
changes are not severe, and mostly involve simplifying and
rationalizing DocBook (for example, articleinfo and chapterinfo just
become db:info), as well as namespacing it.

3) Don't assume DTD-based toolchains (I'm not saying you are; it's
just that many people do). Doctype declarations, for example, should be
optional.

BTW, shouldn't RedCloth have a driver system for different output
modes? It's an innocent question; I have no background in Ruby (more
XSLT).

Bruce