Converting HTML elements into XML/RSS

[mickjames]

Hi,

I'd like to include the whole web page content (as opposed to just the
headlines) into RSS/XML to enable people to read them via rss feed
readers.

Question: how to convert HTML elements such as href, img, b, p, etc
into XML?
I've seen someone use the following in their RSS feed but I don't like
it because <pre> doesn't produce a nice format:

<content:encoded><![CDATA[
<PRE>
blah blah blah..

Here is a sample HTML code. What would be the best way to put it into
XML, more specifically, convert those HTML elements.

----------------
<b>CAESAR</b> Et tu, Brute! Then fall,
<a
href=http://www.epilepsiemuseum.de/raum6/caesar.jpg>Caesar</a>.<br>
Dies
<p>
<b>CINNA</b> Liberty! Freedom! Tyranny is dead!
Run hence, proclaim, cry it about the streets.
<a href=http://www.shakespeare-online.com/>Read more</a>.

 

[Andy Dingley]

I'd like to include the whole web page content (as opposed to just the
headlines) into RSS/XML to enable people to read them via rss feed
readers.

Read this
http://diveintomark.org/archives/2004/02/04/incompatible-rss

Ask again if anything is unclear.

 

[mickjames]

Thanks.

So all the HTML needs to be enclosed in <description> and tags need to
be escaped with &amp;lt; and &amp;gt;?

 

[Nick Kew]

I'd like to include the whole web page content (as opposed to just the
headlines) into RSS/XML to enable people to read them via rss feed
readers.

Uh, that's a lot of content for what users are expecting to be a summary.
Why use a feed if it doesn't save your users anything?

Question: how to convert HTML elements such as href, img, b, p, etc
into XML?

Bearing in mind the above, freely mix it, just using namespaces to
distinguish the elements. Since you're already breaking the purpose
of a feed, working normally with conventional client software presumably
isn't an issue.

Here is a sample HTML code. What would be the best way to put it into

Looks more like tag-soup to me.

 

[mickjames]

Thanks for your reply. Yes, I understand that RSS is meant for summary,
not the whole content, but a lot of readers ask for the whole thing.
One imagines, they prefer to read using an rss feed reader instead of
using a web browser.

One question I didn't get the answer to in all my searching is: how to
code HTML tags such as href, img, p, b, etc when converting an HTML
page to .rss page?

Putting everything in CDATA or is there a better way?
A short example would be helpful.

Thanks a lot!

 

[Andy Dingley]

So all the HTML needs to be enclosed in <description> and tags need to
be escaped with &amp;lt; and &amp;gt;?

Yes. Ampersands might also cause problems and should already have been
escaped, but it's common in HTML that they aren't.

You should also "fix" any entitity references that are in the HTML,
such as &eacute; or &nbsp; This needs to be done whether there are
tags involved or not - they're one of the most common intermittent
reasons for an RSS feed to become invalid. Such entities are defined
in HTML, but aren't already defined in XML or RSS.

"Fixing" them can be either replacing the initial ampersand with &amp;
or replacing the "named" form of the entity reference with the
corresponding numeric form. The numeric form is probably best to use,
because that will render correctly even if the consumer doesn't
properly expand the encoded entities.

 

[Andy Dingley]

Why use a feed if it doesn't save your users anything?

Why do you assume the function of my RSS feed ? I've built many
feeds that are anything but "newsfeeds". I think my record was 20MB
content size in a <description> element, for a very
application-specific intranet task. However it's still perfectly
compliant RSS 1.0

Bearing in mind the above, freely mix it, just using namespaces to
distinguish the elements.

You can't use namespacing, because the content is HTML rather than
XHTML. Apart from the standards-based argument and the fact that
namespacing just doesn't make sense for HTML, it's also impractical to
expect the incoming HTML content to be well-formed as an XML fragment
(or even valid HTML!).

Remember that RSS is a _feed_, not a one-off document (I wish Winer
would recognise this). Like all layered protocols you have to be very
careful that your implementations are not only correct for one
demonstration example, they have to be demonstrably correct for all
possible inputs.

Since you're already breaking the purpose of a feed,

Rubbish. RSS does _NOT_ define any notion of "purpose", or what's
"appropriate" to use it for. Besides which, the notion of content
encoding HTML fragments within the <description> element is very well
established.

 

[Nick Kew]

One imagines, they prefer to read using an rss feed reader instead of
using a web browser.

Hmmm. I think it should be the job of the Client to present it
sensibly. An RSS feed is to the Web as a newsgroup or mail folder
listing (from, subject, date) is to Usenet or Email. IMHO.

(you've presumably seen how Opera presents RSS feeds?)

One question I didn't get the answer to in all my searching is: how to
code HTML tags such as href, img, p, b, etc when converting an HTML
page to .rss page?

The core Site Valet tools offer options to present reports as RDF.
Since these are markup analysis tools, the more verbose options
embed the original markup, so all system messages can be properly
referenced to it. This uses a namespace to describe it, and
looks a little like XSLT with things like:
<ml:element name="a">

Putting everything in CDATA or is there a better way?
A short example would be helpful.

I don't think the above reply is really relevant to your question:
I was solving a different problem! But you already have Andy's reply.

 

[Colin]

Hey,

I'd like to include the whole web page content (as opposed to just the
headlines) into RSS/XML to enable people to read them via rss feed
readers.

Question: how to convert HTML elements such as href, img, b, p, etc
into XML?

Why don't you just use software to create the feed that will convert it for you
so that you don't have to worry about it. There are a couple of options, I know
FeedForAll http://www.feedforall.com has a WYSWIG editor that will do this.

Best,
Colin

 

[mickjames]

WYSIWIG is not an option. I need to do it via script on Linux.

Would someone tell me how the following HTML snippet should be encoded
in an RSS file:

<b>This is a test.</a>
<a href=foo.html>Bar</a>.
<img src=baz.jpg>
<p>

I tried using &amp;lt; etc but RSS readers simply display the
equivalent HTML, rather then rendering it.

 

[Henri Sivonen]

Here is a sample HTML code. What would be the best way to put it into

Looks more like tag-soup to me.[/QUOTE]

"Entity-encoded HTML" *is* tag soup transported over XML character data.
To make things worse, RSS provides no way of communicating whether the
characters reported by the XML processor are presentable text or tag
soup source that needs another level of parsing.

To make matters still worse, the problem has propagated from RSS 0.92
and 2.0 descriptions to titles and even to RSS 0.91 and RSS 1.0
processing, even though there is no spec text supporting "entity-encoded
HTML" in titles in any version of RSS or in descriptions in RSS 0.91 and
RSS 1.0.

For example, Sage misrenders the title "Tag Soup: How Mac IE 5 and
Safari handle <x> <y> </x> </y>" in http://www.hut.fi/u/hsivonen/feed.xml

 

[Andy Dingley]

To make matters still worse, the problem has propagated from RSS 0.92
and 2.0 descriptions to titles and even to RSS 0.91 and RSS 1.0
processing,

No, RSS 1.0 is clear over this - although the others do have a
problem. The RSS 1.0 spec wasn't written in the sloppy manner of the
others.

 

[Henri Sivonen]

No, RSS 1.0 is clear over this - although the others do have a
problem. The RSS 1.0 spec wasn't written in the sloppy manner of the
others.

My point was that the problem has propagated to RSS 1.0 *processing*.
That is, there's software that assumes "entity-escaped HTML" in RSS 1.0
*titles*, even though there is no spec text to back it up.

 

[mickjames]

So can anyone show me how to put this HTML fragment into RSS/XML?

 

[Nick Kew]

Why do you assume the function of my RSS feed ? I've built many

I don't. I made an inference from the wording of the OP.

You can't use namespacing, because the content is HTML rather than
XHTML.

Nonsense. Just map the HTML trivially to XHTML.

Apart from the standards-based argument and the fact that
namespacing just doesn't make sense for HTML, it's also impractical to
expect the incoming HTML content to be well-formed as an XML fragment
(or even valid HTML!).

Not necessary. There's no shortage of software that'll parse HTML
and XHTML to the same representation or event stream.

Remember that RSS is a _feed_, not a one-off document (I wish Winer
would recognise this). Like all layered protocols you have to be very
careful that your implementations are not only correct for one
demonstration example, they have to be demonstrably correct for all
possible inputs.

Yes, and?

Rubbish. RSS does _NOT_ define any notion of "purpose", or what's
"appropriate" to use it for.

Erm, I read the OP as implying a conventional/familiar purpose. What
in the references to "web page content", "rss *feed* readers", or the
ugly-tagsoup-html sample, leads you to suppose otherwise?

 

[thufir.hawat]

Not necessary. There's no shortage of software that'll parse HTML
and XHTML to the same representation or event stream.

[..]

what's meant by "event stream," please?


thanks,

Thufir Hawat

 

[Nick Kew]

[..]

Not necessary. There's no shortage of software that'll parse HTML
and XHTML to the same representation or event stream.

[..]

what's meant by "event stream," please?

Google for SAX.

 

[Venkata Srinivasulu]

hai