does it matter whether References headers are folded?

[Nomen Nescio]

RFC 1036 says the message-IDs in the References header should be
separated by a space, but it's quite common to see them folded
(separated by a newline and some spaces or a newline and a tab).

Does it matter?

Is either folded or unfolded better or worse than the other?

 

[Dr.Ruud]

Nomen Nescio schreef:

RFC 1036 says the message-IDs in the References header should be
separated by a space

s/space/blank/

<quote>
2. Message Format
[...]
A standard USENET message consists of several header lines, followed
by a blank line, followed by the body of the message. Each header
line consist of a keyword, a colon, a blank, and some additional
information. [...] The Internet convention of
continuation header lines (beginning with a blank or tab) is
allowed.
</quote>

 

[Nomen Nescio]

Nomen Nescio schreef:

RFC 1036 says the message-IDs in the References header should be
separated by a space

s/space/blank/

<quote>
2. Message Format
[...]
A standard USENET message consists of several header lines, followed
by a blank line, followed by the body of the message. Each header
line consist of a keyword, a colon, a blank, and some additional
information. [...] The Internet convention of
continuation header lines (beginning with a blank or tab) is
allowed.
</quote>

Right, but are the two formats absolutely equal or is one
considerd a little better than the other?

 

[Dr.Ruud]

Nomen Nescio schreef:

Dr.Ruud:

Nomen Nescio:

RFC 1036 says the message-IDs in the References header should be
separated by a space

s/space/blank/

<quote>
2. Message Format
[...]
A standard USENET message consists of several header lines,
followed by a blank line, followed by the body of the message.
Each header line consist of a keyword, a colon, a blank, and
some additional information. [...] The Internet convention of
continuation header lines (beginning with a blank or tab) is
allowed.
</quote>

Right, but are the two formats absolutely equal or is one
considerd a little better than the other?

Equal. Just don't put a fold directly after the "Keyword:". It is OK to
put each Message-ID on it's own line, but the first (the oldest) must be
on the same line as the "References: ".

You are even allowed to remove Message-IDs from the References header
field (to limit the total (unfolded) length), but certainly don't remove
the one-or-two oldest and newest ones (they are used for the "threaded
view" by many clients).

 

[Nomen Nescio]

RFC 1036 says the message-IDs in the References header should be
separated by a space, but it's quite common to see them folded
(separated by a newline and some spaces or a newline and a tab).

Technically speaking, after a header reaches a certain length it *must*
be folded to prevent risk of breaking some clients/transports. And
either the space or tab method is acceptable.

=46rom an RFC point of view it doesn't matter.

=46rom an anonymity POV it might (you cross posted to APAS).

 

[Anonymous]

RFC 1036 says the message-IDs in the References header should be
separated by a space, but it's quite common to see them folded
(separated by a newline and some spaces or a newline and a tab).

Does it matter?

Is either folded or unfolded better or worse than the other?

It shouldn't matter, but there are times when it matters to some home-grown
applications. For example I've found that the BigApple mail2news software,
currently run by Bikikii I think, barfs on folded headers by only including
the first line. Other times we've seen non-folded headers truncated by a
limit on line length.

What to do.

 

[Curt Welch]

It shouldn't matter, but there are times when it matters to some
home-grown applications. For example I've found that the BigApple
mail2news software, currently run by Bikikii I think, barfs on folded
headers by only including the first line. Other times we've seen
non-folded headers truncated by a limit on line length.

What to do.

References: tend not to be folded and are intelligently truncated to 998
bytes by removing Message-IDs starting with the second on the line
(always leaving the first and as many of the last ones as possible).
Technically, you should be able to fold them but there was a time in the
past that it was felt it would break too many things if you folded the
References: line. I tend to suspect that's not an big issue anymore. The
998 limit was because some news software (INN at the time?) didn't allow
you to post lines longer than that. That was a limit after the folding
was removed so you couldn't "fix" the line length problem with folding.

I have no idea what INN and all the other news servers currently allow for
line lengths and whether the 998 limit is important anymore.

However, References: are one of the fields that are part of XOVER and
keeping References: lines more than 1000 bytes in length I think is just
wasteful at best since all the Message-IDs except the first and last are
generally redundant and of minor importance. So I think it is probably
wise to continue to use a truncation algorithm.

And, if I was writing a new reader, I wouldn't fold References: lines -
just because there's no advantage to doing it that I know of and a slight
risk it would cause an issue.

Here's some more thoughts about References: lines from the GNKSA:

http://www.xs4all.nl/~js/gnksa/gnksa.txt

7) Make sure followups contain valid References

When creating a followup, the software MUST create a "References: "
header line that contains, as its last element, the Message-ID of the
original article. An individual Message-ID MUST never be truncated.

The software MUST include at least three additional Message-IDs from
the original article's References header as well, if they are available.
Try to stay as close as possible to the spirit of "son-of-1036", which
states:

<<Followup agents SHOULD not shorten References headers. If
it is absolutely necessary to shorten the header, as a des-
perate last resort, a followup agent MAY do this by deleting
some of the message IDs. However, it MUST not delete the
first message ID, the last three message IDs (including that
of the immediate precursor), or any message ID mentioned in
the body of the followup.>>

However, it also says:

<<If it is absolutely necessary for an implementation to
impose a limit on the length of header lines, body lines, or
header logical lines, that limit shall be at least 1000
octets, including EOL representations.>>

So, bear in mind that news transports are not guaranteed to be able to
handle arbitrary long lines. Furthermore, keep in mind that some news
transports choke on continued (multi-line) "References: " headers.

Therefore, keep as many Message-IDs as will fit on a line starting with
"References: " with a maximum length of 998 characters. (An octet is a
byte of 8 bits, EOL representation takes two bytes.)

Exception: Damaged (truncated) Message-IDs SHOULD NOT be included.
Neither should `bogus' Message-IDs -- IDs that somehow got inserted (by
a misguided user, maybe) but don't belong to the thread.

Rationale: Threaded news-readers depend on References to do their magic.
This too is basic RFC compliance. Be as complete as the line length
limit allows, but do not propagate errors.

 

[Dr.Ruud]

Curt Welch schreef:

References: tend not to be folded and are intelligently truncated to
998 bytes by removing Message-IDs starting with the second on the
line (always leaving the first and as many of the last ones as
possible). Technically, you should be able to fold them but there
was a time in the past that it was felt it would break too many
things if you folded the References: line. I tend to suspect that's
not an big issue anymore. The 998 limit was because some news
software (INN at the time?) didn't allow you to post lines longer
than that. That was a limit after the folding was removed so you
couldn't "fix" the line length problem with folding.

I have no idea what INN and all the other news servers currently
allow for line lengths and whether the 998 limit is important anymore.

However, References: are one of the fields that are part of XOVER and
keeping References: lines more than 1000 bytes in length I think is
just wasteful at best since all the Message-IDs except the first and
last are generally redundant and of minor importance. So I think it
is probably wise to continue to use a truncation algorithm.

And, if I was writing a new reader, I wouldn't fold References: lines
- just because there's no advantage to doing it that I know of and a
slight risk it would cause an issue.

Here's some more thoughts about References: lines from the GNKSA:

http://www.xs4all.nl/~js/gnksa/gnksa.txt

7) Make sure followups contain valid References

When creating a followup, the software MUST create a "References: "
header line that contains, as its last element, the Message-ID of the
original article. An individual Message-ID MUST never be truncated.

The software MUST include at least three additional Message-IDs from
the original article's References header as well, if they are
available. Try to stay as close as possible to the spirit of
"son-of-1036", which states:

<<Followup agents SHOULD not shorten References headers. If
it is absolutely necessary to shorten the header, as a des-
perate last resort, a followup agent MAY do this by deleting
some of the message IDs. However, it MUST not delete the
first message ID, the last three message IDs (including that
of the immediate precursor), or any message ID mentioned in
the body of the followup.>>

However, it also says:

<<If it is absolutely necessary for an implementation to
impose a limit on the length of header lines, body lines, or
header logical lines, that limit shall be at least 1000
octets, including EOL representations.>>

So, bear in mind that news transports are not guaranteed to be able to
handle arbitrary long lines. Furthermore, keep in mind that some news
transports choke on continued (multi-line) "References: " headers.

Therefore, keep as many Message-IDs as will fit on a line starting
with "References: " with a maximum length of 998 characters. (An
octet is a byte of 8 bits, EOL representation takes two bytes.)

Exception: Damaged (truncated) Message-IDs SHOULD NOT be included.
Neither should `bogus' Message-IDs -- IDs that somehow got inserted
(by
a misguided user, maybe) but don't belong to the thread.

Rationale: Threaded news-readers depend on References to do their
magic. This too is basic RFC compliance. Be as complete as the line
length
limit allows, but do not propagate errors.

 

[Dr.Ruud]

Curt Welch schreef:

[son-of-1036]
<<Followup agents SHOULD not shorten References headers. If
it is absolutely necessary to shorten the header, as a des-
perate last resort, a followup agent MAY do this by deleting
some of the message IDs. However, it MUST not delete the
first message ID, the last three message IDs (including that
of the immediate precursor), or any message ID mentioned in
the body of the followup.>>

There is one (very theoretical) catch in there: a <message-id> can be
that long that a References: header field with four of them wouldn't fit
in 998 characters. But then you would just drop one more (the oldest of
the tail).

 

[J Moreno]

Curt Welch schreef:

[son-of-1036]
<<Followup agents SHOULD not shorten References headers. If
it is absolutely necessary to shorten the header, as a des-
perate last resort, a followup agent MAY do this by deleting
some of the message IDs. However, it MUST not delete the
first message ID, the last three message IDs (including that
of the immediate precursor), or any message ID mentioned in
the body of the followup.>>

There is one (very theoretical) catch in there: a <message-id> can be
that long that a References: header field with four of them wouldn't fit
in 998 characters. But then you would just drop one more (the oldest of
the tail).

Hmn, I'm not sure that's actually true -- or at least I'm not sure it
should be true if people play by the rules. There's a limit on the MID
because of a NNTP command, IMS it also should prevent any four MID from
exceeding the 998 limit.

Of course, not everyone plays by the rules, so you're right...still,
rather unlikely.

 

[BiKiKii]

It shouldn't matter, but there are times when it matters to some
home-grown applications. For example I've found that the BigApple
mail2news software, currently run by Bikikii I think, barfs on folded
headers by only including the first line. Other times we've seen
non-folded headers truncated by a limit on line length.

What to do.

BiKiKii M2N routine includes BA posting software.
Was not aware of folding References issue.
Will look into fix for folded Ref header within BiKiKii's M@N routine.

...apparently she is in need of swift kick (kickstart) too

BiKiKii

 

[BiKiKii]

BiKiKii M2N routine includes BA posting software.
Was not aware of folding References issue.
Will look into fix for folded Ref header within BiKiKii's M@N routine.

...apparently she is in need of swift kick (kickstart) too

Please send test messages to BiKiKii M2N which have folded Ref Header
need raw data to do some fix of routine

BiKiKii