does it matter whether References headers are folded?

Discussion in 'Perl Misc' started by Nomen Nescio, Aug 7, 2006.

  1. Nomen Nescio

    Nomen Nescio Guest

    RFC 1036 says the message-IDs in the References header should be
    separated by a space, but it's quite common to see them folded
    (separated by a newline and some spaces or a newline and a tab).

    Does it matter?

    Is either folded or unfolded better or worse than the other?
    Nomen Nescio, Aug 7, 2006
    #1
    1. Advertising

  2. Nomen Nescio

    Dr.Ruud Guest

    Nomen Nescio schreef:

    > RFC 1036 says the message-IDs in the References header should be
    > separated by a space


    s/space/blank/

    <quote>
    2. Message Format
    [...]
    A standard USENET message consists of several header lines, followed
    by a blank line, followed by the body of the message. Each header
    line consist of a keyword, a colon, a blank, and some additional
    information. [...] The Internet convention of
    continuation header lines (beginning with a blank or tab) is
    allowed.
    </quote>

    --
    Affijn, Ruud

    "Gewoon is een tijger."
    Dr.Ruud, Aug 7, 2006
    #2
    1. Advertising

  3. Nomen Nescio

    Nomen Nescio Guest

    "Dr.Ruud" <> wrote:

    > Nomen Nescio schreef:
    >
    > > RFC 1036 says the message-IDs in the References header should be
    > > separated by a space

    >
    > s/space/blank/
    >
    > <quote>
    > 2. Message Format
    > [...]
    > A standard USENET message consists of several header lines, followed
    > by a blank line, followed by the body of the message. Each header
    > line consist of a keyword, a colon, a blank, and some additional
    > information. [...] The Internet convention of
    > continuation header lines (beginning with a blank or tab) is
    > allowed.
    > </quote>


    Right, but are the two formats absolutely equal or is one
    considerd a little better than the other?
    Nomen Nescio, Aug 7, 2006
    #3
  4. Nomen Nescio

    Dr.Ruud Guest

    Nomen Nescio schreef:
    > Dr.Ruud:
    >> Nomen Nescio:


    >>> RFC 1036 says the message-IDs in the References header should be
    >>> separated by a space

    >>
    >> s/space/blank/
    >>
    >> <quote>
    >> 2. Message Format
    >> [...]
    >> A standard USENET message consists of several header lines,
    >> followed by a blank line, followed by the body of the message.
    >> Each header line consist of a keyword, a colon, a blank, and
    >> some additional information. [...] The Internet convention of
    >> continuation header lines (beginning with a blank or tab) is
    >> allowed.
    >> </quote>

    >
    > Right, but are the two formats absolutely equal or is one
    > considerd a little better than the other?


    Equal. Just don't put a fold directly after the "Keyword:". It is OK to
    put each Message-ID on it's own line, but the first (the oldest) must be
    on the same line as the "References: ".

    You are even allowed to remove Message-IDs from the References header
    field (to limit the total (unfolded) length), but certainly don't remove
    the one-or-two oldest and newest ones (they are used for the "threaded
    view" by many clients).

    --
    Affijn, Ruud

    "Gewoon is een tijger."
    Dr.Ruud, Aug 7, 2006
    #4
  5. Nomen Nescio

    Nomen Nescio Guest

    Nomen Nescio wrote:

    > RFC 1036 says the message-IDs in the References header should be
    > separated by a space, but it's quite common to see them folded
    > (separated by a newline and some spaces or a newline and a tab).


    Technically speaking, after a header reaches a certain length it *must*
    be folded to prevent risk of breaking some clients/transports. And
    either the space or tab method is acceptable.

    =46rom an RFC point of view it doesn't matter.

    =46rom an anonymity POV it might (you cross posted to APAS).
    Nomen Nescio, Aug 7, 2006
    #5
  6. Nomen Nescio

    Anonymous Guest

    In article <>
    Nomen Nescio <> wrote:
    >
    > RFC 1036 says the message-IDs in the References header should be
    > separated by a space, but it's quite common to see them folded
    > (separated by a newline and some spaces or a newline and a tab).
    >
    > Does it matter?
    >
    > Is either folded or unfolded better or worse than the other?


    It shouldn't matter, but there are times when it matters to some home-grown
    applications. For example I've found that the BigApple mail2news software,
    currently run by Bikikii I think, barfs on folded headers by only including
    the first line. Other times we've seen non-folded headers truncated by a
    limit on line length.

    What to do.
    Anonymous, Aug 7, 2006
    #6
  7. Nomen Nescio

    Curt Welch Guest

    Anonymous <> wrote:
    > In article <>
    > Nomen Nescio <> wrote:
    > >
    > > RFC 1036 says the message-IDs in the References header should be
    > > separated by a space, but it's quite common to see them folded
    > > (separated by a newline and some spaces or a newline and a tab).
    > >
    > > Does it matter?
    > >
    > > Is either folded or unfolded better or worse than the other?

    >
    > It shouldn't matter, but there are times when it matters to some
    > home-grown applications. For example I've found that the BigApple
    > mail2news software, currently run by Bikikii I think, barfs on folded
    > headers by only including the first line. Other times we've seen
    > non-folded headers truncated by a limit on line length.
    >
    > What to do.


    References: tend not to be folded and are intelligently truncated to 998
    bytes by removing Message-IDs starting with the second on the line
    (always leaving the first and as many of the last ones as possible).
    Technically, you should be able to fold them but there was a time in the
    past that it was felt it would break too many things if you folded the
    References: line. I tend to suspect that's not an big issue anymore. The
    998 limit was because some news software (INN at the time?) didn't allow
    you to post lines longer than that. That was a limit after the folding
    was removed so you couldn't "fix" the line length problem with folding.

    I have no idea what INN and all the other news servers currently allow for
    line lengths and whether the 998 limit is important anymore.

    However, References: are one of the fields that are part of XOVER and
    keeping References: lines more than 1000 bytes in length I think is just
    wasteful at best since all the Message-IDs except the first and last are
    generally redundant and of minor importance. So I think it is probably
    wise to continue to use a truncation algorithm.

    And, if I was writing a new reader, I wouldn't fold References: lines -
    just because there's no advantage to doing it that I know of and a slight
    risk it would cause an issue.

    Here's some more thoughts about References: lines from the GNKSA:

    http://www.xs4all.nl/~js/gnksa/gnksa.txt

    7) Make sure followups contain valid References

    When creating a followup, the software MUST create a "References: "
    header line that contains, as its last element, the Message-ID of the
    original article. An individual Message-ID MUST never be truncated.

    The software MUST include at least three additional Message-IDs from
    the original article's References header as well, if they are available.
    Try to stay as close as possible to the spirit of "son-of-1036", which
    states:

    <<Followup agents SHOULD not shorten References headers. If
    it is absolutely necessary to shorten the header, as a des-
    perate last resort, a followup agent MAY do this by deleting
    some of the message IDs. However, it MUST not delete the
    first message ID, the last three message IDs (including that
    of the immediate precursor), or any message ID mentioned in
    the body of the followup.>>

    However, it also says:

    <<If it is absolutely necessary for an implementation to
    impose a limit on the length of header lines, body lines, or
    header logical lines, that limit shall be at least 1000
    octets, including EOL representations.>>

    So, bear in mind that news transports are not guaranteed to be able to
    handle arbitrary long lines. Furthermore, keep in mind that some news
    transports choke on continued (multi-line) "References: " headers.

    Therefore, keep as many Message-IDs as will fit on a line starting with
    "References: " with a maximum length of 998 characters. (An octet is a
    byte of 8 bits, EOL representation takes two bytes.)

    Exception: Damaged (truncated) Message-IDs SHOULD NOT be included.
    Neither should `bogus' Message-IDs -- IDs that somehow got inserted (by
    a misguided user, maybe) but don't belong to the thread.

    Rationale: Threaded news-readers depend on References to do their magic.
    This too is basic RFC compliance. Be as complete as the line length
    limit allows, but do not propagate errors.

    --
    Curt Welch http://CurtWelch.Com/
    http://NewsReader.Com/
    Curt Welch, Aug 7, 2006
    #7
  8. Nomen Nescio

    Dr.Ruud Guest

    Curt Welch schreef:
    > Anonymous <> wrote:
    >> In article <>
    >> Nomen Nescio <> wrote:
    >>>
    >>> RFC 1036 says the message-IDs in the References header should be
    >>> separated by a space, but it's quite common to see them folded
    >>> (separated by a newline and some spaces or a newline and a tab).
    >>>
    >>> Does it matter?
    >>>
    >>> Is either folded or unfolded better or worse than the other?

    >>
    >> It shouldn't matter, but there are times when it matters to some
    >> home-grown applications. For example I've found that the BigApple
    >> mail2news software, currently run by Bikikii I think, barfs on folded
    >> headers by only including the first line. Other times we've seen
    >> non-folded headers truncated by a limit on line length.
    >>
    >> What to do.

    >
    > References: tend not to be folded and are intelligently truncated to
    > 998 bytes by removing Message-IDs starting with the second on the
    > line (always leaving the first and as many of the last ones as
    > possible). Technically, you should be able to fold them but there
    > was a time in the past that it was felt it would break too many
    > things if you folded the References: line. I tend to suspect that's
    > not an big issue anymore. The 998 limit was because some news
    > software (INN at the time?) didn't allow you to post lines longer
    > than that. That was a limit after the folding was removed so you
    > couldn't "fix" the line length problem with folding.
    >
    > I have no idea what INN and all the other news servers currently
    > allow for line lengths and whether the 998 limit is important anymore.
    >
    > However, References: are one of the fields that are part of XOVER and
    > keeping References: lines more than 1000 bytes in length I think is
    > just wasteful at best since all the Message-IDs except the first and
    > last are generally redundant and of minor importance. So I think it
    > is probably wise to continue to use a truncation algorithm.
    >
    > And, if I was writing a new reader, I wouldn't fold References: lines
    > - just because there's no advantage to doing it that I know of and a
    > slight risk it would cause an issue.
    >
    > Here's some more thoughts about References: lines from the GNKSA:
    >
    > http://www.xs4all.nl/~js/gnksa/gnksa.txt
    >
    > 7) Make sure followups contain valid References
    >
    > When creating a followup, the software MUST create a "References: "
    > header line that contains, as its last element, the Message-ID of the
    > original article. An individual Message-ID MUST never be truncated.
    >
    > The software MUST include at least three additional Message-IDs from
    > the original article's References header as well, if they are
    > available. Try to stay as close as possible to the spirit of
    > "son-of-1036", which states:
    >
    > <<Followup agents SHOULD not shorten References headers. If
    > it is absolutely necessary to shorten the header, as a des-
    > perate last resort, a followup agent MAY do this by deleting
    > some of the message IDs. However, it MUST not delete the
    > first message ID, the last three message IDs (including that
    > of the immediate precursor), or any message ID mentioned in
    > the body of the followup.>>
    >
    > However, it also says:
    >
    > <<If it is absolutely necessary for an implementation to
    > impose a limit on the length of header lines, body lines, or
    > header logical lines, that limit shall be at least 1000
    > octets, including EOL representations.>>
    >
    > So, bear in mind that news transports are not guaranteed to be able to
    > handle arbitrary long lines. Furthermore, keep in mind that some news
    > transports choke on continued (multi-line) "References: " headers.
    >
    > Therefore, keep as many Message-IDs as will fit on a line starting
    > with "References: " with a maximum length of 998 characters. (An
    > octet is a byte of 8 bits, EOL representation takes two bytes.)
    >
    > Exception: Damaged (truncated) Message-IDs SHOULD NOT be included.
    > Neither should `bogus' Message-IDs -- IDs that somehow got inserted
    > (by
    > a misguided user, maybe) but don't belong to the thread.
    >
    > Rationale: Threaded news-readers depend on References to do their
    > magic. This too is basic RFC compliance. Be as complete as the line
    > length
    > limit allows, but do not propagate errors.




    --
    Affijn, Ruud

    "Gewoon is een tijger."
    Dr.Ruud, Aug 7, 2006
    #8
  9. Nomen Nescio

    Dr.Ruud Guest

    Curt Welch schreef:

    > [son-of-1036]
    > <<Followup agents SHOULD not shorten References headers. If
    > it is absolutely necessary to shorten the header, as a des-
    > perate last resort, a followup agent MAY do this by deleting
    > some of the message IDs. However, it MUST not delete the
    > first message ID, the last three message IDs (including that
    > of the immediate precursor), or any message ID mentioned in
    > the body of the followup.>>


    There is one (very theoretical) catch in there: a <message-id> can be
    that long that a References: header field with four of them wouldn't fit
    in 998 characters. But then you would just drop one more (the oldest of
    the tail).

    --
    Affijn, Ruud

    "Gewoon is een tijger."
    Dr.Ruud, Aug 7, 2006
    #9
  10. Nomen Nescio

    J Moreno Guest

    Dr.Ruud <> wrote:

    > Curt Welch schreef:
    >
    > > [son-of-1036]
    > > <<Followup agents SHOULD not shorten References headers. If
    > > it is absolutely necessary to shorten the header, as a des-
    > > perate last resort, a followup agent MAY do this by deleting
    > > some of the message IDs. However, it MUST not delete the
    > > first message ID, the last three message IDs (including that
    > > of the immediate precursor), or any message ID mentioned in
    > > the body of the followup.>>

    >
    > There is one (very theoretical) catch in there: a <message-id> can be
    > that long that a References: header field with four of them wouldn't fit
    > in 998 characters. But then you would just drop one more (the oldest of
    > the tail).


    Hmn, I'm not sure that's actually true -- or at least I'm not sure it
    should be true if people play by the rules. There's a limit on the MID
    because of a NNTP command, IMS it also should prevent any four MID from
    exceeding the 998 limit.

    Of course, not everyone plays by the rules, so you're right...still,
    rather unlikely.

    --
    JM
    "Everything is futile." -- Marvin of Borg
    J Moreno, Aug 8, 2006
    #10
  11. Nomen Nescio

    BiKiKii Guest

    Anonymous <> wrote in
    news::

    > In article <>
    > Nomen Nescio <> wrote:
    >>
    >> RFC 1036 says the message-IDs in the References header should be
    >> separated by a space, but it's quite common to see them folded
    >> (separated by a newline and some spaces or a newline and a tab).
    >>
    >> Does it matter?
    >>
    >> Is either folded or unfolded better or worse than the other?

    >
    > It shouldn't matter, but there are times when it matters to some
    > home-grown applications. For example I've found that the BigApple
    > mail2news software, currently run by Bikikii I think, barfs on folded
    > headers by only including the first line. Other times we've seen
    > non-folded headers truncated by a limit on line length.
    >
    > What to do.
    >
    >


    BiKiKii M2N routine includes BA posting software.
    Was not aware of folding References issue.
    Will look into fix for folded Ref header within BiKiKii's M@N routine.

    ...apparently she is in need of swift kick (kickstart) too

    BiKiKii
    BiKiKii, Sep 25, 2006
    #11
  12. Nomen Nescio

    BiKiKii Guest

    On 25 Sep 2006, BiKiKii wrote:
    >Anonymous <> wrote in
    >news::
    >
    >> In article <>
    >> Nomen Nescio <> wrote:
    >>>
    >>> RFC 1036 says the message-IDs in the References header should be
    >>> separated by a space, but it's quite common to see them folded
    >>> (separated by a newline and some spaces or a newline and a tab).
    >>>
    >>> Does it matter?
    >>>
    >>> Is either folded or unfolded better or worse than the other?

    >>
    >> It shouldn't matter, but there are times when it matters to some
    >> home-grown applications. For example I've found that the BigApple
    >> mail2news software, currently run by Bikikii I think, barfs on folded
    >> headers by only including the first line. Other times we've seen
    >> non-folded headers truncated by a limit on line length.
    >>
    >> What to do.
    >>
    >>

    >
    >BiKiKii M2N routine includes BA posting software.
    >Was not aware of folding References issue.
    >Will look into fix for folded Ref header within BiKiKii's M@N routine.
    >
    > ...apparently she is in need of swift kick (kickstart) too
    >



    Please send test messages to BiKiKii M2N which have folded Ref Header
    need raw data to do some fix of routine

    BiKiKii
    BiKiKii, Sep 26, 2006
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Roy

    Does it really matter?

    Roy, Jul 12, 2005, in forum: ASP .Net
    Replies:
    8
    Views:
    432
    Robert Smith
    Jul 16, 2005
  2. John

    Does domain still matter?

    John, Jun 29, 2003, in forum: Java
    Replies:
    2
    Views:
    845
    Shripathi Kamath
    Jun 29, 2003
  3. dont bother
    Replies:
    0
    Views:
    759
    dont bother
    Mar 3, 2004
  4. Phil
    Replies:
    4
    Views:
    654
    Gabriel Genellina
    Jan 17, 2010
  5. Ian
    Replies:
    2
    Views:
    1,901
Loading...

Share This Page