Gateway is malfunctioning

T

Trans

Someone recently mentioned missing posts. I just did a comparsion of
comp.lang.ruby and ruby-talk-google and indeed the usenet is missing
posts.

James you want to have a look at the gateway?

T.
 
M

Michael W. Ryder

Trans said:
Someone recently mentioned missing posts. I just did a comparsion of
comp.lang.ruby and ruby-talk-google and indeed the usenet is missing
posts.

James you want to have a look at the gateway?

T.
Click to expand...
I think someone mentioned earlier that the missing posts were mostly
those with HTML in them. Why the HTML posts are filtered out I do not know.
 
D

Damphyr

Michael said:
I think someone mentioned earlier that the missing posts were mostly
those with HTML in them. Why the HTML posts are filtered out I do not
know.
Click to expand...
Not true. I had some missing posts (a couple misdated and a couple not)
and I only send text.
Cheers,
V.-
 
A

ara.t.howard

Someone recently mentioned missing posts. I just did a comparsion of
comp.lang.ruby and ruby-talk-google and indeed the usenet is missing posts.

James you want to have a look at the gateway?
Click to expand...

as far as i can tell it always is - or at least has been for weeks.

can we write a script to compare on a regular basis? a monitor?

are you in charge of this james?

cheers.

-a
 
J

James Edward Gray II

Someone recently mentioned missing posts. I just did a comparsion of
comp.lang.ruby and ruby-talk-google and indeed the usenet is missing
posts.
Click to expand...

Here's the information I have on this currently:

* I rewrote the entire Gateway this last week, so anything before
that is history. The news-to-email side is already deployed. My
email-to-news code is complete, but not deployed. (Our SysAdmin will
get to it soon now.) If you raise issues, please make sure they are
recent.
* I now have very detailed logs on everything the Gateway does, so
please be specific. For example, tell me which messages are missing,
not just that they are.
* Our Usenet host does not allow us to post HTML emails. This is not
changing. (I'm not bothered by this because I do not believe anyone
should be sending HTML email to Ruby Talk.)
* Our Gateway is an NNTP <-> email Gateway. There has been at least
one instance of a Usenet post using ancient header formatting
predating NNTP. These messages are not supported.
James you want to have a look at the gateway?
Click to expand...

I will address individual issues, as long as they are aware of all
four points above.

James Edward Gray II
 
J

James Edward Gray II

Not true. I had some missing posts (a couple misdated and a couple
not) and I only send text.
Click to expand...

Your posts appeared on one side of the Gateway (the Ruby Talk mailing
list or the Usenet group comp.lang.ruby), but did not reach the other
side? If so, please lead me to the posts on the side where they did
appear.

James Edward Gray II
 
A

ara.t.howard

* I rewrote the entire Gateway this last week, so anything before that is
history. The news-to-email side is already deployed. My email-to-news code
is complete, but not deployed. (Our SysAdmin will get to it soon now.) If
you raise issues, please make sure they are recent.
Click to expand...

mine aren't - so ignore for now.
* I now have very detailed logs on everything the Gateway does, so please be
specific. For example, tell me which messages are missing, not just that
they are.
Click to expand...

can't we automate this? monitor/scraper?

* Our Usenet host does not allow us to post HTML emails. This is not
changing. (I'm not bothered by this because I do not believe anyone should
be sending HTML email to Ruby Talk.)
Click to expand...

harp:~ > echo '<html><body><b>__html_2_text?__</b></body></html>' | links -dump -stdin
__html_2_text?__

??
* Our Gateway is an NNTP <-> email Gateway. There has been at least one
instance of a Usenet post using ancient header formatting predating NNTP.
These messages are not supported.
check.


I will address individual issues, as long as they are aware of all four
points above.
Click to expand...

so... i have this rash... ;-)

-a
 
J

James Edward Gray II

can't we automate this? monitor/scraper?
Click to expand...

Probably not too meaningfully. Every Usenet host uses a different
system for numbering posts, so the numbers wouldn't be much help to
the outside world.

Besides, now that I understand the Gateway code intimately, I'm
hoping to get it robust enough to make problems rare.
harp:~ > echo '<html><body><b>__html_2_text?__</b></body></html>'
| links -dump -stdin
__html_2_text?__

??
Click to expand...

I would much rather Ruby Talk rejected HTML emails. ;)

I think they are pure evil, so it will be hard to convince me to
spend much effort on them.
so... i have this rash... ;-)
Click to expand...

<dies laughing>

OK... Gateway issues!

James Edward Gray II
 
T

Trans

James said:
Here's the information I have on this currently:

* I rewrote the entire Gateway this last week, so anything before
that is history. The news-to-email side is already deployed. My
email-to-news code is complete, but not deployed. (Our SysAdmin will
get to it soon now.) If you raise issues, please make sure they are
recent.
* I now have very detailed logs on everything the Gateway does, so
please be specific. For example, tell me which messages are missing,
not just that they are.
* Our Usenet host does not allow us to post HTML emails. This is not
changing. (I'm not bothered by this because I do not believe anyone
should be sending HTML email to Ruby Talk.)
* Our Gateway is an NNTP <-> email Gateway. There has been at least
one instance of a Usenet post using ancient header formatting
predating NNTP. These messages are not supported.
Click to expand...

Have a look at recent topic: Wisdom of including Rakefile in releases.
The 2nd post from Jason Roelofs appears to be missing.

T.
 
R

Robert Klemme

Have a look at recent topic: Wisdom of including Rakefile in releases.
The 2nd post from Jason Roelofs appears to be missing.
Click to expand...

I do not see *any* posting of Jason on the news side and only a single
posting on the email side:

<quote>
Rakefiles allow users to run the tests for themselves, to understand how a
library is put together, etc. The only thing that can go wrong is that the
stuff doesn't work (unless you're grabbing very untrustworthy packages, but
assumptions have to be made somewhere). Not to mention, having the Rakefile
there allows other people to properly make changes to a library if / when
they need to.

I don't see anything bad with including said Rakefiles. Do you have any
specific worries?
</quote>

Cheers

robert
 
J

James Edward Gray II

I have not yet received this message from Trans. I do find it in the
mailing list archives though and I see it on comp.lang.ruby. I'll
assume my host blocked it, ruby talk is hiccuping, or something
because the Gateway logs show it was forwarded.
I do not see *any* posting of Jason on the news side and only a
single posting on the email side:
Click to expand...

Right. I'll assume we are talking about this message:

http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/227459

The log says:

<[email protected]> - Re: Wisdom of including Rakefile in releases
11/30/06-15:21:43: mail2news | nntp.post() result = 441 437 HTML post

Translation: Our NNTP host rejected the HTML post.

James Edward Gray II
 
R

Robert Klemme

I have not yet received this message from Trans. I do find it in the
mailing list archives though and I see it on comp.lang.ruby. I'll
assume my host blocked it, ruby talk is hiccuping, or something because
the Gateway logs show it was forwarded.


Right. I'll assume we are talking about this message:

http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/227459

The log says:

<[email protected]> - Re: Wisdom of including Rakefile in releases
11/30/06-15:21:43: mail2news | nntp.post() result = 441 437 HTML post

Translation: Our NNTP host rejected the HTML post.
Click to expand...

I cannot see that it is a HTML posting - in fact, I do not see any HTML
in that posting. Is it possible that the gateway rejects multipart
messages? James, if you need the source (or what reached my gmail box)
let me know and I forward it to you.

Kind regards

robert
 
J

James Edward Gray II

I cannot see that it is a HTML posting - in fact, I do not see any
HTML in that posting. Is it possible that the gateway rejects
multipart messages? James, if you need the source (or what reached
my gmail box) let me know and I forward it to you.
Click to expand...

Fred, can you comment on this? The link above shows pretty much how
the rejected message was formatted.

James Edward Gray II
 
F

F. Senault

Le 1 décembre 2006 à 23:30, James Edward Gray II a écrit :
On Dec 1, 2006, at 4:20 PM, Robert Klemme wrote:
Click to expand...

I'd like to see the whole raw message, headers included (my email is
valid, please attach the whole message, (g)zipped, or it will be bounced
by my _personal_ filters).
Fred, can you comment on this? The link above shows pretty much how
the rejected message was formatted.
Click to expand...

Are you sure the mailing-list archive software doesn't filter out HTML ?
A multipart message with only one part sounds quite unlikely to me.

James, maybe the gateway should save the rejected messages somewhere, if
only temporarily, don't you think ?

Anyway, I'm using cleanfeed on the server, an omnipresent filtering
package. It's usually quire reliable, even if it has to use some
complicated heuristics from time to time (since MIME isn't widely used
on Usenet). It's a quite standard version, with a bit of tweaking, but
not in that domain.

Why am I telling you all of this, you ask ? Well, even if it is a false
positive, if I tune my filters to let the message pass, it will anyway
have a really bad distribution, being rejected by most of the hosts that
run the same soft.
James Edward Gray II
Click to expand...

Fred
 
F

F. Senault

Le 2 décembre 2006 à 11:16, F. Senault a écrit :
Le 1 décembre 2006 à 23:30, James Edward Gray II a écrit :


I'd like to see the whole raw message, headers included (my email is
valid, please attach the whole message, (g)zipped, or it will be bounced
by my _personal_ filters).
Click to expand...

Ok, Robert sent me what seemed to be the full message, and it's in fact
a message declared with a MIME content-type of multipart/alternative but
only one part (text/plain). This is quite absurd, and I'd suspect that
something, somewhere, decided to strip away the HTML part but leave the
MIME declaration intact.

And, of course, cleanfeed declared the message as HTML on the basis of
the headers...

Fred
(Not sure of my English on this one...)
 
J

James Edward Gray II

Le 2 d=E9cembre 2006 =E0 11:16, F. Senault a =E9crit :


Ok, Robert sent me what seemed to be the full message, and it's in =20
fact
a message declared with a MIME content-type of multipart/=20
alternative but
only one part (text/plain). This is quite absurd, and I'd suspect =20
that
something, somewhere, decided to strip away the HTML part but leave =20=
Click to expand...
the
MIME declaration intact.
Click to expand...

Forgive my weak knowledge of email types here, but if someone sent a =20
short message to the list with an attached Ruby script, would it be a =20=

multipart message? If so, that would be rejected by the NNTP host?

James
 
R

Ross Bamford

Ok, Robert sent me what seemed to be the full message, and it's in =20
fact
a message declared with a MIME content-type of multipart/=20
alternative but
only one part (text/plain). This is quite absurd, and I'd suspect =20
that
something, somewhere, decided to strip away the HTML part but leave =20
the
MIME declaration intact.
Click to expand...
=20
Forgive my weak knowledge of email types here, but if someone sent a =20
short message to the list with an attached Ruby script, would it be a =20
multipart message? If so, that would be rejected by the NNTP host?[/QUOTE]

I have a feeling this is what is, or at least was, happening - I wasn't
going to say anything (since it was prior to December first) but I
noticed that posts with attachments never made it across.=20

ruby-talk 226884, for example, doesn't appear to have made it to the
newsgroup.

--=20
Ross Bamford - (e-mail address removed)
 
F

F. Senault

Le 2 décembre 2006 à 17:10, James Edward Gray II a écrit :
Forgive my weak knowledge of email types here, but if someone sent a
short message to the list with an attached Ruby script, would it be a
multipart message? If so, that would be rejected by the NNTP host?
Click to expand...

Yes, but it won't be multipart/alternative (which means different
representations of the same data, nearly universally HTML + plain text),
more likely multipart/mixed (simply different parts).

For instance, with a quick search, I found this one :

http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/227642

Which has been gated correctly, as it has :

Content-Type: multipart/mixed; boundary="sm4nu43k4a2Rpi4c"

(But it lacks the MIME-Version header, which is, AFAICR, mandatory, if
quite useless...)
James
Click to expand...

Fred
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Gateway Shutting Down 0
Enhancing the Gateway (Help Needed) 24
Gateway In Danger 0
Gateway still down? 2
SPAM from Usenet 71
Gateway -> Mail 9
Gateway broken (1-Feb-2006) 3
Gateway Broken (Aug 25, 2005) 1

Members online

Total: 32 (members: 1, guests: 31)
Robots: 433

Forum statistics

Threads
473,768
Messages
2,569,574
Members
45,050
Latest member
AngelS122

Latest Threads

Top