I want my MOM

[swille]

Err.. Can anyone tell me if there's a MOM for Ruby? Anything like
JMS? I haven't quite found one in my googling.

Also, and this is a long-shot, is there possibly a workflow engine
that anyone has stuffed away?

 

[Austin Ziegler]

Err.. Can anyone tell me if there's a MOM for Ruby? Anything like
JMS? I haven't quite found one in my googling.

Also, and this is a long-shot, is there possibly a workflow engine
that anyone has stuffed away?

http://raa.ruby-lang.org/project/torrentworkflow/

It doesn't seem to have been updated in a while, so it might need some
tweaking for Ruby 1.8 and later.

What's a MOM?

-austin

 

[Sille Wille]

What's a MOM?

-austin

Message-Oriented-Middleware (message queue)
JMS being the Java Messaging Service which is the standard way to talk
to a MOM in Java.

Just looking for what Ruby might have in the same space.

 

[Zed A. Shaw]

I'm kind of in the process of writing something like this, but if you
want take a look at STOMP and the following:

http://stomp.codehaus.org/Ruby+Client

STOMP will then let you talk with several different JMS implementations
(mostly open source).

Be careful though, STOMP isn't as well designed as I'd like. For
example, it uses \0 to end the message body, without any way of
escaping the \0. This means you have to base64 things you send if they
have \0 characters.

Zed A. Shaw
http://www.zedshaw.com/

 

[Brian McCallister]

Zed,

The \0 terminator is used if there is *not* a content-length header,
it is recommended that a content-length header be used. The primary
reason for this is that I *really* want to support telnet and netcat
for emergencies and debugging =)

That said, please keep the criticism coming, I don't think for a
moment that I know everything, and the folks helping with stomp are
all merely trying to solve a problem, no one is pushing an agenda
(that I am aware of!).

I would very much appreciate your further feedback!

-Brian

 

[Brian McCallister]

I'm kind of in the process of writing something like this, but if you
want take a look at STOMP and the following:

http://stomp.codehaus.org/Ruby+Client

A much better ruby client is also available:

http://svn.stomp.codehaus.org/trunk/ruby/

I'll put a gem up as soon as the project is approved on rubyforge, in
the mean time "rake package" will build you one =)

-Brian

 

[Corey Jewett]

Also it looks like ActiveMQ 3.1 has Stomp support built-in. Being a
recent Java convert, JMS was one of the first things I was looking for.
The Ruby community delivers again.

Corey

 

[Zed A. Shaw]

Hey Brian, I've got an alternative hack that fixes my two problems with
the protocol. My big problem is that line-ended protocols are a pain
in the ass to implement and scale. Sure it makes it easy for you to
use telnet rather than write a real test client (which seriously took
me like 10 minutes in Ruby BTW), but line-ended protocols leave open
huge areas for abuse. With that in mind here's my proposal to fix the
STOMP protocol so that it's still telnet accessible but all sizes are
predefined to avoid abuse:

1) Define the official line ending. This is important since most IETF
protocols end lines with \r\n. For the rest of this I assume \n.
2) Header is changed to be: SIZE COMMAND\n with SIZE being a decimal
string of the byte size of the HEADER portion immediatly after this
line.
3) Specify that the first line is never longer than X bytes. This is
important as a classic DOS attack is to "trickle" characters forever
never giving the \n. If you explicitly state that it can't be longer
than say 128 characters then servers can boot bad clients quick.
4) The SIZE of the header specifies the exact length, and removes the
need for the \n\n separator. This is also important because I can boot
a client that tries to send more than SIZE or that says it's going to
send a SIZE that I don't want. This prevents another attack where I
just trickle header lines until the server croaks.
5) Require that the header MUST have the content-length settings. This
is again to make sure that requests which are too large are always
booted. Making this optional just leaves open an attack where you
again slam the server with never ending data until it dies. This also
makes it very clear how the content is always sent and leaves nothing
open for debate.
6) State that the header has an exact format of "key: value\n" and
that the value goes from the space after the ":" until \n always. Even
other ":" characters are included.
7) State that ":" is not a valid header. This means that they can't
send empty headers.
8) Also state that clients SHOULD be booted immediately if they don't
follow these rules exactly. I really don't like protocols that try to
be "nice to users". These protocols are actually just leaving things
open for attackers to exploit. In the end "users" do not write
clients, developers do and being explicit about everything makes it
easier for them. This also makes it easier for server implementers to
boot bad clients without any excuse reducing their attack vectors and
improving their chances of withstanding DOS attacks.

Anyway, that's just my opinion on it. Take it or leave it.

Zed A. Shaw
http://www.zedshaw.com/

 

[Yohanes Santoso]

3) Specify that the first line is never longer than X bytes. This is
important as a classic DOS attack is to "trickle" characters forever
never giving the \n. If you explicitly state that it can't be longer
than say 128 characters then servers can boot bad clients quick.
4) The SIZE of the header specifies the exact length, and removes the
need for the \n\n separator. This is also important because I can boot
a client that tries to send more than SIZE or that says it's going to
send a SIZE that I don't want. This prevents another attack where I
just trickle header lines until the server croaks.
5) Require that the header MUST have the content-length settings. This
is again to make sure that requests which are too large are always
booted. Making this optional just leaves open an attack where you
again slam the server with never ending data until it dies. This also
makes it very clear how the content is always sent and leaves nothing
open for debate.

To add to what Zed's said:

Servers can always prevent trickle attack by specifying a
timeout. Without any timeout from server, a client would always be
able to tie up unnecessary resources (IP port, for example).

Most HTTP/1.1 servers close idle persistent connection after 2 minutes
of inactivity.

Again, server implementations should employ some sort of timeout
mechanism. That's just the right thing to do.

YS.

 

[Brian McCallister]

Feedback is much appreciated!

1) Define the official line ending. This is important since most IETF
protocols end lines with \r\n. For the rest of this I assume \n.

Good call, this has been left hanging while discussing the other issues.

2) Header is changed to be: SIZE COMMAND\n with SIZE being a decimal
string of the byte size of the HEADER portion immediatly after this
line.

What will this buy other than making it easier to implement a client?

3) Specify that the first line is never longer than X bytes. This is
important as a classic DOS attack is to "trickle" characters forever
never giving the \n. If you explicitly state that it can't be longer
than say 128 characters then servers can boot bad clients quick.

Very good point, thank you!

4) The SIZE of the header specifies the exact length, and removes the
need for the \n\n separator. This is also important because I can
boot
a client that tries to send more than SIZE or that says it's going to
send a SIZE that I don't want. This prevents another attack where I
just trickle header lines until the server croaks.

see next response.

5) Require that the header MUST have the content-length settings.
This
is again to make sure that requests which are too large are always
booted. Making this optional just leaves open an attack where you
again slam the server with never ending data until it dies. This also
makes it very clear how the content is always sent and leaves nothing
open for debate.

I disagree with this -- you can bring down a naive server
implementation with or without this change -- just slam data into
whatever buffer is accumulating it until it croaks -- almost every (I
would say every but I don't know for a fact) JMS implementation
available does a full read for messages which are not of type Stream
(which is an option for huge messages).

This change does leave open the option for rejecting messages which
promise to be over the declared size, which is a non-trivial benefit.

6) State that the header has an exact format of "key: value\n" and
that the value goes from the space after the ":" until \n always.
Even
other ":" characters are included.

This is how it is, other : characters are allowed and the space is
not required, but is allowed (and trimmed)

7) State that ":" is not a valid header. This means that they can't
send empty headers.

What is wrong with an empty header? It wastes bytes, but then so does
a long message.

foo:
bar:
baz:

all are headers with empty values

:

is bizarre, and wastes a little bit of space, but doesn't *hurt* the
protocol. In a defensive server, it would be fair to consider it a
potential attack and consider dropping the client. I am not sure
specifying this behavior adds anything to the spec except "oh yeah,
make sure of this" for implementors.

HTTP and SMTP also allow these attacks ... certainly folks have had
to do a lot of work to prevent DOS attacks such as you describe, and
preventing them in the protocol is a pretty good suggestion.

8) Also state that clients SHOULD be booted immediately if they don't
follow these rules exactly. I really don't like protocols that try to
be "nice to users". These protocols are actually just leaving things
open for attackers to exploit. In the end "users" do not write
clients, developers do and being explicit about everything makes it
easier for them. This also makes it easier for server implementers to
boot bad clients without any excuse reducing their attack vectors and
improving their chances of withstanding DOS attacks.

Good point.

Anyway, that's just my opinion on it. Take it or leave it.

Happily take it!

Your points which I have disagreed with I believe I see your point,
and do not wholeheartedly disagree -- one of core design principles
has been allowing for telnet clients a la http and smtp (neither of
which uses a size-of-headers for clients posting messages), though.

I completely agree on limiting line length and limiting headers -- it
seems arbitrary, but allowing a max of N headers provides the same
anti-DOS behavior, while still allowing for telnet clients.

-Brian

ps: forwarded this to the stomp discussion list as well, I very much
appreciate your feedback, as you have very good points. Thank you for
raising them =)

 

[Brian McCallister]

--Apple-Mail-3--648923769
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
charset=US-ASCII;
delsp=yes;
format=flowed

What will this buy other than making it easier to implement a client?

Ignore that question, you made what it provides abundantly clear!

-brian
--Apple-Mail-3--648923769--

 

[Zed A. Shaw]

5) Require that the header MUST have the

To add to what Zed's said:

Servers can always prevent trickle attack by specifying a
timeout. Without any timeout from server, a client would always be
able to tie up unnecessary resources (IP port, for example).

Most HTTP/1.1 servers close idle persistent connection after 2 minutes
of inactivity.

Again, server implementations should employ some sort of timeout
mechanism. That's just the right thing to do.

YS.

That misses the point of the trickle attack. I just trickle enough to
never be idle but slowly eat up memory. Since the server can't boot me
as I might be legit, it's got to keep everything I send around.
Eventually I'll fill up the process' memory. Most servers just set a
max size for the read buffer, but then they have to always read that
amount before detecting the attack. By simply predefining the size bad
clients can be booted immediately for going over size and booted before
the data has been read.

Predefined size protocols also have the advantage that any data can be
sent without needing to encode the data. If the server knows how much
it must read then it just reads that much. With line ended protocols
you're constantly negotiating some encoding so that you avoid binary
data tripping up the parsing.

Zed A. Shaw
http://www.zedshaw.com/

 

[Zed A. Shaw]

Feedback is much appreciated!

On Oct 23, 2005, at 9:17 PM, Zed A. Shaw wrote:

I disagree with this -- you can bring down a naive server
implementation with or without this change -- just slam data into
whatever buffer is accumulating it until it croaks -- almost every
(I would say every but I don't know for a fact) JMS implementation
available does a full read for messages which are not of type Stream
(which is an option for huge messages).

You can bring down any server (especially a niave one), it's just a
hell of a lot harder if the server is able to always know
received data sizes and to boot clients if they are over the
allowed sizes.

Normally these specifications are not designed to prevent
attacks totally (can't be done) but rather to make it easier for a
server to have a reason to boot bad clients. Additionally, it lets the
server properly size receive resources so that it can scale
appropriately. If your server knows the size of all data coming in
then it's a piece of cake to meet service needs by setting input/output
sizes appropriately.

Another thing a pre-sized protocol does is make it much harder to have
buffer overflows. Since all buffers always have a size, and the
protocol always has a size for each read, then you don't get buffer
overflows (as much).

In other words, yes morons will always write bad clients/servers, and
yes other people write their implementations with line-endings, but that
sure as hell don't make line-ended protocols right. As a bit of
history, line-ended protocols were almost exclusively the domain of
UNIX systems since the BSD sockets guys convinced everyone that "a
socket is just a file". This led people to think, "Oh, then I'll just
read lines from it like a file." Problem is, how many files do you
open up where there's a bad guy on the other end messing with your
file's data using a dirty connection? This is why "sockets are files"
and line-ended protocols are super bad. Sockets are not files, be safe.

My best analogy is it's the difference between wearing a condom
with your wife and going to a glory hole. Uh, ok, that's a *really* bad
analogy. I'll stop right there.

But, like I said previously, saying that your protocol is telnet
accessible is just a super bad idea. The only people who ever need
telnet access are developers (who should be able to write a client), or
attackers (who should not be given any more help than they already
have).

Anyway, that's my argument. Let me know if it goes anywhere with the
STOMP folks.

What is wrong with an empty header? It wastes bytes, but then so
does a long message.

Empty headers are fine, I meant the null header (see below).

:

is bizarre, and wastes a little bit of space, but doesn't *hurt* the
protocol. In a defensive server, it would be fair to consider it a
potential attack and consider dropping the client. I am not sure
specifying this behavior adds anything to the spec except "oh yeah,
make sure of this" for implementors.

It's simply a suggestion to close a potential loop-hole.

Thanks for listening. Let me know what comes of it. I'm basically
basing a kind of little messaging system on STOMP with these changes.

Zed A. Shaw
http://www.zedshaw.com/

 

[Brian McCallister]

But, like I said previously, saying that your protocol is telnet
accessible is just a super bad idea. The only people who ever need
telnet access are developers (who should be able to write a
client), or
attackers (who should not be given any more help than they already
have).

Actually, I did more manual http and smtp in my days as a sysadmin
than I have ever done as a developer =) I am not sure your statement
is accurate, but point is taken.

Anyway, that's my argument. Let me know if it goes anywhere with the
STOMP folks.

Happily will!

-Brian

 

[Yohanes Santoso]

That misses the point of the trickle attack. I just trickle enough to
never be idle but slowly eat up memory.

Since the server can't boot me as I might be legit,

The server must have the ability to boot you if you pass a certain
timeout. And there are many kinds of timeout policies, e.g.: a minimum
traffic rate of x byte/y time (i've never seen this one), a maximum of
x time for a request/response process to be done (i see this a lot in
financial-related protocols), a maximum of x time for idle connection
(HTTP/1.1 uses this particular one).

YS.

 

[Sacha Schlegel]

Hi swille

I once wrote a ruby client for xmlblaster (www.xmlblaster.org). The
client was just about sufficient for my needs.

Sacha

PS: you can find it at www.schlegel.li/ebXML/direct-access called
rubyXMLBlaster.tar.gz (ruby licence)

 

[paul.rogers]

Im not entirely sure its what you are looking for, and I have no
experience of using it, but spread might work for you

http://www.spread.org/