JMS vs Sockets -- bandwidth, size, speed, etc.

[Kevin McMurtrie]

That may be how you use message queue solutions.

But there are other that use message queues without the
reliability options.


????

How?

The ActiveMQ jars and the applications jars should not be
mixed at all. Either different JVM or different apps within the
same app server. Only exception I can think of is embedded
inside an SE app, which is not a common scenario.

Arne

I agree about how it should work but there is no minimal ActiveMQ
client. The best you can do is a trial-and-error process to create
exclusion lists in Ivy/Maven rather than pulling down the full client
JAR.

 

[Lew]

I think that I didn't phrase my question well enough.

You are mistaken. You phrased it just fine.

There are two metrics that I am curious about--bandwidth usage and speed.

We understood that. We are not stupid.

The advice given stands.

For X messages of N length (assume a constant size) going to each of Y consumers (so, X * Y messages total), what is the comparison?

Okay ...

I can test the speed and so far, the sockets seem to win as long as there
Good.

are not a lot of consumers (otherwise the threading seems to choke it).

This is consistent with the advice you've been given - measure under conditions
approximating your projected deployment, compare costs and benefits, and to
consider that faster than fast enough might not be worth it.

It is not usually simple to create meaningful benchmarks, and without knowing
the protocols, as pointed out upthread, it is impossible to make meaningful
statements about the results.

You state that you've run tests. Good on ye. Other people's experience is in
general not at all comparable to yours. The answers upthread gave as much as
possible given your parameters here.

That leaves the bandwidth question--how much larger (if any) is a JMS message
on the wire vs in a socket. I would hazard (as a newbie) that the socket is

That depends entirely on the protocol used to encode the message in either
case.

smaller--you don't have wrappers or envelopes or meta data but instead just

The use of sockets vs. JMS is not germane to message size. Sockets have nothing
to say about message size whatsoever. Zip. Nada. The train will grow as many
cars as it needs to carry however much cargo you lade.

JMS is an API. There is some overhead in coordinating message senders and
recipients. That overhead exists also in roll-your-own schemes.

the data. So how much larger is the JMS message?

Sigh.

 

[Arne Vajhøj]

I agree about how it should work but there is no minimal ActiveMQ
client. The best you can do is a trial-and-error process to create
exclusion lists in Ivy/Maven rather than pulling down the full client
JAR.

The client code can obviously not be separated from the app.

But my experience is rather different.

The all jar and a SLF logger and it works.

And the all jar only contains ActiveMQ, KahaDB and some SLF API.

Very far from half the internet.

Arne

 

[Arne Vajhøj]

The client code can obviously not be separated from the app.

But my experience is rather different.

The all jar and a SLF logger and it works.

And the all jar only contains ActiveMQ, KahaDB and some SLF API.

Very far from half the internet.

(version 5.x)

Arne

 

[Arved Sandstrom]

They're almost lightweight without the high reliability options, but
those are the same options that cause people to choose JMS rather than
an in-house solution. FFMQ is a lightweight codebase but it's still
going to hit disk I/O fairly hard. ActiveMQ is a resource-consuming
beast when faced with reliable delivery to clients that are lagging.
ActiveMQ also has dependencies on half the Internet so it's prone to
creating library conflicts.

What I was getting at is, if you're going to use the terms "lightweight"
and "heavyweight" at all - and I eschew both because they are too
confusing for people from different communities - then by comparison to
J2EE/JavaEE/JEE servers running humungous apps, or the big iron message
brokers (*)/ESBs, the messaging providers like WMQ or JMS providers are
still "lightweight".

Consider that if you're in an environment that demands a certain level
of reliability (all the various options), that everything working in
that environment will have to satisfy those criteria. So sure, you beef
up your JMS provider setup, but you're doing the same to the "heavier" apps.

AHS

* A "message broker", like IBM WMB, effectively being an ESB, not a
messaging provider. Terminology has gotten very buggered up these days.

 

[Arved Sandstrom]

Good morning,

I think that I didn't phrase my question well enough. There are two metrics that I am curious about--bandwidth usage and speed. For X messages of N length (assume a constant size) going to each of Y consumers (so, X * Y messages total), what is the comparison? I can test the speed and so far, the sockets seem to win as long as there are not a lot of consumers (otherwise the threading seems to choke it). That leaves the bandwidth question--how much larger (if any) is a JMS message on the wire vs in a socket. I would hazard (as a newbie) that the socket is smaller--you don't have wrappers or envelopes or meta data but instead just the data. So how much larger is the JMS message?

Thank you,
me 2

I'm wondering why you are worried about bandwidth and speed. Message
throughput on properly-tuned *single* instances of MQ or JMS messaging
providers (or ESBs for that matter) can easily be thousands, or tens of
thousands, of decent-sized (KB or tens of KB) messages per second.

I'd embrace all the advantages you get from a JMS messaging provider.
Odds are that if performance becomes a concern that horizontal and/or
vertical scaling using your selected JMS software will address the problem.

Some messaging provider development teams are better than others, and
some products are more mature than others, but rest assured that
practically all of them have devoted way more time, and amassed much
more expertise, to/on messaging intricacies than you ever will.

AHS

 

[Arne Vajhøj]

What I was getting at is, if you're going to use the terms "lightweight"
and "heavyweight" at all - and I eschew both because they are too
confusing for people from different communities

Those two terms are two of the most misused terms in the
Java world and they can mean very different things:

hw lw

big as in many classes small as in few classes
big overhead ratio low overhead ratio
very intrusive in code not intrusive in code
use many resources use few resources

so there are better terms to explain what we mean.

Arne

 

[Arne Vajhøj]

The socket code clocked faster when processing 1 consumer and 1
producer than my quick throw-together of JMS with 5 consumers and 1
producer. Both sent 10 character strings 10,000 times. I'm just
getting back into Java, so I'm sure that the socket programming was
not as optimized as it could be--but there was 3 consumers and 1
producer and it still was faster than the 1 producer/5 consumer JMS
setup. The multiple consumers connected to a socket/thread setup as
illustrated by Oracle in their KnockKnock example--so I'm not sure
how this would scale for 10 consumers, 100 consumers, etc.

Given that your message queue also uses sockets, then you know that
sockets can do better with more clients.

The bandwidth size is the most important priority because of a number
of considerations--

Message queues are usually used within data centers with
Gbit networks.

I can't justify using JMS if it uses 10 times the
bandwidth--so I have to figure out how to measure that.

Unless you are charged per byte, then I don't think there is any
point.

Actual throughput is what is relevant.

The next
thing is speed--how fast can the X number of messages go through.
The next priority is maintainability and somewhere else down the list
is scalability.

Have you proven that the most simple and most maintainable solutions
actually have a performance problem?

Arne

 

[Joerg Meier]

Those two terms are two of the most misused terms in the
Java world and they can mean very different things:

hw lw

big as in many classes small as in few classes
big overhead ratio low overhead ratio
very intrusive in code not intrusive in code
use many resources use few resources

so there are better terms to explain what we mean.

You forgot about the Swing definition of heavyweight and lightweight
components - something to do with transparency, I believe, but I'm not a
Swing person.

Liebe Gruesse,
Joerg

 

[John B. Matthews]

AWT Component w/peer Swing JComponent

You forgot about the Swing definition of heavyweight and
lightweight components - something to do with transparency, I
believe, but I'm not a Swing person.

Good point; not transparency so much as association with a
native screen resource:

<http://www.oracle.com/technetwork/articles/java/mixing-components-433992.html>

 

[me 2]

Greetings,



I am a newbie to JMS and would appreciate some advice.



Has anyone compared JMS to socket programming? If I have N number of clients that need to connect to and send messages to 1 server, what is the comparison? I would guess that sockets--direct from a client to the server--would be the fastest for speed and maybe take the least bandwidth. But I would expect that there would only be negligible size increases for the JMS overhead once the connection was established and I would expect that a pub-sub topic would be able to smoke through sending the N number of clients messages, rather than loop through the connections/sockets and sending the message to each of them.



Has anyone else looked at this? I'm going through the exercise to set upa JMS server, but I thought maybe someone else could point me in the rightdirection.



Cheers,

me2

I figured that if anyone else had this problem, then it would be worth passing on this information.

http://activemq.2283324.n4.nabble.com/Message-bandwidth-overheads-how-to-reduce-td3718866.html

According to this, an empty message with no other information is about 300 and some odd bytes. So, the plain socket in our extremely low bandwidth project might be better served with plain sockets. Not the answer that I wanted, but still, at least, an answer.

Cheers all and thank you to everyone who responded.
me 2

 

[Arne Vajhøj]

I figured that if anyone else had this problem, then it would be
worth passing on this information.

http://activemq.2283324.n4.nabble.com/Message-bandwidth-overheads-how-to-reduce-td3718866.html

According to this, an empty message with no other information is
about 300 and some odd bytes. So, the plain socket in our extremely
low bandwidth project might be better served with plain sockets. Not
the answer that I wanted, but still, at least, an answer.

That may be the case. Or it may not be the case.

If you want to know then measure.

If you expectation is that:

number of messages per time unit = constant * theoretical bandwidth /
message size

then you will be wrong for most networks.

I wrote a little test program (Java socket) to measure
the time it takes to send 1 million messages for
different message sizes.

Result:

size 100 : 4633 ms
size 200 : 4681 ms
size 300 : 4697 ms
size 400 : 4712 ms
size 500 : 4728 ms
size 600 : 4743 ms
size 700 : 4791 ms
size 800 : 4805 ms
size 900 : 4822 ms
size 1000 : 4852 ms
size 1100 : 4853 ms
size 1200 : 4899 ms
size 1300 : 4931 ms
size 1400 : 4915 ms
size 1500 : 6834 ms
size 1600 : 6912 ms
size 1700 : 6865 ms
size 1800 : 7005 ms
size 1900 : 6974 ms
size 2000 : 7130 ms
size 2100 : 7255 ms
size 2200 : 7318 ms
size 2300 : 7317 ms
size 2400 : 7302 ms
size 2500 : 7380 ms
size 2600 : 7395 ms
size 2700 : 7567 ms
size 2800 : 7583 ms
size 2900 : 7442 ms
size 3000 : 8082 ms
size 3100 : 8222 ms
size 3200 : 8191 ms
size 3300 : 8332 ms
size 3400 : 8222 ms
size 3500 : 9330 ms
size 3600 : 9407 ms
size 3700 : 9533 ms
size 3800 : 9377 ms
size 3900 : 9423 ms
size 4000 : 9439 ms
size 4100 : 9439 ms
size 4200 : 9517 ms
size 4300 : 9533 ms
size 4400 : 10359 ms
size 4500 : 10484 ms
size 4600 : 10407 ms
size 4700 : 10499 ms
size 4800 : 10578 ms
size 4900 : 10484 ms
size 5000 : 10500 ms

You will see that it does not grow linear instead it jumps
at multiples of X (where X is somewhere between 1400 and 1500
bytes).

Your absolute number may be very different.

And your X may be different.

But you should not be surprised if you see the same
effect.

And it does impact your conclusion.

If payload+300 < X then the 300 has practically
no impact on performance.

If payload < X < payload+300 then the 300 will have
a huge impact on performance.

And so on.

So you need to analyze and test more - otherwise you
could just as well have flipped a coin.

Arne

PS: If somebody want a copy of the code, then I will be happy
to post it, but it is rather trivial and somewhat unpolished.

 

[Gene Wirchenko]

[snip]

I wrote a little test program (Java socket) to measure
the time it takes to send 1 million messages for
different message sizes.

Result:
[snip]

You will see that it does not grow linear instead it jumps
at multiples of X (where X is somewhere between 1400 and 1500
bytes).

An Ethernet frame contains up to 1500 bytes of payload. I
suspect that this is the explanation for the results you got.

[snip]

Sincerely,

Gene Wirchenko

 

[Arne Vajhøj]

[snip]

I wrote a little test program (Java socket) to measure
the time it takes to send 1 million messages for
different message sizes.

Result:
[snip]

You will see that it does not grow linear instead it jumps
at multiples of X (where X is somewhere between 1400 and 1500
bytes).

An Ethernet frame contains up to 1500 bytes of payload. I
suspect that this is the explanation for the results you got.

[snip]

Yes.

Arne

 

[Robert Klemme]

I figured that if anyone else had this problem, then it would be worth passing on this information.

http://activemq.2283324.n4.nabble.com/Message-bandwidth-overheads-how-to-reduce-td3718866.html

According to this, an empty message with no other information is about 300 and some odd bytes. So, the plain socket in our extremely low bandwidth project might be better served with plain sockets. Not the answer that I wanted, but still, at least, an answer.

As far as I can see that's just _one_ product. That does not necessarily mean that all others have the same overhead, does it?

Kind regards

robert

 

[Mark]

Hmmm.

How did you test JMS from C++?

e.g. ActiveMQ-CPP

 

[Arne Vajhøj]

e.g. ActiveMQ-CPP

Most message queues has a C/C++ client lib.

But JMS ...

Arne

 

[Roedy Green]

What is JMS using under the covers? UDP, TCP/IP?

 

[Lew]

What is JMS using under the covers? UDP, TCP/IP?

Shared memory.

As Arne pointed out near the start of this discussion:

JMS is just an API.

Systems I've worked on have done different things depending on the
configuration. Shared memory, dedicated pipes, specialized sockets
are three things that can come in the very same JMS system.

GIYF.

 

[Arne Vajhøj]

What is JMS using under the covers? UDP, TCP/IP?

I will tell you right after you tell me what transport classes
implementing this interface:

public interface ThisIsJustAnAPI {
public void doSomething();
}

use!



Or maybe you realize something ....

Arne