Mini-rant on Java REST (JAX-RS), JSON, XML, JAXB etc...

[Arved Sandstrom]

I just had an epiphany today at work. For years upon years I have used
Java libraries which have grown increasingly cumbersome and finicky and
unreliable to do XML or JSON in a REST context...99 percent of which is
actually super-simple.

The epiphany came when neither Jettison (the default Jersey JSON
library) nor Jackson (which is undoubtedly more modern) could, out of
the box without arcane configuration tweaks, convert a single-item list
into a JSON array. They both just converted the item into a JSON object
and forgot about the array.

Apparently this catches out a lot of people, judging by googling. The
Jackson solutions are many. The point being - leading to my epiphany -
why the hell is this even a problem?

I have probably wasted tens of weeks on the arcana of Jersey, Jackson,
XStream to some extent, JAXB...I'm ditching most all of it. It is an
obstacle.

XStream I actually like for producing and consuming XML. It works
nicely. I'll keep it in the toolbox. But for almost everything I do with
Java REST, there is no call for Jersey (nor Jackson et al. for JSON
(de-)serialization). It's a bunch of extra JARs for no added value.

It occurred to me that for over 90 percent of my POJOs I can write
reliable toJSON() methods that *will not break* and are fully under my
control in a matter of minutes. For anything more I might give
simple-json a whirl - it actually has appealing simplicity.

And Jersey has got to go. Why do we even drink that Kool-Aid? Once
you've got your JSON string a handful of lines of code with an HTTP
client will take care of your REST call. A lot more reliable.

SOAP is a different story. I'm sticking with CXF and JAXB and all that
good stuff for SOAP.

Just an opinion piece. I'm getting tired of bloated APIs and unreliable
JARs that, in their quest to make everything possible, make the simple
things difficult.

AHS

 

[Joerg Meier]

Apparently this catches out a lot of people, judging by googling. The
Jackson solutions are many. The point being - leading to my epiphany -
why the hell is this even a problem?

I have probably wasted tens of weeks on the arcana of Jersey, Jackson,
XStream to some extent, JAXB...I'm ditching most all of it. It is an
obstacle.

It occurred to me that for over 90 percent of my POJOs I can write
reliable toJSON() methods that *will not break* and are fully under my
control in a matter of minutes. For anything more I might give
simple-json a whirl - it actually has appealing simplicity.

I must admit that if it weren't for the tight tie-in with RESTEasy, I would
likely dump Jackson as well. For something as simple as my use cases, I
find myself having to write a surprising amount of annotations and shit to
make it work right, and for everything there appear to be a myriad of
options that I honestly don't understand ever made it in the standard
release - which is gigantic!

That being said, I can't see myself going to write my own toJSON stuff.
Maybe my development style is too dynamic and fluid, but the mere extra
cost just from refactoring (and the bugs when I forget to properly
refactor) alone makes that an easy decision.

Code generating annotations, on the other hand, appear to be outright
trivial, and certainly much more flexible, and I could see myself writing
my own limit case JSON generating annotations.

But realistically, I won't, because that would just be easy grunt work, and
no part of that would be tricky to figure out, and in the end, I do live
for those tricky problems that are hard to figure out

Liebe Gruesse,
Joerg

 

[Stefan Ram]

I just had an epiphany today at work.

See also:

http://www.propylon.com/news/ctoarticles/APIs_considered_20020418.html

(I first posted a URI of this article here in 2006 in
<[email protected]>. In 2006,
that URI was still http://www.itworld.com/nl/xml_prac/04182002/.)

 

[Arved Sandstrom]

See also:

http://www.propylon.com/news/ctoarticles/APIs_considered_20020418.html

(I first posted a URI of this article here in 2006 in
<[email protected]>. In 2006,
that URI was still http://www.itworld.com/nl/xml_prac/04182002/.)

It's a good read, thanks. I agree with it. That kind of thinking is why
I use REST Console and SoapUI religiously, because they let me test the
receiving end and iron out that piece of the puzzle; it also reminds me
of how basic the communication actually is.

I'm not going to go right to sockets. But I can see using
HttpComponents HttpClient, because I am familiar with it and it's
simple. In conjunction with something like XStream built-in JSON
writing, or json-simple, and a simple class that encapsulates HttpClient
GET, POST, PUT and DELETE with path and payload parameters, I'm talking
a few lines of code for each REST call.

All that extra cruft in Jersey is maybe fine if you're working
full-blown REST resource discovery and automated use and what not, but I
don't. I have simple REST usages, which may be the case for a whole
bunch of people.

AHS

 

[Kevin McMurtrie]

I just had an epiphany today at work. For years upon years I have used
Java libraries which have grown increasingly cumbersome and finicky and
unreliable to do XML or JSON in a REST context...99 percent of which is
actually super-simple.

The epiphany came when neither Jettison (the default Jersey JSON
library) nor Jackson (which is undoubtedly more modern) could, out of
the box without arcane configuration tweaks, convert a single-item list
into a JSON array. They both just converted the item into a JSON object
and forgot about the array.

Apparently this catches out a lot of people, judging by googling. The
Jackson solutions are many. The point being - leading to my epiphany -
why the hell is this even a problem?

I have probably wasted tens of weeks on the arcana of Jersey, Jackson,
XStream to some extent, JAXB...I'm ditching most all of it. It is an
obstacle.

XStream I actually like for producing and consuming XML. It works
nicely. I'll keep it in the toolbox. But for almost everything I do with
Java REST, there is no call for Jersey (nor Jackson et al. for JSON
(de-)serialization). It's a bunch of extra JARs for no added value.

It occurred to me that for over 90 percent of my POJOs I can write
reliable toJSON() methods that *will not break* and are fully under my
control in a matter of minutes. For anything more I might give
simple-json a whirl - it actually has appealing simplicity.

And Jersey has got to go. Why do we even drink that Kool-Aid? Once
you've got your JSON string a handful of lines of code with an HTTP
client will take care of your REST call. A lot more reliable.

SOAP is a different story. I'm sticking with CXF and JAXB and all that
good stuff for SOAP.

Just an opinion piece. I'm getting tired of bloated APIs and unreliable
JARs that, in their quest to make everything possible, make the simple
things difficult.

AHS

A lot of people like to preach open source projects, modern APIs, and
the elegance of abstraction layers. "Don't reinvent the wheel!" they
say. It's all good until it's time to fix bad performance, change a
feature, or track down a strange conflict. The cost of that must be
weighed against the cost of creating a simple custom solution. Some
APIs give you a train, tracks, and train stations when all you wanted
was the wheel.

My experience with Jackson is good except for the documentation sucking.
It's a nicely layered API so you get only the services you need. The
only thing it doesn't handle is large values. Take a look at whether or
not its poor documentation has led to it being used incorrectly.

SOAP is one of the worst widely adopted protocols ever. It makes LDAP
seem sensible. Just say "no."

 

[Arved Sandstrom]

A lot of people like to preach open source projects, modern APIs, and
the elegance of abstraction layers. "Don't reinvent the wheel!" they
say. It's all good until it's time to fix bad performance, change a
feature, or track down a strange conflict. The cost of that must be
weighed against the cost of creating a simple custom solution. Some
APIs give you a train, tracks, and train stations when all you wanted
was the wheel.

My experience with Jackson is good except for the documentation sucking.
It's a nicely layered API so you get only the services you need. The
only thing it doesn't handle is large values. Take a look at whether or
not its poor documentation has led to it being used incorrectly.

My experience with Jackson up until this week was OK. Not great, but on
a par with other Java APIs/libraries that I personally classify as
"acceptable". But my overall experience with the entire Jersey + JAXB +
Jettison/Jackson combo for REST+JSON has been deteriorating over the
months, so I'm dispensing with all of it as being badly done and
unwieldy and overkill for me.

Jackson, out of that group of libraries, is not the major offender. But
I've realized that it's overkill. Some of the other libraries aren't
just overkill but also actively suck.

SOAP is one of the worst widely adopted protocols ever. It makes LDAP
seem sensible. Just say "no."

It can be unpleasant. SOAP itself is not a major headache for me, it's
the Java libraries for handling it that are, none of which are good and
some of which are horrible. If I took any one SOAP situation that I had
I could hand-write the request (or response, where I own both ends) in a
few minutes, they would be easy to understand, and I could telnet the
request on the C.L. too...just as for REST.

What complicates matters is the structure imposed by WSDL and XSDs, but
I consider that in those situations where SOAP is a better choice than
REST, that it's exactly that extra structure I need. Unfortunately even
the better Java libraries and tools for dealing with SOAP create code
that no human being should ever look at. Just as for REST, if you
started getting closer to the bare metal - short of telnetting - you'd
find that you could dispense with 10's of MBs of libraries and hundreds
of library classes and 10's of thousands of lines of generated code if
you wrote your code using much more basic APIs...with the side benefit
of knowing what's going on. In the Java SOAP case this might be SAAJ and
some hand-rolled wrappers to simplify the use thereof.

And overall, yes, documentation and example code could use major
improvements for almost everyone. For example, I've developed
substantial proficiency with CXF, before that with Axis. The wasted
hours I burned because of poor docs or examples, that's what everyone is
going through. A lot of these teams, you can tell they never worked
through a "what's this look like to a novice user?" exercise.

AHS

 

[Fredrik Jonson]

SOAP is one of the worst widely adopted protocols ever. It makes LDAP
seem sensible. Just say "no."

Funny, this makes me think that each generation of programmers is doomed to
reinvent the wheel. Only, it isn't the wheel, it is RMI, CORBA, SOAP, or
some other remote service invocation protocol that's widely used.

A few years ago I had the impression that most parties had actually just
begun to become proficient in designing reasonable and usable SOAP services.
There was light at the end of the tunnel. Then all of a sudden it seemed as
everyone decided JSON over REST is the new black. And we were back where we
started again.

Now I find my colleagues and business relations discussing how to approach
schema validation for the JSON models published by our REST services - as if
that is a novel concept in computer science. And while that goes on in one
corner, elsewhere new REST-services are published every day, without a
thought given to versioning, maintenance, or if the current database
ER-model really is suitable as raw types in the REST service API too.

Yes, I'm well aware of the arguments why JSON is better than XML, and I
agree that XML has its deficiencies. And I'm not saying that SOAP was a
silver bullet either, far from it. I'm only asking, do we really have to
start from scratch every time?

A few years from now someone else is going to come by and state that
"REST/JSON is the worst widely adopted protocol ever, just say no. Let's all
use Protocol Buffers and MQTT instead".

</rant>

 

[Arne Vajhøj]

I just had an epiphany today at work. For years upon years I have used
Java libraries which have grown increasingly cumbersome and finicky and
unreliable to do XML or JSON in a REST context...99 percent of which is
actually super-simple.

The epiphany came when neither Jettison (the default Jersey JSON
library) nor Jackson (which is undoubtedly more modern) could, out of
the box without arcane configuration tweaks, convert a single-item list
into a JSON array. They both just converted the item into a JSON object
and forgot about the array.

Apparently this catches out a lot of people, judging by googling. The
Jackson solutions are many. The point being - leading to my epiphany -
why the hell is this even a problem?

I have probably wasted tens of weeks on the arcana of Jersey, Jackson,
XStream to some extent, JAXB...I'm ditching most all of it. It is an
obstacle.

XStream I actually like for producing and consuming XML. It works
nicely. I'll keep it in the toolbox. But for almost everything I do with
Java REST, there is no call for Jersey (nor Jackson et al. for JSON
(de-)serialization). It's a bunch of extra JARs for no added value.

It occurred to me that for over 90 percent of my POJOs I can write
reliable toJSON() methods that *will not break* and are fully under my
control in a matter of minutes. For anything more I might give
simple-json a whirl - it actually has appealing simplicity.

And Jersey has got to go. Why do we even drink that Kool-Aid? Once
you've got your JSON string a handful of lines of code with an HTTP
client will take care of your REST call. A lot more reliable.

I think we need to split the stuff in 3 parts:
A) server side framework to enable declarative JAX-RS to work in a
servlet container
B) the JSON/POX serializer
C) client side framework

re A)

I don't hear you argue against that. And I don't recall much
criticism from other either.

re B)

You don't like the common libraries. I know several people that
don't like them either (I don't have so much personal experience).

But is it really the concept that is wrong or is it just the
implementations?

My guess is still the implementations. I am not too keen on
toJSON, toXML, toJSONAlternative, toXMLALternative etc.etc.
on all DTO's.

We may not always like SOAP and all the associated standards,
but sometimes the "there is only one right way" philosophy
do make life easier.

re C)

I think it is rather common to use plain HttpClient.

Non-Java client to Java service is probably also a very
common combination.

Arne

 

[Arne Vajhøj]

A lot of people like to preach open source projects, modern APIs, and
the elegance of abstraction layers. "Don't reinvent the wheel!" they
say. It's all good until it's time to fix bad performance, change a
feature, or track down a strange conflict. The cost of that must be
weighed against the cost of creating a simple custom solution. Some
APIs give you a train, tracks, and train stations when all you wanted
was the wheel.

My experience with Jackson is good except for the documentation sucking.
It's a nicely layered API so you get only the services you need. The
only thing it doesn't handle is large values. Take a look at whether or
not its poor documentation has led to it being used incorrectly.

I have seen people work around Jackson instead of work with Jackson
as well.

SOAP is one of the worst widely adopted protocols ever. It makes LDAP
seem sensible. Just say "no."

It works.

Arne

 

[Arne Vajhøj]

[snip]

SOAP is one of the worst widely adopted protocols ever. It makes LDAP
seem sensible. Just say "no."

Phew, so I'm not alone then. I remember doing some SOAP early adopter
work for a French company back in the day and decided it was better to
try to speak French than SOAP...

Well - if you speak SOAP then you are doing SOAP the wrong way.

Arne

 

[Arne Vajhøj]

Funny, this makes me think that each generation of programmers is doomed to
reinvent the wheel. Only, it isn't the wheel, it is RMI, CORBA, SOAP, or
some other remote service invocation protocol that's widely used.

A few years ago I had the impression that most parties had actually just
begun to become proficient in designing reasonable and usable SOAP services.
There was light at the end of the tunnel. Then all of a sudden it seemed as
everyone decided JSON over REST is the new black. And we were back where we
started again.

Fashion in IT waste billions of dollars.

Now I find my colleagues and business relations discussing how to approach
schema validation for the JSON models published by our REST services - as if
that is a novel concept in computer science. And while that goes on in one
corner, elsewhere new REST-services are published every day, without a
thought given to versioning, maintenance, or if the current database
ER-model really is suitable as raw types in the REST service API too.

Yes. We are learning that WS technology invented by frontend
JavaScript and PHP developers may be more lightweight than those
invented by billion dollar software companies, but that more lightweight
does not automatically mean good.

Yes, I'm well aware of the arguments why JSON is better than XML,

Besides data size (bandwidth) I don't see much at all.

Well - I did not count fashion.

Arne

 

[David Lamb]

It works.

That doesn't negate what Kevin said; it could "work", as in do useful
stuff, while still being the worst widely adopted protocol. I don't know
enough about SOAP to have a personal opinion, though.

 

[Arne Vajhøj]

That doesn't negate what Kevin said; it could "work", as in do useful
stuff, while still being the worst widely adopted protocol.

It certainly could.

I am just very reluctant to discard stuff that actually works.

Arne

 

[Arved Sandstrom]

That doesn't negate what Kevin said; it could "work", as in do useful
stuff, while still being the worst widely adopted protocol. I don't know
enough about SOAP to have a personal opinion, though.

As a model of RPC it makes me want to re-embrace CORBA. It's badly designed.

Having said that, if you work with simple subsets it's understandable,
robust, and very widely supported. The use of XML is very defensible, IMO.

I've seen plenty of much more terrible protocols. SOAP doesn't even come
close to some of the Cthulhian mind-blasting horrors I've dealt with.

AHS

 

[Arved Sandstrom]

I think we need to split the stuff in 3 parts:
A) server side framework to enable declarative JAX-RS to work in a
servlet container
B) the JSON/POX serializer
C) client side framework

re A)

I don't hear you argue against that. And I don't recall much
criticism from other either.

re B)

You don't like the common libraries. I know several people that
don't like them either (I don't have so much personal experience).

But is it really the concept that is wrong or is it just the
implementations?

My guess is still the implementations. I am not too keen on
toJSON, toXML, toJSONAlternative, toXMLALternative etc.etc.
on all DTO's.

We may not always like SOAP and all the associated standards,
but sometimes the "there is only one right way" philosophy
do make life easier.

re C)

I think it is rather common to use plain HttpClient.

Non-Java client to Java service is probably also a very
common combination.

Arne

For all of A, B, C, I have no problem with JSON as a data notation. I
have no major problems with SOAP as a protocol, or with the principles
of true RESTful web services [1].

For all of A, B, C, it is specifically implementations that I have a
problem with. Whether language-specific APIs or libraries. JSON is
simple. SOAP *messages* are usually quite simple (WSDL and schemas might
not be, but you infrequently occupy your time with those). JSON as a
payload over HTTP as part of a REST method is simple. SOAP XML via HTTP,
JMS, SMTP is usually quite simple.

*When we look at what is going back and forth*, that's what is simple.
What can be, and often is, elaborately over-engineered, clumsy, buggy,
and not simple are the client and server APIs and libraries.

As to toJSON() or fromXml() type methods, I'm not overly keen on them
either, for maintainability reasons. We very often have to explain to
the client or server framework how to handle the DTO<->JSON/XML
conversions, and annotations are superior to hardcoding for this. Except
when they're not.

AHS

[1] My uses of REST are almost always trivial. There is no
hypertext-driven application state because the interactions are
stateless in the REST sense. They may not be stateless in the system
(DELETE, PUT, POST) but I never expect the client to follow-up on a REST
call.

 

[Roedy Green]

A few years ago I had the impression that most parties had actually just
begun to become proficient in designing reasonable and usable SOAP services.

Programmers seem to prefer to reinvent the wheel than create something
new. Think how many JSP-like beasts we have, XML-like beasts we have,
how many IDEs. How many parsers. It may be a personality problem.
Technicians have no imagination. They need a rigidly defined problem.

Every time you add another 3rd party tool to your toolbox you must
consider the cost:
1. learning curve, ongoing learning investment to stay competent.
2. dependency. (what if it disappears, changes drastically, gets
expensive)
3. if you bring new people into your project they too need to learn
it, even if for just a small usage.
4. What to do when the tool won't quite fit? Sometimes, if you are
only using a small part of a tool's ability, writing custom code you
can malleate later may prove cheaper.

 

[Arne Vajhøj]

I think we need to split the stuff in 3 parts:
A) server side framework to enable declarative JAX-RS to work in a
servlet container
B) the JSON/POX serializer
C) client side framework

re A)

I don't hear you argue against that. And I don't recall much
criticism from other either.

re B)

You don't like the common libraries. I know several people that
don't like them either (I don't have so much personal experience).

But is it really the concept that is wrong or is it just the
implementations?

My guess is still the implementations. I am not too keen on
toJSON, toXML, toJSONAlternative, toXMLALternative etc.etc.
on all DTO's.

We may not always like SOAP and all the associated standards,
but sometimes the "there is only one right way" philosophy
do make life easier.

re C)

I think it is rather common to use plain HttpClient.

Non-Java client to Java service is probably also a very
common combination.

For all of A, B, C, I have no problem with JSON as a data notation. I
have no major problems with SOAP as a protocol, or with the principles
of true RESTful web services [1].

For all of A, B, C, it is specifically implementations that I have a
problem with. Whether language-specific APIs or libraries. JSON is
simple. SOAP *messages* are usually quite simple (WSDL and schemas might
not be, but you infrequently occupy your time with those). JSON as a
payload over HTTP as part of a REST method is simple. SOAP XML via HTTP,
JMS, SMTP is usually quite simple.

*When we look at what is going back and forth*, that's what is simple.
What can be, and often is, elaborately over-engineered, clumsy, buggy,
and not simple are the client and server APIs and libraries.

As to toJSON() or fromXml() type methods, I'm not overly keen on them
either, for maintainability reasons. We very often have to explain to
the client or server framework how to handle the DTO<->JSON/XML
conversions, and annotations are superior to hardcoding for this. Except
when they're not.

Time to write your own annotation processing library??



Arne

 

[Arne Vajhøj]

Programmers seem to prefer to reinvent the wheel than create something
new. Think how many JSP-like beasts we have, XML-like beasts we have,

1? I can not think of anything like XML - neither SGML in general
nor JSON seems to match.

how many IDEs.

The number seems to be constantly shrinking and we have not seen
a new one for many years.

Arne

 

[Arne Vajhøj]

As a model of RPC it makes me want to re-embrace CORBA. It's badly
designed.

Having said that, if you work with simple subsets it's understandable,
robust, and very widely supported. The use of XML is very defensible, IMO.

I've seen plenty of much more terrible protocols. SOAP doesn't even come
close to some of the Cthulhian mind-blasting horrors I've dealt with.

If one expose a SOAP/HTTP service implemented in language X and
generate a client stub for language Y from the WSDL, then there
is a pretty good chance that it will just work.

Arne

 

[Joerg Meier]

Time to write your own annotation processing library??

Even though it was said more in jest, I've been burning to share this
excellent series of blog posts:

<http://deors.wordpress.com/2011/09/26/annotation-types/>


Liebe Gruesse,
Joerg