Shortage of qualified Java programmers

C

Chris Smith

Pete Barrett said:
Sometimes it has to be. There's one particular server I have to
connect to and send XML via an HTTP request. If I assemble the
request, headers and body together, and then flush the stream, it
doesn't work (reading the response times out); if I write the headers,
then flush, then write the body, then flush, it works. Don't ask me
why - it's their server!

I can tell you why with a fair degree of certainty. The programmers who
wrote the server didn't bother to learn TCP. If you send header and
body content in one message, then they get confused and don't deal with
it properly.

The technique you've chosen is a good one (I'd also set TCP_NODELAY on
the socket to be extra safe), and probably the only reasonable response.
But at least you can understand what's wrong. Perhaps you can even let
them know that their server is broken.

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation
 
T

Tris Orendorff

[...]
When I first started this process I was actually concerned that such
questions might be too easy for programmers with 3+ years Java
experience. In fact the opposite has turned out to be the case. Some
candidates even seem surprised that I expect them to know the core
Java API and be able to use it to solve actual problems. I could
understand if I were asking about obscure stuff like reference queues,
but my questions generally pertain to basic aspects of the Java API
that I use frequently in my work.

At the risk of tarred and feathered by the readers of this group, I suggest that you check to see if they have
passed any of the Java certification tests from Sun and others. They are two to three hour multiple-choice
exams where you need to know the basic APIs, threads, collections, syntax vs. run-time errors, I/O,
exceptions, etc. like the back of your hand.

--
Sincerely,

Tris Orendorff
[Two antennae meet on a roof, fall in love and get married. The ceremony wasn't much, but the reception
was excellent.]
 
B

Bjorn Borud

[Chris Smith <[email protected]>]
|
| I can tell you why with a fair degree of certainty. The programmers
| who wrote the server didn't bother to learn TCP. If you send header
| and body content in one message, then they get confused and don't
| deal with it properly.

hmm, I am not sure I follow you here. what do you mean when you say
"in one message"?

-Bjørn
 
B

Bjorn Borud

[Tris Orendorff <[email protected]>]
|
| At the risk of tarred and feathered by the readers of this group, I
| suggest that you check to see if they have passed any of the Java
| certification tests from Sun and others. They are two to three hour
| multiple-choice exams where you need to know the basic APIs,
| threads, collections, syntax vs. run-time errors, I/O, exceptions,
| etc. like the back of your hand.

are there any examples on-line of what these tests look like?

-Bjørn
 
C

Chris Smith

Bjorn Borud said:
[Chris Smith <[email protected]>]
|
| I can tell you why with a fair degree of certainty. The programmers
| who wrote the server didn't bother to learn TCP. If you send header
| and body content in one message, then they get confused and don't
| deal with it properly.

hmm, I am not sure I follow you here. what do you mean when you say
"in one message"?

Sorry, I mistyped. I meant "in one IP packet".

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation
 
B

Bjorn Borud

[Chris Smith <[email protected]>]
| > [Chris Smith <[email protected]>]
| > |
| > | I can tell you why with a fair degree of certainty. The programmers
| > | who wrote the server didn't bother to learn TCP. If you send header
| > | and body content in one message, then they get confused and don't
| > | deal with it properly.
| >
| > hmm, I am not sure I follow you here. what do you mean when you say
| > "in one message"?
|
| Sorry, I mistyped. I meant "in one IP packet".

so what you are saying is that in order for the server to work
properly it is important that the header and body not be sent in
separate IP packets but in the same packet?

uh, you've lost me. how is this important? and how do you guarantee
it?

-Bjørn
 
B

Bjorn Borud

[Alan Krueger <[email protected]>]
|
| I know what he's referring to. It's when the developer mistakenly
| believes that the read()/write() on either side of the stream are
| coordinated. A write() on one side may correspond to zero, one, or
| many read() calls on the other side, not exactly one.

I think we've established that two posts ago. I was trying to provoke
him into thinking.

-Bjørn
 
A

Aquila Deus

eglato said:
Lately I have been conducting technical interviews to fill some Java
developer positions, and this process is becoming highly frustrating. Good
candidates are very hard to come by in the Atlanta area. I don't know
whether there is an actual shortage of Java programmers in the U.S., or
whether the situation is unique to our metro area, or whether we are just
relying on the wrong recruiting companies. In any case I repeatedly find
that a candidate will list every Java technology in the world on his
resume, but then when I quiz him on specifics he can't answer. Or perhaps
he knows the buzzwords, but then when I ask him how he would apply the
concept in a specific situation he gets it all wrong.

Here is a sampling of some of the questions I like to ask candidates:

* Write a program that reads strings from a text file, one per line, and
outputs a list of the unique strings along with their frequency of
occurrence. For example, this input:

Tuesday
Friday
Tuesday
Saturday

results in this output:

Tuesday: 2
Friday: 1
Saturday: 1

(To do this right they need to know about FileReader, BufferedReader, and
some collections API stuff.)

* Write a program to compute the number of hours in the current day,
correctly accounting for locale-specific time changes that might make the
current day more or less than 24 hours in length. (Simple Date/Calendar
API test.)

* Suppose I have the following class:
public class A { public String key; }
Write the code to sort a list of A's in ascending order by "key" attribute
using Collections.sort(). (Basic Comparator test).

When I first started this process I was actually concerned that such
questions might be too easy for programmers with 3+ years Java experience.
In fact the opposite has turned out to be the case. Some candidates even
seem surprised that I expect them to know the core Java API and be able to
use it to solve actual problems. I could understand if I were asking about
obscure stuff like reference queues, but my questions generally pertain to
basic aspects of the Java API that I use frequently in my work.

Is my experience typical? I read a lot about unemployment in the IT
field, but it just seems to me that if unemployment were a significant
problem there would be at least somewhat competent people sending out
resumes looking for work, and I'm just not finding that to be the case. Of
course my experience is limited to the Java job market in the Atlanta area
and I can't speak for other technologies or locales.

ehhh... what about their OO skills?
 
A

Alan Krueger

Bjorn said:
so what you are saying is that in order for the server to work
properly it is important that the header and body not be sent in
separate IP packets but in the same packet?

uh, you've lost me. how is this important?

Because it's broken behavior.
and how do you guarantee it?

You can't, that's why it's broken.

I know what he's referring to. It's when the developer mistakenly
believes that the read()/write() on either side of the stream are
coordinated. A write() on one side may correspond to zero, one, or many
read() calls on the other side, not exactly one.
 
A

Aquila Deus

Pete said:
Sometimes it has to be. There's one particular server I have to
connect to and send XML via an HTTP request. If I assemble the
request, headers and body together, and then flush the stream, it
doesn't work (reading the response times out); if I write the headers,
then flush, then write the body, then flush, it works. Don't ask me
why - it's their server!

hmmmm... maybe their server does this:

read() -- for request
read() -- for headers
while (read()) -- for body

if so then the server is obviously coded by very inexperienced
developers... and need a rewrite!

similiar problem occurs in one of our recent projects as well (the dev
has left so I can criticize now :), he thought it's ok because there is
no error shown during several weeks of (manned) tests....
 
C

Chris Uppal

so what you are saying is that in order for the server to work
properly it is important that the header and body not be sent in
separate IP packets but in the same packet?

Other way around, from the sound of it. The server-side code "expects" there
to be a package boundary between the header and body, presumably a read() into
a large buffer that they brokenly assume will not contain the whole header plus
(some of) the body.

Particularly easy to get into that state if the server-side code uses buffered
input, but once it has read the header from its buffers, it then passes the raw
socket connection to (say) another process that will set up its own buffer
around the socket and read from that. Since the first process may have read
more than just the header into its buffers, the second process will not see the
start of the body.

The result of combining two perfectly normal and correct strategies (buffering
input and handing off processing to another process) is to create a broken
server.

-- chris
 
C

Chris Smith

Bjorn Borud said:
so what you are saying is that in order for the server to work
properly it is important that the header and body not be sent in
separate IP packets but in the same packet?

My guess is that it's the opposite. The server probably reads the
headers, then later on reads the body. That's okay, except that the
server is assuming that the last header read() call will end after the
header is completed. That's not necessarily the case; it is quite
possible that a single IP packet contains *both* the end of the header
and the beginning of the body.

This is all speculation on my part... but it's a common enough mistake
by TCP/IP programmers in C, and it fits your symptoms perfectly, so I'm
fairly confident in saying that this is the problem. It is NOT your
bug, but you can attempt to work around it. The result is probablistic
(i.e., not guaranteed to work) but it represents a decent shot at
solving the problem. To do so:

1. Use a BufferedOutputStream or BufferedWriter, with plenty of buffer
space to hold the entire HTTP header section at once. (Unless you're
doing something odd, the default buffer size should be fine.)

2. Disable Nagle's algorithm on the output socket by calling
setTcpNoDelay(true) on the Socket class BEFORE connecting.

3. Write the header section first, and then call flush() on the stream.

4. Write the body after the flush() call.

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation
 
B

Bjorn Borud

["Chris Uppal" <[email protected]>]
|
| Other way around, from the sound of it. The server-side code
| "expects" there to be a package boundary between the header and
| body, [...]

it is vital to understand that in TCP there is no concept of packet
boundaries available to the user. none. to the user, TCP is a stream-
based transport that makes no guarantees as to how the data is sliced
and diced at the system call interfaces and in transit. the only
thing it tries to guarantee is that the same sequence of bytes arrive
in the same order.

robust implementations must take this into account.

-Bjørn
 
B

Bjorn Borud

[Chris Smith <[email protected]>]
|
| Excuse me? How does learning TCP help you to deal with that broken
| server?

the way to do it is to fix the broken server. not to add to the
problem by kludging the client code. you are not *fixing* things by
kludging it, you are just adding to the problem.

-Bjørn
 
C

Chris Smith

Bjorn Borud said:
no, learn about TCP instead of fumbling blindly in the dark.

Excuse me? How does learning TCP help you to deal with that broken
server? We already know the server is broken. What we're attempting to
do is suggest a way for the client to act in a way that's least likely
to trigger that broken behavior. You can't just fix the server by fiat
alone.

May I suggest that you take the time to understand what's being said
before condescending to others? You are being very rude.

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation
 
D

David Alex Lamb

[Chris Smith <[email protected]>]
|
| Excuse me? How does learning TCP help you to deal with that broken
| server?

the way to do it is to fix the broken server. not to add to the
problem by kludging the client code. you are not *fixing* things by
kludging it, you are just adding to the problem.

And what if one has no power to make those responsible for the server change
anything in timely fashion?
 
A

Alan Krueger

Aquila said:
hmmmm... maybe their server does this:

read() -- for request
read() -- for headers
while (read()) -- for body

if so then the server is obviously coded by very inexperienced
developers... and need a rewrite!

similiar problem occurs in one of our recent projects as well (the dev
has left so I can criticize now :), he thought it's ok because there is
no error shown during several weeks of (manned) tests....

In our development shop, consultants were hired to build an asynchronous
messaging communications framework.

However, the consultants hired to do this used TCP and drove the entire
event loop off a single read(), so that every once in a while a larger
message would only be partially read. The stream would then be out of
sync and crash or need to be rebooted.

Not sure why, but this always seems to happen with people who leave and
don't have to clean up the mess left behind.
 
B

Bjorn Borud

[[email protected] (David Alex Lamb)]
|
| And what if one has no power to make those responsible for the
| server change anything in timely fashion?

then you need to ask yourself how it is that you are wasting time on a
product built by people who do not know what they are doing. I can
certainly relate to people's urge to "fix things", but this isn't
fixing things. sorry.

-Bjørn
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top