servlet/applet communication problem or Linux/Windows trouble ?

[Guest]

I have since some time a bunch of cooperating servlets and applets. They
have been happily used generally by myself or by colleagues here in the
institute (on the same LAN), more rarely by colleagues in another
institute in the same city. Now I am trying to use them with a colleague
in another country (Belgium vs Italy) and I am having trouble.

I do not understand whether it is a bug in my code, a problem of "noise"
on the line, or an incompatibility between the Windows client in Belgium
with our Linux system (so far only Linux clients have used it).

------------------------------------------------------------------------
The arrangement and the protocol

- the user is presented with a HTML frameset with two frames. The left
frame contains a little form which has the servlet (running on my
machine) as action. The right hand frame has initially a button
used to load a page with the applet (but this is done later).

The user presses the Connect button in the left frame and this calls
the servlet.

- the servlet init method does a
if (serverSocket == null) serverSocket = new ServerSocket(14444);

(I have three servlets, one uses port 4444, one 14444 and one 54444,
I just picked up random unprivileged ports. The one I'm now testing
is the one with 14444

The servlet doPost or doGet methods (equivalent) ultimately process
the Connect request doing ...

clientSocket = (Socket) session.getValue("clientsocket") ;
if (clientSocket == null || clientSocket.isClosed()) {
clientSocket = serverSocket.accept();
...
}

i.e. a message is issued, then the servlet listens on port 14444
until a connection request is received, and opens a clientSocket.
(there is a separate one per session, kept in a session Vector)

this occurs when the user loads the HTML page with the applet

at the same time the servlet opens a JDBC connection to
a mysql server on another machine (local database server in our
institute)

- the start method of the applet attaches the client socket and
opens two streams on it

if( mySocket == null) mySocket = new
Socket("mymachine.mydomain.inaf.it", 14444);
if (appletout == null) appletout = new
PrintWriter(mySocket.getOutputStream(), true);
if (appletin == null) appletin = new BufferedReader(new
InputStreamReader(mySocket.getInputStream()));

- the user then inputs a number in the left hand side form and presses
a Go button. As result the servlet does a database search for item
"number", does other chores, then displays on the user screen a
"locked waiting for applet requests" and enters a loop reading from
the applet output stream (which is input for the servlet)

- the user can press one of three buttons in the applet (get image,
get regions, close communication).

correspondlingy the applet printlns on its output stream a
command (GOIMA, GOREG or END)

- if the servlet receives a GOIMA, sends to the applet IMGREADY

- when the applet receives IMGREADY sends back to the servlet
a SENDURL

- when the servlet receives SENDURL sends back a number N followed
by n lines each one containing an URL

- the applet reads and stores the N URLs (and uses one of them to
retrieve via http an image which it displays ... from another
"image" server)

- if the servlet receives a GOREG, sends to the applet REGREADY

- when the applet receives REGREADY sends back to the servlet
a SENDREG

- when the servlet receives SENDREG sends back a number M followed
by m lines each one containing some text

- the applet reads, parses and stores the m lines in a JTable
and does something else with them too

- if the servlet receives an END, it exits from the communication loop
and writes an "unlocked" message to the user screen

- if the user presses the Disconnect button in the left frame, the
servlet closes the clientSocket streams, cleans up the session
vectors, and closes the mysql connection

------------------------------------------------------------------------
The tests

- all this has always been working locally since ages

- when I had one external user (elsewhere in town) I had to tune
somehow the delays. Also he originally had a firewall blocking
port 14444 so he had to open it

- when now I had troubles with a user in Belgium, I thought it
was another firewall problem. So I asked her to do a test I
already used in the past :

- one has to start the servlet "instance" using the left hand
side form, but WITHOUT starting the applet ...

... instead of this she telnets to my machine on port 14444
and issues the GOIMA SENDURL GOREG SENDREG END commands to
exercise the handshake (giving normal servlet commands from
the form)

- while this is done, I keep a watch (lsof -i loop) on the sockets
for the servlet user and pid

apparently the "typical course of events" I see is (if I exercise
this locally)

- Connect button
I see port 14444 in LISTEN
I see an arbitrary port attached to the db host mysql port
I see my port http-alt (?) attached to an arbitrary high port

- telnet 14444 or start applet

The db mysql port remains
The rest is replaced by my 14444 attached to an arbitrary high port

Now, apart why exactly http-alt, this is clear and clean

------------------------------------------------------------------------
The problem

My user in Belgium can run the telnet test with no problems (i.e. the
servlet replies correctly, there are no packets lost, the handshake is
fine).

However when she starts the applet and then tries to exercise the
handshake under real conditions I see funny things on my ports.

I initially see a connection from my http-alt to some arbitrary
port on her system (which my lsof decodes with funny names like
cogitate, tomato-springs, or other names each time different).

Also typically one sees the port attached to http-alt shift from
one name to another but not go away, while 14444 attaches to
another arbitrary port with a funny name

... after this I guess the system gets confused as there are
two sockets between applet and servlet and the commands are
routed in some random way

My correspondent is using Firefox/3.0.5 under Windows XP 2002 SP2,
if that matters anyhow.

Any obvious suggestion that this behaviour might be due to a
Windows/Linux incompatibility ? Or to a bug in my code (but it did work
with other Linux users) ?

If anybody wishing to have a test run, we could negotiate access to the
real thing (although it accesses proprietary astronomical data, I guess
it could be arranged).

 

[John B. Matthews]

I initially see a connection from my http-alt to some arbitrary
port on her system (which my lsof decodes with funny names like
cogitate, tomato-springs, or other names each time different).

FWIW, http-alt is a Well Known Port, while cogitate & tomato-springs are
all Registered Ports:

<http://www.iana.org/assignments/port-numbers>

 

[markspace]

I initially see a connection from my http-alt to some arbitrary
port on her system (which my lsof decodes with funny names like
cogitate, tomato-springs, or other names each time different).

As John pointed out, these are registered ports. However, that means
little, your system can re-use the port numbers with out problems. I'm
surprised lsof is bothering to list ports by name, there's only a very
small chance this information will be useful.

I think these "funny names" are just a red-herring and not relevant to
your problem at all.

Any obvious suggestion that this behaviour might be due to a
Windows/Linux incompatibility ? Or to a bug in my code (but it did work
with other Linux users) ?

What I'm seeing mostly here is a lack of any sort of log file or other
actual debugging mechanism. Dumping port connections is fine to
establish that the software can talk, but it doesn't actually tell us
anything, or at least doesn't tell me anything. You should make some
kind of log of the session that you can review. This is the only way
that I know of short of attaching a debugger.

It could be a compatibility problem. Java mostly gets around that, but
clever programmers can still find lots of ways to introduce bugs.

Since you've established that the systems can connect (open ports), I
think the next step is to add some debugging information to the servlet
to dump session information, so you can have some hope of debugging the
problem (adding logs to the applet will be harder). Assume it's
software and debug from that point of view. Bonus points if the logs
can be changed per user, so you don't have to write reams of logs for
other users that are working correctly.

 

[Guest]

LC's No-Spam Newsreading account wrote:

As John pointed out, these are registered ports. However, that means
little, your system can re-use the port numbers with out problems.

Not sure about the technical meaning of "registered ports". I assumed
lsof is translating port numbers into whatever finds in /etc/services.

What puzzled me is that if I run the "foreign application" (i.e. the
applet) locally, I see its ports are some arbitrary high numbered port
(which seems fine to me). When the applet runs remotely, I see the odd
names (which according to /etc/services are relatively low numbered
ports) ... so I suppose it is the remote system which is offering such
port number for the client sockets.

I think these "funny names" are just a red-herring and not relevant to
your problem at all.

Essentially you are saying that using such low numbered registered
ports is harmless. Correct ?

Is http-alt something which java (or JSDK/jsdk2.1 servlets) does
routinely use when listening on a server socket ?

What I'm seeing mostly here is a lack of any sort of log file or other
actual debugging mechanism. Dumping port connections is fine to
establish that the software can talk [...] Since you've established
that the systems can connect (open ports)

well, on one hand I've established more than that. Not only ports are
open, but the servlet can respond correctly to the simple handshake
protocol I designed when the remote client is a manual telnet session on
the designated port.

On the other hand, I do have logging facilities :

- I start the servlet in a dedicated account ssh'd in a dedicated
window. The servlet writes to System.out all what it is doing.
So I can see there what it does.

- The servlet duplicates the same information to the out stream of
the http request. So the remote user can see (in the left hand
frame of the frameset) what the servlet is doing.

- the applet is a rather complex arrangement (a tabbed pane with an
image area, a JTable, several controls AND a scrollable message
area. I have a button which enables verbose messages, so that the
applet wirtes to the message area what it is doing.

The point is that all this shows things are working perfectly when I run
the applet locally. When it is run remotely simply it gets stuck at some
point.

So I FIRST had the logging, and since that did not give me any clue, I
did lsof port monitoring.

The other peculiarity I can mention is that the applet runs natively in
my (rather old) firefox on Linux, while my remote user had to install
some java plugin to run it in her firefox on Windows.

 

[Lew]

The other peculiarity I can mention is that the applet runs natively in
my (rather old) firefox on Linux, while my remote user had to install
some java [sic] plugin to run it in her firefox on Windows.

Java doesn't run natively. Your Firefox already had the plugin, that's all.

 

[markspace]

Essentially you are saying that using such low numbered registered
ports is harmless. Correct ?

Port numbers should not matter. It's just a port. If you can talk on
it at all, then it's fine.

The point is that all this shows things are working perfectly when I run
the applet locally. When it is run remotely simply it gets stuck at some
point.

You need to find more precisely what that point is. I don't think a log
window for the user to look at is going to help here. You'll need finer
grained info. Either use a SocketHandler in you app to send logs
directly to the server, or write logs to a file and ask the client to
upload (or email) them to you.

After that, the next step is to attach a debugger directly to the remote
system and step through the code yourself.

 

[John B. Matthews]

LC's No-Spam Newsreading account wrote:

[...]
Not sure about the technical meaning of "registered ports".

IIUC, the distinction between Well Known Port and Registered Port
reflects the Unix dichotomy between privileged ports and user ports:

I assumed lsof is translating port numbers into whatever [it]
finds in /etc/services.

Yes, `man lsof` mentions this.

What puzzled me is that if I run the "foreign application" (i.e. the
applet) locally, I see its ports are some arbitrary high numbered
port (which seems fine to me). When the applet runs remotely, I see
the odd names (which according to /etc/services are relatively low
numbered ports) ... so I suppose it is the remote system which is
offering such port number for the client sockets.


Essentially you are saying that using such low numbered registered
ports is harmless. Correct?

I was intrigued to learn that Windows does not implement the notion of
privileged ports; such low numbers may be normal:

Is http-alt something which java (or JSDK/jsdk2.1 servlets) does
routinely use when listening on a server socket?

AFAIK, Java allocates such ports using the host OS. The default range
for dynamic ports on Windows appears to be 1025-5000:

<http://www.hsc.fr/ressources/articles/win_net_srv/ephem_port_alloc.html>

Is there any evidence that the affected host has been altered in this
regard?

[...]

 

[Guest]

Either use a SocketHandler in you app to send logs directly to the
server,

I've looked up SocketHandler's and found on Sun Tech Tips a simple
LogServer, which listens on ANOTHER port of my machine (5000). I
modified it to print also the host and port it connects to it.

I ran the exerciser program from my machine and another local one, to
verify such LogServer could receive connections from two hosts at a
time.

I modified my servlet (see thread) to log messages on the socket
handler. I arranged it so that it tests at the beginning whether it can
connect to the log server (if not the servlet tells me on System.out),
sets a boolean, and then a doLog method logs messages only if such
boolean is true.

The idea is that it can run without recompiling, and attaches to the log
server only if I have started it because I need it.

I then modified my applet to also log messages on the socket handler.
I arranged it to test if it can connect to the log server as for the
servlet.

When I tested the applet from the standard web page, I got no log
messages. So I started the applet from the appletviewer (in order to see
stdout messages).

I get "access denied (java.util.logging.LoggingPermission control)"

The way the servlet or applet start logging is in an initLog() method
whose core is

try {
Handler handler = new SocketHandler("myownhost", 5000);
handler.setFormatter(new SimpleFormatter());
logger.addHandler(handler);
logger.setLevel(Level.ALL);
logEnabled = true;
} catch (Exception e) { ...

I call initLog in the init method of the servlet and in the init method
of the applet JUST BEFORE the standard swing idiom ...

initLog();
doLog(Level.WARNING,"applet init");
try {
SwingUtilities.invokeAndWait(new Runnable() {
public void run() {
realMain() ;
}
});
} catch (Exception e) { ...

what's wrong with permissions ?
the applet usually attaches happily to other ports. Other apps attach
happily to port 5000.

I'll investigate tomorrow, but if anybody is around and can give an hint
while I go home for sleep, it will be much appreciated

 

[Guest]

I've looked up SocketHandler's and found on Sun Tech Tips a simple LogServer,
which listens on ANOTHER port of my machine (5000).

I modified my servlet (see thread) to log messages on the socket handler.
I then modified my applet to also log messages on the socket handler.

[...] from the appletviewer [...]
I get "access denied (java.util.logging.LoggingPermission control)"

what's wrong with permissions ?

Well, I found it. NOTHING was wrong with permissions !

I was simply misled by the way the log server worked. Since it did work
running the test exerciser IN SEQUENCE from two hosts, I assumed it
could share port 5000 between n>1 connections.

This is instead not the case ! If the servlet connects to port 5000 of
the log server (and remains running as it should be), the applet can't
connect.

I spent most of the morning playing with the java.policy file to grant
the logging permission to the applet (testing it with the appletviewer),
but finally found some old notes of mine, which said that java.policy is
unnecessary when the applet is signed (as it is) and runs in the
browser.

So I replaced the stdout messages with popups, and saw that the applet
was not denied by the log server, and did issue log messages, simply
they weren't received !

Or actually, under particular conditions, they were received, but the
servlet one weren't. The first app which got hold of port 5000
controlled the log server.

The solution (to the logging issue ... the debugging of the real
problem will come later using the log files, I hope) has been to have
two separate log servers on port 5000 and 5001, and having servlet and
applet using separate log ports !

(probably I'd write a little [Fortran, do not laugh] utility to merge
the two log files based on time stamps !)