Class Files and Class Loading in NT and UNIX

[matt melton]

Hello,
I am having a spot of bother transmitting the bytes of a class file
over a TCP Socket from a JVM on Linux to a JVM on NT 4.

If I have compiled the class file on NT I can send it's bytes it to
the Linux machine and use it, without a problem.



If I compile the class file on NT and use a JVM on Linux to send the
class bytes to another JVM on Linux I get "bad magic number". I have
read that this is due to the CAFEBABE bytes at the beginning of the
calss file so I did an octal dump on both class files ( od -x ) on
the nfs filesystem. The mtab file says the file system type is autofs
if that makes any difference.

If I compile the class file on the linux system it can be transmitted
to an NT machine and used, but it still cannot be transmitted to
another Linux machine without the same error occuring.


Here are the truncated octal dumps they appear the same to me,but diff
says that they differ but it does not say where.



Compiled on Linux.

0000000 feca beba 0000 2e00 9200 000a 0034 063c
0000020 0040 0000 0000 0000 000a 003d 093e 3d00
0000040 3f00 000a 0040 0a41 4000 4200 0009 0043
0000060 0744 4500 000a 0009 083c 4600 000a 0009
0000100 0a47 0900 4800 0008 0a49 0900 4a00 000a
0000120 004b 064c f13f 9999 9999 9a99 000a 004d
0000140 064e 6f40 00e0 0000 0000 000a 004b 064f
....


Compiled on NT.

0000000 feca beba 0000 2e00 9200 000a 0034 063c
0000020 0040 0000 0000 0000 000a 003d 093e 3d00
0000040 3f00 000a 0040 0a41 4000 4200 0009 0043
0000060 0744 4500 000a 0009 083c 4600 000a 0009
0000100 0a47 0900 4800 0008 0a49 0900 4a00 000a
0000120 004b 064c f13f 9999 9999 9a99 000a 004d
0000140 064e 6f40 00e0 0000 0000 000a 004b 064f

....


do I need to reorder the bytes if I am reading from a Linux or NFS
filesystem before I transmit them.

I assumed that the two systems would work the same though I have
limited knowledge of the filesystems and how java would read in form
each. I notice that the cafe babe is in a different byte ordering. I
transfer my files so taht I can access them from Linux using a Mapped
drive on NT to the Samba Server NFS.


Any help would be greatly appreciated. In the mean time I will
attempt to reorder the bytes and send the class files, and post if
this works.

Thanks for any help.

MAtthew Melton

 

[Gordon Beaton]

I am having a spot of bother transmitting the bytes of a class file
over a TCP Socket from a JVM on Linux to a JVM on NT 4.

If I have compiled the class file on NT I can send it's bytes it to
the Linux machine and use it, without a problem.

If I compile the class file on NT and use a JVM on Linux to send the
class bytes to another JVM on Linux I get "bad magic number".

You should be able to use the same identical classfiles on both
systems, without any changes to byte order. The byte order used in the
classfile is well defined, i.e. it doesn't just use host byte order.

Perhaps you could describe this "sending" you are referring to. I
understand this to mean something other than simply using a classfile
on a network mounted disk, and I suspect that's where the problem
arises.

Have you written the code you use to transfer your classfiles among
machines? Post it.

/gordon

 

[Sudsy]

Perhaps you could describe this "sending" you are referring to. I
understand this to mean something other than simply using a classfile
on a network mounted disk, and I suspect that's where the problem
arises.

Have you written the code you use to transfer your classfiles among
machines? Post it.

/gordon

Excellent point! I've often seen problems when FTP is used to transfer
files between system in the wrong mode, i.e. ASCII instead of BINARY.

 

[matt melton]

Hello,
Sorry I should have explained my self a bit more clearly.
I am setting up a P2P type distributed processing application, using
java.

The Peers when discovered via Multicast, allow a user to select how
many peers they want to run an application on , and which class file
they want to run.


This class file is read by a FileTransferServer Thread into a byte
array.
The server then waits for the specified number of connections.


When the peers receive a message that they are going to be used, they
attempt to connect to the remote Serving Peer over a TCP Socket using
a RemoteClassLoader class that I have defined. There is a simple
protocol that sends the name of the class being transmitted, the size
in bytes and the data itself. If the data does not arrive intact it is
sent again.
The Linux machines never seem to be able to define the class, wether
it is served by a Linux machine or an NT machine, so it just keeps
requesting a resend.

###########################
The Server code is here:
###########################

public class FileTransferServer extends Thread{
ServerSocket ssocket;
File file ;
int numberToServe;

/** **/

public FileTransferServer( int port , File file , int numberToServe){
try{
if( Debug.DEBUGFILETRANSFERSERVER ){
System.out.println(" FileTransferServer Creating Socket on port "
+ port );
}
ssocket = new ServerSocket( port , 1 );
}
catch( SocketException se ){
System.out.println("FileTransferServer: There appears to be a
socket on " + port + " already");
System.exit(1);
}
catch( IOException ioe ){
System.out.println("FileTransferServer: An IOException occured
opening socket on already");
}
this.file = file;
this.numberToServe = numberToServe;
}

/** Waits for connections ( one at a time at the moment), waits for
the client to send OK then writes the file to the socket. **/

public void run(){
FileInputStream fstream = null;

if( Debug.DEBUGFILETRANSFERSERVER ){
System.out.println("FileTransferServer: Opening file for reading "
+ file.toString() );
}

while( fstream == null){
try{
fstream = new FileInputStream( file );

}
catch( FileNotFoundException fnfe ){
System.out.println("FileTransferServer: that file does not exist
");
System.exit(1);
}
}

// do I need to check wether I have got a huge file???

byte[] bytes = new byte[ (int) file.length() ];

if( Debug.DEBUGFILETRANSFERSERVER ){
System.out.println("FileTransferServer: Reading in " + bytes.length
+ " bytes of data ");
}

int bytesread = 0;

// this shouldn't be a problem , but double checking doesn't hurt.
while( bytesread < ( int ) file.length() ){
try{

bytesread = fstream.read( bytes , 0 , (int) file.length() );


}
catch( FileNotFoundException fnfe ){
System.out.println( "FileTransferServer: The file specified does
not exist.");
System.exit(1);
}
catch( IOException ioe ){
System.out.println( "FileTransferServer: An IOException occured
using this file" );
ioe.printStackTrace();
System.exit(1);
}
finally{
if( fstream != null ){
try{ fstream.close(); } catch( IOException ioe){}
}
}

}



if( Debug.DEBUGFILETRANSFERSERVER ){
System.out.println( "FileTransferServer: Data Read. Accepting
Connections... ");
}



// this is just a basic count we really should have a check list of
Peers that we are expecting to
// contact us....
int numberDelivered = 0;
while( numberToServe > numberDelivered ){
Socket s = null;
BufferedReader breader = null;
BufferedOutputStream bostream = null;
DataOutputStream dstream = null;
PrintWriter pwriter = null;



try{

if( Debug.DEBUGFILETRANSFERSERVER ){
System.out.println("FileTransferServer: NumberDelivered " +
numberDelivered + " of " + numberToServe );

}

s = ssocket.accept();

breader = new BufferedReader( new InputStreamReader(
s.getInputStream() ) );
//bostream = new BufferedOutputStream( );
dstream = new DataOutputStream( s.getOutputStream() );



//should send how many bytes are coming...
int bytesWritten = 0;

boolean done = false;
while( ( bytesWritten < bytes.length ) && !done ){
String response = breader.readLine();


if( response == null ){
break;
}
if( response.equals( "OKSENDNAME" ) ){

String name = file.toString().substring( 0 ,
file.toString().length() - 6);
dstream.writeChars( name + "\n" );
dstream.flush();


if( Debug.DEBUGFILETRANSFERSERVER ){
System.out.println("FileTransferServer: Name Sent");
}

}
else if( response.equals( "OKSENDLENGTH" ) ){
dstream.writeInt( (int) file.length() );
dstream.flush();
if( Debug.DEBUGFILETRANSFERSERVER ){
System.out.println("FileTransferServer: Length Sent");
}
}
else if( response.equals( "OKSENDCLASS" ) ||
response.equals("BADSENDCLASS")){
dstream.write( bytes , 0 , bytes.length );
dstream.flush();

}
else if( response.equals("OKDONE") ){
done = true;
}
}










++numberDelivered;

}
catch(IOException ioe ){
System.out.println( "FileTransferServer: Connection closed");
++numberDelivered;
}
finally{
if( dstream != null ){
try{
dstream.close();
}
catch( IOException ioe ){}
}


if( bostream != null ){

}


if( pwriter != null ){
pwriter.close();
}



if( s != null ){
try{ s.close(); } catch( IOException ioe ){}
}


}

}
}
}


################################################

And here is the RemoteClassLoader class

################################################

public class RemoteClassLoader extends ClassLoader{
Socket socket;
byte[] classBytes;
String className;

public RemoteClassLoader( InetAddress address , int port ){


while( socket == null ){
if( Debug.DEBUGREMOTECLASSLOADER ){
System.out.println( "RemoteClassLoader: Attempting to Connect to "
+ address + " on " + port );
}



try{
socket = new Socket( address , port );
}
catch( SocketException se ){
if( Debug.DEBUGREMOTECLASSLOADER ){
System.out.println( "RemoteClassLoader: Waiting for socket on
Peer " + address + " on Port " + port );
}
}
catch( IOException ioe ){
System.out.println( "RemoteClassLoader: There was an IOException
creating socket on port " + port);
System.exit(1);
}


try{ Thread.sleep( 4000 ); } catch( InterruptedException ie ){}

}


if( Debug.DEBUGREMOTECLASSLOADER ){
System.out.println("RemoteClassLoader: Socket " + socket.toString()
+ " created on port " + socket.getLocalPort() + " \n" +
"is connected to " + socket.getInetAddress() + " on port " +
socket.getPort() );
}


}



private Class getRemoteFile(){
Class result = null;
try{ //new BufferedInputStream(
DataInputStream dstream = new DataInputStream(
socket.getInputStream() );
PrintWriter pwriter = new PrintWriter( socket.getOutputStream() ,
true );
pwriter.println( "OKSENDNAME" );


className = "";
char nextchar = '\n';
while( (nextchar = dstream.readChar() ) != '\n'){
className = className.concat( (new Character(nextchar)).toString()
);
}


if( Debug.DEBUGREMOTECLASSLOADER ){
System.out.println( "RemoteClassLoader: Recieved class name " +
className );
}
pwriter.println( "OKSENDLENGTH" );


int length = dstream.readInt();
classBytes = new byte[ length ];
if( Debug.DEBUGREMOTECLASSLOADER ){
System.out.println( "RemoteClassLoader Received File Length " +
length);
}


pwriter.println( "OKSENDCLASS");
pwriter.flush();

int bytesRead = 0;
boolean complete = false;
while( !complete ){

bytesRead = dstream.read( classBytes , 0 , classBytes.length );



if( bytesRead < classBytes.length ){
pwriter.println( "BADSENDCLASS");
}
else{
try{
result = defineClass( className , classBytes , 0 ,
classBytes.length );
pwriter.println( "OKDONE ");
complete = true;

}
catch( ClassFormatError cfe ){
System.out.println("Broken Class file requesting resend.");
pwriter.println("BADSENDCLASS");
}
}

}

if( Debug.DEBUGREMOTECLASSLOADER ){
System.out.println( "RemoteClassLoader Received " + bytesRead + "
bytes" );
}
}
catch( IOException ioe ){
System.out.println("RemoteClassLoader: IOEexception thrown opening
readers on Socket " );
ioe.printStackTrace();
}
finally{
if( socket != null ){
try{ socket.close(); }catch( IOException ioe ){}
}
}


return result;


}



public Class retrieveClass(){
if( Debug.DEBUGREMOTECLASSLOADER ){
System.out.println("RemoteClassLoader: Starting Download");
}

return getRemoteFile();
}


}

###################################################

I have tried compiling the class file I am sending on both a linux
machine and an NT machine, just in case something funky was happening
but I seem to get the same result.

The problem only seems to occur when, a linux machine is serving the
class. The NT Machines can always read the class and instantiate an
object from it.

The linux machines can instantiate the class only if it is served by
an NT machine.

I am a little confused by this. It seems like an odd bug to have.
Unless I am ding something daft and have gone code blind ( Which
wouldn't be th first time ).


Thanks for your help.

Matthew Melton

 

[Gordon Beaton]

[ code ]

Ok I had a *quick* look through the code and off hand noticed
something...

Your server uses a loop like the following to read the classfile:

while (bytesread < ( int )file.length()) {
bytesread = fstream.read(bytes, 0, (int) file.length());
}

This will only work correctly if you manage to read the *entire* file
in one gulp. The second pass through the loop (if any) will overwrite
the start of the buffer. You need to increment the starting offset as
well as reduce the amount of data you request from fstream.read():

bytesread += fstream.read(bytes, bytesRead, (int)file.length() - bytesRead);

Then when you send the data, you should check the return value from
dstream.write() and loop if necessary. Don't assume that write() will
send all of the data at once.

Similarly, the client expects the entire classfile in a single read:

boolean complete = false;
while(!complete) {
bytesRead = dstream.read(classBytes, 0, classBytes.length);

if (bytesRead != classBytes.length) {
// error
}
else {
complete = true;
}
}

Here I'd use a test more like the one used by the server, i.e. compare
the read bytes with the expected number, changing the offset and the
request size on each call to dstream.read():

while (bytesRead < classBytes.length) {
bytesRead += dstream.read(classBytes , bytesRead , classBytes.length - bytesRead);
}

I don't know if this is where your problem is, but I'd suggest that
you debug the file transfer part separately from the classloader
logic. Can you transfer and store the classfile, then compare the
result with the original?

/gordon

 

[matt melton]

hello

Thanks for your help,
I changed my reading method as you suggested, and it worked first
time,

I did a little more research and it appears that Linux, will preempt a
thread that is reading, so that only a smaller amount of bytes will be
read. Apparently this does not happen on NT, so that is why the NT
computers worked the whole time and the linux ones didn't.

My faith in Java portability has been restored, ( I have apologised to
my Boxes for my language).

Thanks again

Matthew Melton

 

[Gordon Beaton]

I did a little more research and it appears that Linux, will preempt
a thread that is reading, so that only a smaller amount of bytes
will be read. Apparently this does not happen on NT, so that is why
the NT computers worked the whole time and the linux ones didn't.

I think you'll find that read() from a SocketInputStream isn't
guaranteed to be atomic on NT either. Whether it seems to be in some
cases depends on the size of the request and the relative speeds of
the reader, the writer and the network.

/gordon