Class Files and Class Loading in NT and UNIX

Discussion in 'Java' started by matt melton, Sep 5, 2003.

  1. matt melton

    matt melton Guest

    Hello,
    I am having a spot of bother transmitting the bytes of a class file
    over a TCP Socket from a JVM on Linux to a JVM on NT 4.

    If I have compiled the class file on NT I can send it's bytes it to
    the Linux machine and use it, without a problem.



    If I compile the class file on NT and use a JVM on Linux to send the
    class bytes to another JVM on Linux I get "bad magic number". I have
    read that this is due to the CAFEBABE bytes at the beginning of the
    calss file so I did an octal dump on both class files ( od -x ) on
    the nfs filesystem. The mtab file says the file system type is autofs
    if that makes any difference.

    If I compile the class file on the linux system it can be transmitted
    to an NT machine and used, but it still cannot be transmitted to
    another Linux machine without the same error occuring.


    Here are the truncated octal dumps they appear the same to me,but diff
    says that they differ but it does not say where.



    Compiled on Linux.

    0000000 feca beba 0000 2e00 9200 000a 0034 063c
    0000020 0040 0000 0000 0000 000a 003d 093e 3d00
    0000040 3f00 000a 0040 0a41 4000 4200 0009 0043
    0000060 0744 4500 000a 0009 083c 4600 000a 0009
    0000100 0a47 0900 4800 0008 0a49 0900 4a00 000a
    0000120 004b 064c f13f 9999 9999 9a99 000a 004d
    0000140 064e 6f40 00e0 0000 0000 000a 004b 064f
    ....


    Compiled on NT.

    0000000 feca beba 0000 2e00 9200 000a 0034 063c
    0000020 0040 0000 0000 0000 000a 003d 093e 3d00
    0000040 3f00 000a 0040 0a41 4000 4200 0009 0043
    0000060 0744 4500 000a 0009 083c 4600 000a 0009
    0000100 0a47 0900 4800 0008 0a49 0900 4a00 000a
    0000120 004b 064c f13f 9999 9999 9a99 000a 004d
    0000140 064e 6f40 00e0 0000 0000 000a 004b 064f

    ....


    do I need to reorder the bytes if I am reading from a Linux or NFS
    filesystem before I transmit them.

    I assumed that the two systems would work the same though I have
    limited knowledge of the filesystems and how java would read in form
    each. I notice that the cafe babe is in a different byte ordering. I
    transfer my files so taht I can access them from Linux using a Mapped
    drive on NT to the Samba Server NFS.


    Any help would be greatly appreciated. In the mean time I will
    attempt to reorder the bytes and send the class files, and post if
    this works.

    Thanks for any help.

    MAtthew Melton
     
    matt melton, Sep 5, 2003
    #1
    1. Advertising

  2. On 5 Sep 2003 04:06:57 -0700, matt melton wrote:
    > I am having a spot of bother transmitting the bytes of a class file
    > over a TCP Socket from a JVM on Linux to a JVM on NT 4.
    >
    > If I have compiled the class file on NT I can send it's bytes it to
    > the Linux machine and use it, without a problem.
    >
    > If I compile the class file on NT and use a JVM on Linux to send the
    > class bytes to another JVM on Linux I get "bad magic number".


    You should be able to use the same identical classfiles on both
    systems, without any changes to byte order. The byte order used in the
    classfile is well defined, i.e. it doesn't just use host byte order.

    Perhaps you could describe this "sending" you are referring to. I
    understand this to mean something other than simply using a classfile
    on a network mounted disk, and I suspect that's where the problem
    arises.

    Have you written the code you use to transfer your classfiles among
    machines? Post it.

    /gordon

    --
    [ do not email me copies of your followups ]
    g o r d o n + n e w s @ b a l d e r 1 3 . s e
     
    Gordon Beaton, Sep 5, 2003
    #2
    1. Advertising

  3. matt melton

    Sudsy Guest

    Gordon Beaton wrote:
    > On 5 Sep 2003 04:06:57 -0700, matt melton wrote:
    >
    >>I am having a spot of bother transmitting the bytes of a class file
    >>over a TCP Socket from a JVM on Linux to a JVM on NT 4.

    <snip>

    > Perhaps you could describe this "sending" you are referring to. I
    > understand this to mean something other than simply using a classfile
    > on a network mounted disk, and I suspect that's where the problem
    > arises.
    >
    > Have you written the code you use to transfer your classfiles among
    > machines? Post it.
    >
    > /gordon


    Excellent point! I've often seen problems when FTP is used to transfer
    files between system in the wrong mode, i.e. ASCII instead of BINARY.
     
    Sudsy, Sep 5, 2003
    #3
  4. matt melton

    matt melton Guest

    Hello,
    Sorry I should have explained my self a bit more clearly.
    I am setting up a P2P type distributed processing application, using
    java.

    The Peers when discovered via Multicast, allow a user to select how
    many peers they want to run an application on , and which class file
    they want to run.


    This class file is read by a FileTransferServer Thread into a byte
    array.
    The server then waits for the specified number of connections.


    When the peers receive a message that they are going to be used, they
    attempt to connect to the remote Serving Peer over a TCP Socket using
    a RemoteClassLoader class that I have defined. There is a simple
    protocol that sends the name of the class being transmitted, the size
    in bytes and the data itself. If the data does not arrive intact it is
    sent again.
    The Linux machines never seem to be able to define the class, wether
    it is served by a Linux machine or an NT machine, so it just keeps
    requesting a resend.

    ###########################
    The Server code is here:
    ###########################

    public class FileTransferServer extends Thread{
    ServerSocket ssocket;
    File file ;
    int numberToServe;

    /** **/

    public FileTransferServer( int port , File file , int numberToServe){
    try{
    if( Debug.DEBUGFILETRANSFERSERVER ){
    System.out.println(" FileTransferServer Creating Socket on port "
    + port );
    }
    ssocket = new ServerSocket( port , 1 );
    }
    catch( SocketException se ){
    System.out.println("FileTransferServer: There appears to be a
    socket on " + port + " already");
    System.exit(1);
    }
    catch( IOException ioe ){
    System.out.println("FileTransferServer: An IOException occured
    opening socket on already");
    }
    this.file = file;
    this.numberToServe = numberToServe;
    }

    /** Waits for connections ( one at a time at the moment), waits for
    the client to send OK then writes the file to the socket. **/

    public void run(){
    FileInputStream fstream = null;

    if( Debug.DEBUGFILETRANSFERSERVER ){
    System.out.println("FileTransferServer: Opening file for reading "
    + file.toString() );
    }

    while( fstream == null){
    try{
    fstream = new FileInputStream( file );

    }
    catch( FileNotFoundException fnfe ){
    System.out.println("FileTransferServer: that file does not exist
    ");
    System.exit(1);
    }
    }

    // do I need to check wether I have got a huge file???

    byte[] bytes = new byte[ (int) file.length() ];

    if( Debug.DEBUGFILETRANSFERSERVER ){
    System.out.println("FileTransferServer: Reading in " + bytes.length
    + " bytes of data ");
    }

    int bytesread = 0;

    // this shouldn't be a problem , but double checking doesn't hurt.
    while( bytesread < ( int ) file.length() ){
    try{

    bytesread = fstream.read( bytes , 0 , (int) file.length() );


    }
    catch( FileNotFoundException fnfe ){
    System.out.println( "FileTransferServer: The file specified does
    not exist.");
    System.exit(1);
    }
    catch( IOException ioe ){
    System.out.println( "FileTransferServer: An IOException occured
    using this file" );
    ioe.printStackTrace();
    System.exit(1);
    }
    finally{
    if( fstream != null ){
    try{ fstream.close(); } catch( IOException ioe){}
    }
    }

    }



    if( Debug.DEBUGFILETRANSFERSERVER ){
    System.out.println( "FileTransferServer: Data Read. Accepting
    Connections... ");
    }



    // this is just a basic count we really should have a check list of
    Peers that we are expecting to
    // contact us....
    int numberDelivered = 0;
    while( numberToServe > numberDelivered ){
    Socket s = null;
    BufferedReader breader = null;
    BufferedOutputStream bostream = null;
    DataOutputStream dstream = null;
    PrintWriter pwriter = null;



    try{

    if( Debug.DEBUGFILETRANSFERSERVER ){
    System.out.println("FileTransferServer: NumberDelivered " +
    numberDelivered + " of " + numberToServe );

    }

    s = ssocket.accept();

    breader = new BufferedReader( new InputStreamReader(
    s.getInputStream() ) );
    //bostream = new BufferedOutputStream( );
    dstream = new DataOutputStream( s.getOutputStream() );



    //should send how many bytes are coming...
    int bytesWritten = 0;

    boolean done = false;
    while( ( bytesWritten < bytes.length ) && !done ){
    String response = breader.readLine();


    if( response == null ){
    break;
    }
    if( response.equals( "OKSENDNAME" ) ){

    String name = file.toString().substring( 0 ,
    file.toString().length() - 6);
    dstream.writeChars( name + "\n" );
    dstream.flush();


    if( Debug.DEBUGFILETRANSFERSERVER ){
    System.out.println("FileTransferServer: Name Sent");
    }

    }
    else if( response.equals( "OKSENDLENGTH" ) ){
    dstream.writeInt( (int) file.length() );
    dstream.flush();
    if( Debug.DEBUGFILETRANSFERSERVER ){
    System.out.println("FileTransferServer: Length Sent");
    }
    }
    else if( response.equals( "OKSENDCLASS" ) ||
    response.equals("BADSENDCLASS")){
    dstream.write( bytes , 0 , bytes.length );
    dstream.flush();

    }
    else if( response.equals("OKDONE") ){
    done = true;
    }
    }










    ++numberDelivered;

    }
    catch(IOException ioe ){
    System.out.println( "FileTransferServer: Connection closed");
    ++numberDelivered;
    }
    finally{
    if( dstream != null ){
    try{
    dstream.close();
    }
    catch( IOException ioe ){}
    }


    if( bostream != null ){

    }


    if( pwriter != null ){
    pwriter.close();
    }



    if( s != null ){
    try{ s.close(); } catch( IOException ioe ){}
    }


    }

    }
    }
    }


    ################################################

    And here is the RemoteClassLoader class

    ################################################

    public class RemoteClassLoader extends ClassLoader{
    Socket socket;
    byte[] classBytes;
    String className;

    public RemoteClassLoader( InetAddress address , int port ){


    while( socket == null ){
    if( Debug.DEBUGREMOTECLASSLOADER ){
    System.out.println( "RemoteClassLoader: Attempting to Connect to "
    + address + " on " + port );
    }



    try{
    socket = new Socket( address , port );
    }
    catch( SocketException se ){
    if( Debug.DEBUGREMOTECLASSLOADER ){
    System.out.println( "RemoteClassLoader: Waiting for socket on
    Peer " + address + " on Port " + port );
    }
    }
    catch( IOException ioe ){
    System.out.println( "RemoteClassLoader: There was an IOException
    creating socket on port " + port);
    System.exit(1);
    }


    try{ Thread.sleep( 4000 ); } catch( InterruptedException ie ){}

    }


    if( Debug.DEBUGREMOTECLASSLOADER ){
    System.out.println("RemoteClassLoader: Socket " + socket.toString()
    + " created on port " + socket.getLocalPort() + " \n" +
    "is connected to " + socket.getInetAddress() + " on port " +
    socket.getPort() );
    }


    }



    private Class getRemoteFile(){
    Class result = null;
    try{ //new BufferedInputStream(
    DataInputStream dstream = new DataInputStream(
    socket.getInputStream() );
    PrintWriter pwriter = new PrintWriter( socket.getOutputStream() ,
    true );
    pwriter.println( "OKSENDNAME" );


    className = "";
    char nextchar = '\n';
    while( (nextchar = dstream.readChar() ) != '\n'){
    className = className.concat( (new Character(nextchar)).toString()
    );
    }


    if( Debug.DEBUGREMOTECLASSLOADER ){
    System.out.println( "RemoteClassLoader: Recieved class name " +
    className );
    }
    pwriter.println( "OKSENDLENGTH" );


    int length = dstream.readInt();
    classBytes = new byte[ length ];
    if( Debug.DEBUGREMOTECLASSLOADER ){
    System.out.println( "RemoteClassLoader Received File Length " +
    length);
    }


    pwriter.println( "OKSENDCLASS");
    pwriter.flush();

    int bytesRead = 0;
    boolean complete = false;
    while( !complete ){

    bytesRead = dstream.read( classBytes , 0 , classBytes.length );



    if( bytesRead < classBytes.length ){
    pwriter.println( "BADSENDCLASS");
    }
    else{
    try{
    result = defineClass( className , classBytes , 0 ,
    classBytes.length );
    pwriter.println( "OKDONE ");
    complete = true;

    }
    catch( ClassFormatError cfe ){
    System.out.println("Broken Class file requesting resend.");
    pwriter.println("BADSENDCLASS");
    }
    }

    }

    if( Debug.DEBUGREMOTECLASSLOADER ){
    System.out.println( "RemoteClassLoader Received " + bytesRead + "
    bytes" );
    }
    }
    catch( IOException ioe ){
    System.out.println("RemoteClassLoader: IOEexception thrown opening
    readers on Socket " );
    ioe.printStackTrace();
    }
    finally{
    if( socket != null ){
    try{ socket.close(); }catch( IOException ioe ){}
    }
    }


    return result;


    }



    public Class retrieveClass(){
    if( Debug.DEBUGREMOTECLASSLOADER ){
    System.out.println("RemoteClassLoader: Starting Download");
    }

    return getRemoteFile();
    }


    }

    ###################################################

    I have tried compiling the class file I am sending on both a linux
    machine and an NT machine, just in case something funky was happening
    but I seem to get the same result.

    The problem only seems to occur when, a linux machine is serving the
    class. The NT Machines can always read the class and instantiate an
    object from it.

    The linux machines can instantiate the class only if it is served by
    an NT machine.

    I am a little confused by this. It seems like an odd bug to have.
    Unless I am ding something daft and have gone code blind ( Which
    wouldn't be th first time ).


    Thanks for your help.

    Matthew Melton
     
    matt melton, Sep 6, 2003
    #4
  5. On 6 Sep 2003 03:06:15 -0700, matt melton wrote:
    > [ code ]


    Ok I had a *quick* look through the code and off hand noticed
    something...

    Your server uses a loop like the following to read the classfile:

    while (bytesread < ( int )file.length()) {
    bytesread = fstream.read(bytes, 0, (int) file.length());
    }

    This will only work correctly if you manage to read the *entire* file
    in one gulp. The second pass through the loop (if any) will overwrite
    the start of the buffer. You need to increment the starting offset as
    well as reduce the amount of data you request from fstream.read():

    bytesread += fstream.read(bytes, bytesRead, (int)file.length() - bytesRead);

    Then when you send the data, you should check the return value from
    dstream.write() and loop if necessary. Don't assume that write() will
    send all of the data at once.

    Similarly, the client expects the entire classfile in a single read:

    boolean complete = false;
    while(!complete) {
    bytesRead = dstream.read(classBytes, 0, classBytes.length);

    if (bytesRead != classBytes.length) {
    // error
    }
    else {
    complete = true;
    }
    }

    Here I'd use a test more like the one used by the server, i.e. compare
    the read bytes with the expected number, changing the offset and the
    request size on each call to dstream.read():

    while (bytesRead < classBytes.length) {
    bytesRead += dstream.read(classBytes , bytesRead , classBytes.length - bytesRead);
    }

    I don't know if this is where your problem is, but I'd suggest that
    you debug the file transfer part separately from the classloader
    logic. Can you transfer and store the classfile, then compare the
    result with the original?

    /gordon

    --
    [ do not email me copies of your followups ]
    g o r d o n + n e w s @ b a l d e r 1 3 . s e
     
    Gordon Beaton, Sep 6, 2003
    #5
  6. matt melton

    matt melton Guest

    hello

    Thanks for your help,
    I changed my reading method as you suggested, and it worked first
    time,

    I did a little more research and it appears that Linux, will preempt a
    thread that is reading, so that only a smaller amount of bytes will be
    read. Apparently this does not happen on NT, so that is why the NT
    computers worked the whole time and the linux ones didn't.

    My faith in Java portability has been restored, ( I have apologised to
    my Boxes for my language).

    Thanks again

    Matthew Melton
     
    matt melton, Sep 8, 2003
    #6
  7. On 8 Sep 2003 02:26:46 -0700, matt melton wrote:
    > I did a little more research and it appears that Linux, will preempt
    > a thread that is reading, so that only a smaller amount of bytes
    > will be read. Apparently this does not happen on NT, so that is why
    > the NT computers worked the whole time and the linux ones didn't.


    I think you'll find that read() from a SocketInputStream isn't
    guaranteed to be atomic on NT either. Whether it seems to be in some
    cases depends on the size of the request and the relative speeds of
    the reader, the writer and the network.

    /gordon

    --
    [ do not email me copies of your followups ]
    g o r d o n + n e w s @ b a l d e r 1 3 . s e
     
    Gordon Beaton, Sep 8, 2003
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    12
    Views:
    1,654
    Dave Thompson
    Jan 10, 2005
  2. Replies:
    18
    Views:
    626
    Dave Thompson
    Jan 10, 2005
  3. Replies:
    3
    Views:
    1,310
    Walter Roberson
    May 1, 2006
  4. Robert Wallace

    my own perl "dos->unix"/"unix->dos"

    Robert Wallace, Jan 21, 2004, in forum: Perl Misc
    Replies:
    7
    Views:
    284
    Michele Dondi
    Jan 22, 2004
  5. Replies:
    4
    Views:
    312
Loading...

Share This Page