Is it possible to "decipher" Java serialization data?

[ohaya]

Hi,

I have a file which contains the contents of a binary attribute from an
LDAP directory, and I've been trying to "decipher" the contents of this
attribute. After a bit of searching around, it appeared that this
attribute looks basically like a serialized Java object. I went to
http://www.iconv.com/file.htm, and pointed it my dump/file, and sure
enough, it came back with "Java serialization data, version 5".

I was wondering if there is either a tool, or if it would be possible to
write a small Java program to "decipher" this file, i.e., such that I
could print/display the variables/objects that are in the file? I'm not
looking for anything really fancy, but need to be able to see the
variable contents and, if possible, the associated variable names.

I'm not an experienced Java programmer, so my apologies if this is a
somewhat vague question..

For your reference, here's a partial hex dump of one of files:

00000000 AC ED 00 05 74 00 0F 72-65 71 5F 69 73 73 75 65
.....t..req_issue
00000010 64 5F 63 65 72 74 75 72-00 02 5B 42 AC F3 17 F8
d_certur..[B....
00000020 06 08 54 E0 02 00 00 78-70 00 00 03 08 AC ED 00
...T....xp.......
00000030 05 73 72 00 33 6E 65 74-73 63 61 70 65 2E 73 65
..sr.3netscape.se
00000040 63 75 72 69 74 79 2E 78-35 30 39 2E 58 35 30 39
curity.x509.X509
00000050 43 65 72 74 49 6D 70 6C-24 43 65 72 74 69 66 69
CertImpl$Certifi
00000060 63 61 74 65 52 65 70 31-B7 B9 E3 43 D3 33 68 27
cateRep1...C.3h'
00000070 02 00 02 5B 00 05 64 61-74 61 31 74 00 02 5B 42
....[..data1t..[B
00000080 4C 00 05 74 79 70 65 31-74 00 12 4C 6A 61 76 61
L..type1t..Ljava
00000090 2F 6C 61 6E 67 2F 53 74-72 69 6E 67 3B 78 70 75
/lang/String;xpu
000000A0 72 00 02 5B 42 AC F3 17-F8 06 08 54 E0 02 00 00
r..[B......T....
000000B0 78 70 00 00 02 77 30 82-02 73 30 82 01 DC A0 03
xp...w0..s0.....
000000C0 02 01 02 02 01 0A 30 0D-06 09 2A 86 48 86 F7 0D
.......0...*.H...
000000D0 01 01 05 05 00 30 6D 31-0B 30 09 06 03 55 04 06
......0m1.0...U..
000000E0 13 02 55 53 31 0B 30 09-06 03 55 04 08 13 02 56
...US1.0...U....V
000000F0 41 31 0F 30 0D 06 03 55-04 07 13 06 4F 61 6B 74
A1.0...U....Oakt
00000100 6F 6E 31 0E 30 0C 06 03-55 04 0A 13 05 4A 69 6D
on1.0...U....Jim
00000110 43 6F 31 12 30 10 06 03-55 04 0B 13 09 54 65 73
Co1.0...U....Tes
00000120 74 20 44 65 70 74 31 1C-30 1A 06 03 55 04 03 13 t
Dept1.0...U...
00000130 13 43 65 72 74 69 66 69-63 61 74 65 20 4D 61 6E .Certificate
Man
00000140 61 67 65 72 30 1E 17 0D-30 34 30 36 32 39 32 31
ager0...04062921
00000150 35 30 31 37 5A 17 0D 30-34 31 32 32 36 32 31 35
5017Z..041226215
00000160 30 31 37 5A 30 6B 31 17-30 15 06 03 55 04 03 13
017Z0k1.0...U...
00000170 0E 6A 69 6D 6E 65 77 2E-66 6F 6F 2E 63 6F 6D 31
..jimnew.foo.com1
00000180 14 30 12 06 03 55 04 0B-13 0B 54 65 73 74 20 44
..0...U....Test D
00000190 65 70 74 20 33 31 0F 30-0D 06 03 55 04 0A 13 06 ept
31.0...U....
..
<snip>
..
00000320 63 E9 50 4D FC 24 10 A8-7A DA 4D C3 8C 74 00 05
c.PM.$..z.M..t..
00000330 58 2E 35 30 39 74 00 0C-72 65 71 75 65 73 74 4E
X.509t..requestN
00000340 6F 74 65 73 75 71 00 7E-00 01 00 00 00 28 AC ED
otesuq.~.....(..
00000350 00 05 74 00 21 54 45 53-54 20 53 45 52 56 45 52 ..t.!TEST
SERVER
00000360 20 43 45 52 54 20 46 4F-52 20 52 45 51 55 45 53 CERT FOR
REQUES
00000370 54 20 23 31 30 37 74 00-0B 72 65 71 75 65 73 74 T
#107t..request
00000380 54 79 70 65 75 71 00 7E-00 01 00 00 00 11 AC ED
Typeuq.~........
00000390 00 05 74 00 0A 65 6E 72-6F 6C 6C 6D 65 6E 74 74
...t..enrollmentt
000003A0 00 10 69 73 45 6E 63 72-79 70 74 69 6F 6E 43 65
...isEncryptionCe
000003B0 72 74 75 71 00 7E 00 01-00 00 00 0B AC ED 00 05
rtuq.~..........
000003C0 74 00 04 74 72 75 65 74-00 0C 63 65 72 74 5F 72
t..truet..cert_r
000003D0 65 71 75 65 73 74 75 71-00 7E 00 01 00 00 04 DF
equestuq.~......
000003E0 AC ED 00 05 74 04 D8 2D-2D 2D 2D 2D 42 45 47 49
.....t..-----BEGI
000003F0 4E 20 4E 45 57 20 43 45-52 54 49 46 49 43 41 54 N NEW
CERTIFICAT
00000400 45 20 52 45 51 55 45 53-54 2D 2D 2D 2D 2D 0D 0A E
REQUEST-----..
00000410 4D 49 49 44 52 6A 43 43-41 71 38 43 41 51 41 77
MIIDRjCCAq8CAQAw
00000420 61 7A 45 58 4D 42 55 47-41 31 55 45 41 78 4D 4F
azEXMBUGA1UEAxMO
00000430 61 6D 6C 74 62 6D 56 33-4C 6D 5A 76 62 79 35 6A
amltbmV3LmZvby5j
00000440 62 32 30 78 46 44 41 53-42 67 4E 56 42 41 73 54
b20xFDASBgNVBAsT
00000450 0D 0A 43 31 52 6C 63 33-51 67 52 47 56 77 64 43
...C1Rlc3QgRGVwdC
00000460 41 7A 4D 51 38 77 44 51-59 44 56 51 51 4B 45 77
AzMQ8wDQYDVQQKEw
..
<snip>
..
00000820 4A 53 47 44 32 5A 6C 59-32 31 49 69 7A 69 0D 0A
JSGD2ZlY21Iizi..
00000830 44 54 50 67 5A 69 37 78-61 61 61 4E 4A 72 5A 7A
DTPgZi7xaaaNJrZz
00000840 42 39 75 73 57 70 44 49-50 70 79 36 51 4E 7A 48
B9usWpDIPpy6QNzH
00000850 4A 72 67 55 79 66 5A 49-79 52 69 4F 55 61 4E 6B
JrgUyfZIyRiOUaNk
00000860 51 52 33 41 57 34 5A 35-32 61 71 33 32 61 74 59
QR3AW4Z52aq32atY
00000870 0D 0A 4C 67 69 39 74 50-74 38 47 77 6D 30 58 4C
...Lgi9tPt8Gwm0XL
00000880 39 4A 63 45 4E 48 75 72-79 55 6B 6F 74 54 6E 73
9JcENHuryUkotTns
00000890 72 48 6F 51 6B 3D 0D 0A-2D 2D 2D 2D 2D 45 4E 44
rHoQk=..-----END
000008A0 20 4E 45 57 20 43 45 52-54 49 46 49 43 41 54 45 NEW
CERTIFICATE
000008B0 20 52 45 51 55 45 53 54-2D 2D 2D 2D 2D 0D 0A 74
REQUEST-----..t
000008C0 00 07 70 72 6F 66 69 6C-65 75 71 00 7E 00 01 00
...profileuq.~...
000008D0 00 00 0B AC ED 00 05 74-00 04 74 72 75 65 74 00
........t..truet.
000008E0 11 63 65 72 74 5F 72 65-71 75 65 73 74 5F 74 79
..cert_request_ty
000008F0 70 65 75 71 00 7E 00 01-00 00 00 0D AC ED 00 05
peuq.~..........
00000900 74 00 06 70 6B 63 73 31-30 74 00 0F 72 65 71 75
t..pkcs10t..requ
00000910 65 73 74 6F 72 5F 70 68-6F 6E 65 75 71 00 7E 00
estor_phoneuq.~.
00000920 01 00 00 00 13 AC ED 00-05 74 00 0C 31 30 37 2D
..........t..107-
..
<snip>
..
00001370 3B 78 70 77 6D 30 6B 31-17 30 15 06 03 55 04 03
;xpwm0k1.0...U..
00001380 13 0E 6A 69 6D 6E 65 77-2E 66 6F 6F 2E 63 6F 6D
...jimnew.foo.com
00001390 31 14 30 12 06 03 55 04-0B 13 0B 54 65 73 74 20
1.0...U....Test
000013A0 44 65 70 74 20 33 31 0F-30 0D 06 03 55 04 0A 13 Dept
31.0...U...
000013B0 06 4A 69 6D 63 6F 33 31-0F 30 0D 06 03 55 04 07
..Jimco31.0...U..
000013C0 13 06 4F 61 6B 74 6F 6E-31 0B 30 09 06 03 55 04
...Oakton1.0...U.
000013D0 08 13 02 56 41 31 0B 30-09 06 03 55 04 06 13 02
....VA1.0...U....
000013E0 55 53 78 74 00 0C 70 72-6F 66 69 6C 65 53 65 74
USxt..profileSet
000013F0 49 64 75 71 00 7E 00 01-00 00 00 0B AC ED 00 05
Iduq.~..........
00001400 74 00 04 73 65 74 31 70-
t..set1p

 

[Roedy Green]

I'm not an experienced Java programmer, so my apologies if this is a
somewhat vague question..

It is a good question. One of the big objections people have to
serialized or other binary formats in the way they have got stung in
past not knowing which class files are needed to reconstitute a
stream, and perhaps no longer having them since they have evolved.
People are often sloppy about documenting binary formats or tracking
which formats belong with which files. I hope some day that
information will be as much as part of the file as the timestamp.

Bill Wilkinson did a fair bit of poking around to figure out how
serialisation works. Perhaps he has some notes buried he would share.

I have been trying for a long time to get him to publish his promised
essay. Perhaps if you ask too.

See http://mindprod.com/jgloss/serialization.html

Pass on what you find. Perhaps I can work this up into a student
project outline.

First you want to figure out just what is in the pickled stream. I
think it may well be enough to reconstruct some Java source for the
various classes (without the transients or methods). At least then you
could recover the data.

Figure this out by serialising some trivial objects and gradually
adding complexity. You will need a hex viewer. See
http://mindprod.com/jgloss/hex.html

 

[ohaya]

It is a good question. One of the big objections people have to
serialized or other binary formats in the way they have got stung in
past not knowing which class files are needed to reconstitute a
stream, and perhaps no longer having them since they have evolved.
People are often sloppy about documenting binary formats or tracking
which formats belong with which files. I hope some day that
information will be as much as part of the file as the timestamp.

Bill Wilkinson did a fair bit of poking around to figure out how
serialisation works. Perhaps he has some notes buried he would share.

I have been trying for a long time to get him to publish his promised
essay. Perhaps if you ask too.

See http://mindprod.com/jgloss/serialization.html

Pass on what you find. Perhaps I can work this up into a student
project outline.

First you want to figure out just what is in the pickled stream. I
think it may well be enough to reconstruct some Java source for the
various classes (without the transients or methods). At least then you
could recover the data.

Figure this out by serialising some trivial objects and gradually
adding complexity. You will need a hex viewer. See
http://mindprod.com/jgloss/hex.html

Hi Roedy,

I don't know if I'm going on the wrong tack, but I wrote a small test
program that SEEMS to be displaying the variable names extracted from
the blob/file. I'm attaching a copy of the program below, along with
output from when I ran it against a test blob/file.

If this program is doing what I think that it's doing, then what I need
now (besides a stiff drink) is to figure out how to extract the variable
contents from the blob/file.

If anyone has any ideas, please post!!

Thanks,
Jim


Here's the program:

import java.io.*;
import java.util.*;
import java.ibjectInputStream.*;

public class test5 {

public static void main(String[] aArguments) {
try {
FileInputStream fis = new FileInputStream("22.ser");
ObjectInputStream ois = new ObjectInputStream(fis);

boolean done = false;
Object x;
String y;

while (done == false) {
x = ois.readObject();
if (x != null) {
y = x.toString();
System.out.println("Field: [" + y + "]");
if (y.equals("requestor_name")) {
System.out.println("Other contents: [" +
ois.readObject() + "]\n");
} else {
System.out.println("String contents: ["
+ ois.readObject() + "]\n");
}
} else {
done = true;
}
}


} catch (IOException e) {System.out.println(e);}
catch (ClassNotFoundException e) {System.out.println("Class
error");}

}


}


And, here's the output:

Field: [req_issued_cert]
String contents: [[B@5601ea]

Field: [requestNotes]
String contents: [[B@17d257]

Field: [requestType]
String contents: [[B@7259da]

Field: [isEncryptionCert]
String contents: [[B@6930e2]

Field: [cert_request]
String contents: [[B@8786b]

Field: [profile]
String contents: [[B@19c082]

Field: [cert_request_type]
String contents: [[B@2dd2dd]

Field: [requestor_phone]
String contents: [[B@6ee36c]

Field: [req_locale]
String contents: [[B@14df86]

Field: [req_key]
String contents: [[B@5efa1a]

Field: [profileRemoteAddr]
String contents: [[B@75da06]

Field: [requestor_name]
Other contents: [[B@3d0108]

Field: [profileApprovedBy]
String contents: [[B@ed465]

Field: [requestor_email]
String contents: [[B@765291]

Field: [profileId]
String contents: [[B@26e431]

Field: [profileRemoteHost]
String contents: [[B@4f8dab]

Field: [req_x509info]
String contents: [[B@5debc3]

Field: [req_seq_num]
String contents: [[B@218aa2]

Field: [requestId]
String contents: [[B@14ca6c]

Field: [requestVersion]
String contents: [[B@7590db]

Field: [dbStatus]
String contents: [[B@7943a4]

Field: [requestStatus]
String contents: [[B@480457]

Field: [req_extensions]
String contents: [[B@14fe5c]

Field: [updatedBy]
String contents: [[B@47858e]

Field: [req_subject_name]
String contents: [[B@1134f4]

Field: [profileSetId]
String contents: [[B@2bbd86]

 

[ohaya]

It is a good question. One of the big objections people have to
serialized or other binary formats in the way they have got stung in
past not knowing which class files are needed to reconstitute a
stream, and perhaps no longer having them since they have evolved.
People are often sloppy about documenting binary formats or tracking
which formats belong with which files. I hope some day that
information will be as much as part of the file as the timestamp.

Bill Wilkinson did a fair bit of poking around to figure out how
serialisation works. Perhaps he has some notes buried he would share.

I have been trying for a long time to get him to publish his promised
essay. Perhaps if you ask too.

See http://mindprod.com/jgloss/serialization.html

Pass on what you find. Perhaps I can work this up into a student
project outline.

First you want to figure out just what is in the pickled stream. I
think it may well be enough to reconstruct some Java source for the
various classes (without the transients or methods). At least then you
could recover the data.

Figure this out by serialising some trivial objects and gradually
adding complexity. You will need a hex viewer. See
http://mindprod.com/jgloss/hex.html

Hi Roedy,

I don't know if I'm going on the wrong tack, but I wrote a small test
program that SEEMS to be displaying the variable names extracted from
the blob/file. I'm attaching a copy of the program below, along with
output from when I ran it against a test blob/file.

If this program is doing what I think that it's doing, then what I need
now (besides a stiff drink) is to figure out how to extract the variable
contents from the blob/file.

If anyone has any ideas, please post!!

Thanks,
Jim

Here's the program:

import java.io.*;
import java.util.*;
import java.ibjectInputStream.*;

public class test5 {

public static void main(String[] aArguments) {
try {
FileInputStream fis = new FileInputStream("22.ser");
ObjectInputStream ois = new ObjectInputStream(fis);

boolean done = false;
Object x;
String y;

while (done == false) {
x = ois.readObject();
if (x != null) {
y = x.toString();
System.out.println("Field: [" + y + "]");
if (y.equals("requestor_name")) {
System.out.println("Other contents: [" +
ois.readObject() + "]\n");
} else {
System.out.println("String contents: ["
+ ois.readObject() + "]\n");
}
} else {
done = true;
}
}

} catch (IOException e) {System.out.println(e);}
catch (ClassNotFoundException e) {System.out.println("Class
error");}

}

}

And, here's the output:

Field: [req_issued_cert]
String contents: [[B@5601ea]

Field: [requestNotes]
String contents: [[B@17d257]

Field: [requestType]
String contents: [[B@7259da]

Field: [isEncryptionCert]
String contents: [[B@6930e2]

Field: [cert_request]
String contents: [[B@8786b]

Field: [profile]
String contents: [[B@19c082]

Field: [cert_request_type]
String contents: [[B@2dd2dd]

Field: [requestor_phone]
String contents: [[B@6ee36c]

Field: [req_locale]
String contents: [[B@14df86]

Field: [req_key]
String contents: [[B@5efa1a]

Field: [profileRemoteAddr]
String contents: [[B@75da06]

Field: [requestor_name]
Other contents: [[B@3d0108]

Field: [profileApprovedBy]
String contents: [[B@ed465]

Field: [requestor_email]
String contents: [[B@765291]

Field: [profileId]
String contents: [[B@26e431]

Field: [profileRemoteHost]
String contents: [[B@4f8dab]

Field: [req_x509info]
String contents: [[B@5debc3]

Field: [req_seq_num]
String contents: [[B@218aa2]

Field: [requestId]
String contents: [[B@14ca6c]

Field: [requestVersion]
String contents: [[B@7590db]

Field: [dbStatus]
String contents: [[B@7943a4]

Field: [requestStatus]
String contents: [[B@480457]

Field: [req_extensions]
String contents: [[B@14fe5c]

Field: [updatedBy]
String contents: [[B@47858e]

Field: [req_subject_name]
String contents: [[B@1134f4]

Field: [profileSetId]
String contents: [[B@2bbd86]

Hi,

I was just doing some reading about what actually goes on when Java
serializes, and I was wondering...

Is it possible that the reason that I'm getting what appears to be
garbage in the "String Contents" might be that the program that is
creating the blob may be using an older version of
JRE/JVM/Java/whatever?

I was reading "Core Java" book, and it mentions that the format for the
output from serialization has changed over time. What I'm thinking is
if I'm running J2SDK 1.4.whatever, and, say, the system that is
producing the blob is using an older version of Java, and if the older
version serialized in a different format than the newer version I'm
using, would that be causing this problem???

Thanks,
Jim

 

[Roedy Green]

Is it possible that the reason that I'm getting what appears to be
garbage in the "String Contents" might be that the program that is
creating the blob may be using an older version of
JRE/JVM/Java/whatever?

You want to start with something self contained that both generates a
KNOWN serial file, and reads it and decodes it. Once you get that
going, you can tackle the more difficult question of reading your
unknown serial file.

Also too if you want us to help you we need something that generates
the file and decodes it to follow along with you.

 

[ohaya]

quoted :

I'm not an experienced Java programmer, so my apologies if this is a
somewhat vague question..

It is a good question. One of the big objections people have to
serialized or other binary formats in the way they have got stung in
past not knowing which class files are needed to reconstitute a
stream, and perhaps no longer having them since they have evolved.
People are often sloppy about documenting binary formats or tracking
which formats belong with which files. I hope some day that
information will be as much as part of the file as the timestamp.

Bill Wilkinson did a fair bit of poking around to figure out how
serialisation works. Perhaps he has some notes buried he would share.

I have been trying for a long time to get him to publish his promised
essay. Perhaps if you ask too.

See http://mindprod.com/jgloss/serialization.html

Pass on what you find. Perhaps I can work this up into a student
project outline.

First you want to figure out just what is in the pickled stream. I
think it may well be enough to reconstruct some Java source for the
various classes (without the transients or methods). At least then you
could recover the data.

Figure this out by serialising some trivial objects and gradually
adding complexity. You will need a hex viewer. See
http://mindprod.com/jgloss/hex.html

Hi Roedy,

I don't know if I'm going on the wrong tack, but I wrote a small test
program that SEEMS to be displaying the variable names extracted from
the blob/file. I'm attaching a copy of the program below, along with
output from when I ran it against a test blob/file.

If this program is doing what I think that it's doing, then what I need
now (besides a stiff drink) is to figure out how to extract the variable
contents from the blob/file.

If anyone has any ideas, please post!!

Thanks,
Jim

Here's the program:

import java.io.*;
import java.util.*;
import java.ibjectInputStream.*;

public class test5 {

public static void main(String[] aArguments) {
try {
FileInputStream fis = new FileInputStream("22.ser");
ObjectInputStream ois = new ObjectInputStream(fis);

boolean done = false;
Object x;
String y;

while (done == false) {
x = ois.readObject();
if (x != null) {
y = x.toString();
System.out.println("Field: [" + y + "]");
if (y.equals("requestor_name")) {
System.out.println("Other contents: [" +
ois.readObject() + "]\n");
} else {
System.out.println("String contents: ["
+ ois.readObject() + "]\n");
}
} else {
done = true;
}
}

} catch (IOException e) {System.out.println(e);}
catch (ClassNotFoundException e) {System.out.println("Class
error");}

}

}

And, here's the output:

Field: [req_issued_cert]
String contents: [[B@5601ea]

Field: [requestNotes]
String contents: [[B@17d257]

Field: [requestType]
String contents: [[B@7259da]

Field: [isEncryptionCert]
String contents: [[B@6930e2]

Field: [cert_request]
String contents: [[B@8786b]

Field: [profile]
String contents: [[B@19c082]

Field: [cert_request_type]
String contents: [[B@2dd2dd]

Field: [requestor_phone]
String contents: [[B@6ee36c]

Field: [req_locale]
String contents: [[B@14df86]

Field: [req_key]
String contents: [[B@5efa1a]

Field: [profileRemoteAddr]
String contents: [[B@75da06]

Field: [requestor_name]
Other contents: [[B@3d0108]

Field: [profileApprovedBy]
String contents: [[B@ed465]

Field: [requestor_email]
String contents: [[B@765291]

Field: [profileId]
String contents: [[B@26e431]

Field: [profileRemoteHost]
String contents: [[B@4f8dab]

Field: [req_x509info]
String contents: [[B@5debc3]

Field: [req_seq_num]
String contents: [[B@218aa2]

Field: [requestId]
String contents: [[B@14ca6c]

Field: [requestVersion]
String contents: [[B@7590db]

Field: [dbStatus]
String contents: [[B@7943a4]

Field: [requestStatus]
String contents: [[B@480457]

Field: [req_extensions]
String contents: [[B@14fe5c]

Field: [updatedBy]
String contents: [[B@47858e]

Field: [req_subject_name]
String contents: [[B@1134f4]

Field: [profileSetId]
String contents: [[B@2bbd86]

Hi,

I was just doing some reading about what actually goes on when Java
serializes, and I was wondering...

Is it possible that the reason that I'm getting what appears to be
garbage in the "String Contents" might be that the program that is
creating the blob may be using an older version of
JRE/JVM/Java/whatever?

I was reading "Core Java" book, and it mentions that the format for the
output from serialization has changed over time. What I'm thinking is
if I'm running J2SDK 1.4.whatever, and, say, the system that is
producing the blob is using an older version of Java, and if the older
version serialized in a different format than the newer version I'm
using, would that be causing this problem???

Thanks,
Jim

Hi,

Umm. My theory above is probably not the problem. I just looked at a
hex dump of the original test file vs. a file that I created on my
desktop with a small Java program, and it LOOKS like they're the same
version.

Jim

 

[ohaya]

You want to start with something self contained that both generates a
KNOWN serial file, and reads it and decodes it. Once you get that
going, you can tackle the more difficult question of reading your
unknown serial file.

Also too if you want us to help you we need something that generates
the file and decodes it to follow along with you.

Roedy,

The application that created the blobs that I'm trying to decode is
Netscape's Certificate Management System - V6.2. The blob/file that I
am using to test the program that I posted earlier was retrieved from
the LDAP directory that's associated with the CMS.

I'm trying to decode the blob/file using the program that I posted
earlier in the thread.

Jim

 

[Roedy Green]

The application that created the blobs that I'm trying to decode is
Netscape's Certificate Management System - V6.2.

That's fine as your goal. But to understand how this works we need to
start with decoding something simpler, something with simple known
structure with some completely self contained code that any newbie can
run. The CODE must generate the file on each test.

 

[Roedy Green]

Pass on what you find. Perhaps I can work this up into a student
project outline.

see http://mindprod.com/projects/serializedrecovery.html

 

[ohaya]

That's fine as your goal. But to understand how this works we need to
start with decoding something simpler, something with simple known
structure with some completely self contained code that any newbie can
run. The CODE must generate the file on each test.

Hi Roedy,

I did as you suggested, and put together 2 small programs, save.java and
load.java, that saved a class called "UserInfo". I can create a file
called "test.ser" by running save.java, and then when I run load.java,
it displays the right information.

Ok, so here's the strange thing. If I run the program I posted earlier
(test5.java), all I get is basically garbage !

I really don't understand what's going on here, but I'm kind of coming
to the conclusion that when I thought my earlier program was kind of
half working, that was just a fluke (i.e., it wasn't really working).

At this point, I want to try to go ahead as you suggested, using my
save/load programs, but I don't quite understand what I should be
looking for in the output file.

Thanks for your patience...

Jim

P.S. Here's the code for save.java, load.java, and UserInfo.java:

save.java:


import java.io.*;
import java.util.*;
import java.ibjectInputStream.*;

public class save {

public static void main(String[] aArguments) {
try {
FileOutputStream fis = new FileOutputStream("test.ser");
ObjectOutputStream ois = new ObjectOutputStream(fis);
UserInfo user = new UserInfo();
user.setUserName("ABC");
user.setUserPassword("DEF");
ois.writeObject(user);
ois.flush();
ois.close();
} catch (IOException e) {System.out.println(e);}

}

}



load.java:

import java.io.*;
import java.util.*;
import java.ibjectInputStream.*;

public class load {

public static void main(String[] aArguments) {
try {
FileInputStream fis = new FileInputStream("test.ser");
ObjectInputStream ois = new ObjectInputStream(fis);
UserInfo user = (UserInfo) ois.readObject();
System.out.println(user.getUserName());
System.out.println(user.getUserPassword());
ois.close();
} catch (IOException e) {System.out.println(e);}
catch (ClassNotFoundException e) {System.out.println("Class
error");}

}

}



And, UserInfo.java:

package serialize;

public class UserInfo implements java.io.Serializable {
private String userName = "";
private String userPassword = "";
public UserInfo() {
}

public String getUserName() {
return userName;
}

public void setUserName(String s) {
userName = s;
}

public String getUserPassword() {
return userPassword;
}

public void setUserPassword(String s) {
userPassword = s;
}
}


And, here's a dump of the test.ser file:

00000000 AC ED 00 05 73 72 00 08-55 73 65 72 49 6E 66 6F
.....sr..UserInfo
00000010 9A 31 9C CD 65 9F 0B 7D-02 00 02 4C 00 08 75 73
..1..e..}...L..us
00000020 65 72 4E 61 6D 65 74 00-12 4C 6A 61 76 61 2F 6C
erNamet..Ljava/l
00000030 61 6E 67 2F 53 74 72 69-6E 67 3B 4C 00 0C 75 73
ang/String;L..us
00000040 65 72 50 61 73 73 77 6F-72 64 71 00 7E 00 01 78
erPasswordq.~..x
00000050 70 74 00 03 41 42 43 74-00 03 44 45 46
pt..ABCt..DEF

 

[Mike Schilling]

It is a good question. One of the big objections people have to
serialized or other binary formats in the way they have got stung in
past not knowing which class files are needed to reconstitute a
stream, and perhaps no longer having them since they have evolved.
People are often sloppy about documenting binary formats or tracking
which formats belong with which files. I hope some day that
information will be as much as part of the file as the timestamp.

Bill Wilkinson did a fair bit of poking around to figure out how
serialisation works. Perhaps he has some notes buried he would share.

The Java serialization format is fully documented. See

http://java.sun.com/j2se/1.4.2/docs/guide/serialization/spec/protocol.html#wp8101

At times I've had to analyze a serialization file armed only with the spec
and a hex dump utility. It's not fun, but it's quite feasible.

 

[Roedy Green]

At this point, I want to try to go ahead as you suggested, using my
save/load programs, but I don't quite understand what I should be
looking for in the output file.

Let's start simple. What do you get in the output file when you save
an object with one int field public int field. Give the int a
distinctive value like 0x01234567. I would expect to find perhaps
class names and field names. What happens when you dump two of them?

I would expect the class and field name once, and some sort of
reference to a offset in the stream or a token number for the second
one.

Now try this with a single string as the object.

Then with a string in an Object.

Then with a byte and an int in an object.

then a reference to an X in an X object.

then dump out a small chain of objects, just writing the first one.

Expect to find length bytes, type info, token definitions.

Make a theory. Use that theory to make a prediction. Then test the
prediction. Repeat until you can explain all the sample files you can
create.