Matching Java md5 digests

[douglasforrest]

I need to match md5 outputs being generated by a java app which in
relevent part uses the following code:

byte[] digest = md.digest();

for (int i=0;i<digest.length;i++) {

hexString.append(Integer.toHexString(digest & 0xFF));

}

return hexString.toString();

where md is an instance of MessageDigest.getInstance("MD5");

Using a particular value to be encoded, this function returns the
31-digit hexedecimal string

a1ee2c082ee66978aeca3d377fd11a5

Using the perl Digest::MD5 md5_hex function on the same string returns
thie following 32-digit hexidecimal string, which is the same as the
java output with a leading "0" added.

0a1ee2c082ee66978aeca3d377fd11a5

I can't change the java so I am trying to to re-create the java results
using perl. I've been using various permutations of unpack and sprintf
on the 16-bit binary value generated by Digest::MD5 md5 without
success so far.

Any suggestions/pointers/ full-blown implementations would be much
appreciated!

 

[A. Sinan Unur]

(e-mail address removed) wrote in

I need to match md5 outputs being generated by a java app which in
relevent part uses the following code:

byte[] digest = md.digest();

for (int i=0;i<digest.length;i++) {

hexString.append(Integer.toHexString(digest & 0xFF));

}

return hexString.toString();

where md is an instance of MessageDigest.getInstance("MD5");

Using a particular value to be encoded, this function returns the
31-digit hexedecimal string

a1ee2c082ee66978aeca3d377fd11a5

Using the perl Digest::MD5 md5_hex function on the same string returns
thie following 32-digit hexidecimal string, which is the same as the
java output with a leading "0" added.

0a1ee2c082ee66978aeca3d377fd11a5

I can't change the java so I am trying to to re-create the java
results using perl. I've been using various permutations of unpack
and sprintf on the 16-bit binary value generated by Digest::MD5 md5
without success so far.

I am not sure what 16-bit value you are talking about. MD5 produces a
128-bit digest.

I am not exactly sure what you are trying to do, but ...

Are you just trying to remove leading zeros?

#!/usr/bin/perl

use strict;
use warnings;

my $md5 = '0a1ee2c082ee66978aeca3d377fd11a5';
$md5 =~ s{\A 0+ }{}x;
print "$md5\n";

__END__

 

[douglasforrest]

I meant 16-byte, not 16-bit; Digest::MD5 returns the 128-bits as a
16-byte binary string.

Unfortunately, the difference is not always a leading 0.

What I need to do, I think, is to take each byte in a form to which I
can bit-add 0xFF, convert the result to hex and concatenate the results
into a string. The question is how?

I tried unpacking using formats "i*" and "I*" into scalars and an array
and then looping through, &'ing 0xFF to each value and then
concatinating but the results are way off.

 

[douglasforrest]

Solved!

@temp = unpack("c*", $md5);

for ($i=0; $i < scalar(@temp); $i++) {

$result .= sprintf "%lx", $temp[$i] & 0xFF;
}

 

[Dr.Ruud]

douglasforrest:

I need to match md5 outputs being generated by a java app which in
relevent part uses the following code:

for (int i=0;i<digest.length;i++) {
hexString.append(Integer.toHexString(digest & 0xFF));
}

If that .toHexString() returns a single character for 0-15 (so not
00-0F), then that code is asking for more collisions than md5 already
does: for example the string 110 can come from 11-00 and from 01-10.

 

[Bart Lateur]

What I need to do, I think, is to take each byte in a form to which I
can bit-add 0xFF, convert the result to hex and concatenate the results
into a string. The question is how?

bit-and. And that's pretty much a noop, at least in Perl, somebody
probably added this to the Java code to convert signed integers to
unsigned. Just ignore it, it cannot be the cause of the difference.

 

[Bart Lateur]

I can't change the java so I am trying to to re-create the java results
using perl.

Wow. You'll keep using a buggy implementation and patch a bugfree
implementation (AFAIK) to match its results. Way to go. Don't be
surprised if you have to share data with other people and for them, the
digest never matches.

 

[douglasforrest]

Thanks everybody for all the comments and suggestions.

A few final comments on my part:

The java implementation is apparently not uncommon -- I found similar
versions all over via Googling.

As to matching the java program's results vs. changing the program, I
have no choice: the java program is not within my control and changing
it is not an option. The java program is being used by a company with
over $1 billion in annual sales to generate access tokens to their data
and to provide the access that my clients require, I have to match the
tokens being generated by their java, additional collisions and all.

The bit-and is exactly the cause of the differences.. My solution as
written produces exactly the same results as the java.