convert byte array to hex string using BigInteger

Discussion in 'Java' started by Laura Schmidt, Jun 20, 2013.

  1. Hi,

    I try to convert a byte array to a hex string like this:

    private static String hex_encode (byte [] val)
    {
    BigInteger b = new BigInteger(val);

    String t = b.toString(16);

    return (t);
    }

    For a long byte array it returns a "negative" hex string, i. e. starting
    with a "-" sign.

    But I want just the bytes in the array converted to hex representation,
    each one ranging from "00" to "FF". There should be no minus sign then.

    Can you help?
    Thanks!

    Laura
     
    Laura Schmidt, Jun 20, 2013
    #1
    1. Advertisements

  2. Laura Schmidt

    Eric Sosman Guest

    // Untested:
    private static String hexEncode(byte[] val) {
    StringBuilder buff = new StringBuilder(2 * val.length);
    for (byte b : val) {
    buff.append("0123456789ABCDEF".charAt((b >> 4) & 0xF));
    buff.append("0123456789ABCDEF".charAt(b & 0xF));
    }
    return buff.toString();
    }

    There are other ways, too.
     
    Eric Sosman, Jun 20, 2013
    #2
    1. Advertisements

  3. You are right, sorry.

    Now I get a hex string without sign. But now the decode method does not
    return the original bytes anymore:

    private byte [] hex_decode (String val)
    {
    BigInteger b = new BigInteger (val,16);
    byte [] t = b.toByteArray();

    return (t);
    }

    There is no other constructor for byte arrays.
    And I don't really understand why it doesn't return the original byte array.

    Laura
     
    Laura Schmidt, Jun 20, 2013
    #3
  4. Laura Schmidt

    markspace Guest


    Laura, if you're still reading, please note especially this part of the
    sample program. If you want to know if your idea is going to work, you
    have to test it. This list of test vectors is excellent, and also an
    excellent example of how to unit test a method.

    You'll need to do testing like this not only for yourself, now, to
    assure you that your method really works, but also for posterity, by
    which I mean also yourself, but three months from now when you have
    completely forgotten the details of any method you implement today.

    (Or if you're like me, posterity means about two weeks from now. Or less.)
     
    markspace, Jun 20, 2013
    #4
  5. Laura Schmidt

    Joerg Meier Guest

    If you are not married to reinventing the wheel, such as by a homework
    assignment or whatever, you could always use:

    DatatypeConverter.printHexBinary(val);

    instead.

    Liebe Gruesse,
    Joerg
     
    Joerg Meier, Jun 20, 2013
    #5
  6. Laura Schmidt

    Lew Guest

    That's spelled "two's-complement".
    And the purpose of that constructor is just to convert a representation
    in that scheme to the correct number.
    If there isn't, then don't use that constructor.
     
    Lew, Jun 20, 2013
    #6
  7. Laura Schmidt

    Roedy Green Guest

    Here is a low level direct way to convert byte[] to a hex String. It
    should be quite a bit faster than using high level routines.

    /**
    * Fast convert a byte array to a hex string
    * with possible leading zero.
    * @param b array of bytes to convert to string
    * @return hex representation, two chars per byte.
    */
    public static String toHexString ( byte[] b )
    {
    StringBuilder sb = new StringBuilder( b.length * 2 );
    for ( int i=0; i<b.length; i++ )
    {
    // look up high nibble char
    sb.append( hexChar [( b & 0xf0 ) >>> 4] );

    // look up low nibble char
    sb.append( hexChar [b & 0x0f] );
    }
    return sb.toString();
    }

    /**
    * table to convert a nibble to a hex char.
    */
    static char[] hexChar = {
    '0' , '1' , '2' , '3' ,
    '4' , '5' , '6' , '7' ,
    '8' , '9' , 'a' , 'b' ,
    'c' , 'd' , 'e' , 'f'};
     
    Roedy Green, Jun 21, 2013
    #7
  8. Laura Schmidt

    Joerg Meier Guest



    Note that this is pretty much a copy and paste version of the converter
    that already comes with the JRE:

    from javax.xml.bind.DatatypeConverterImpl:

    public String printHexBinary(byte[] data) {
    StringBuilder r = new StringBuilder(data.length * 2);
    for (byte b : data) {
    r.append(hexCode[(b >> 4) & 0xF]);
    r.append(hexCode[(b & 0xF)]);
    }
    return r.toString();
    }

    For a reason I was unable to discern it was actually a teensy bit slower
    than the Oracle implementation.

    Liebe Gruesse,
    Joerg
     
    Joerg Meier, Jun 21, 2013
    #8
  9. Laura Schmidt

    Roedy Green Guest

    I forgot to make my static array final. That might have done it.
    The corrected version is posted at http://mindprod.com/jgloss/hex.html

    How did you think to look for such a method in that package by that
    name?
     
    Roedy Green, Jun 23, 2013
    #9
  10. Laura Schmidt

    Roedy Green Guest

    The code looks almost identical. Probably the difference was caused by
    different "warmup" times before starting to measure. You have to give
    it time to apply the hotspot.
     
    Roedy Green, Jun 23, 2013
    #10
  11. Laura Schmidt

    Joerg Meier Guest

    It has not made any difference in my test case, but I will readily admit
    that I only jotted down a very primitive one, so it's entirely possible
    that the issue was on my end. Either way, it was only around 5-10%, so I
    didn't bother to look into it too much.
    I must admit that I don't recall how I first came upon it, but ever since I
    did, I retained to memory that Java "has that", and the few times I needed
    it, I just used Google until I found "the one in the JRE". It's not the
    most obvious of places, I agree.

    Liebe Gruesse,
    Joerg
     
    Joerg Meier, Jun 23, 2013
    #11
  12. Laura Schmidt

    Joerg Meier Guest

    I looped both versions multiple times, and while times varied, yours was
    consistently slower. But, like I said in my other post, not nearly by
    enough as to prefer one over the other.

    For what it's worth, I just noticed that my code is from OpenJDK, but I
    tested the code from Oracle. Is there still a difference with these
    classes, or are they identical now ? I'm not near my dev PC with the Oracle
    JDK/sources right now.

    Liebe Gruesse,
    Joerg
     
    Joerg Meier, Jun 23, 2013
    #12
  13. Laura Schmidt

    Arne Vajhøj Guest

    You have already received several working solutions.

    I will strongly recommend you drop the idea about using
    BigInteger for this.

    It becomes extremely tricky to get it right - you have already
    found the sign problem - next problem will be the leading zero
    problem.

    See below for coding examples (and I am not even sure that I got the
    BigInteger hack correct).

    Arne

    ====

    import java.math.BigInteger;
    import java.util.Arrays;

    import javax.xml.bind.DatatypeConverter;

    public class ToHex {
    private static void test(byte[] ba, ByteArrayHex cvt) {
    String s = cvt.encode(ba);
    System.out.println(s);
    byte[] ba2 = cvt.decode(s);
    if(ba2.length != ba.length) {
    System.out.println("Length does not match: " + ba2.length + " != " +
    ba.length);
    }
    for(int i = 0; i < Math.min(ba2.length, ba.length); i++) {
    if(ba2 != ba) {
    System.out.println("Byte " + i + " does not match: " + ba2 + " !=
    " + ba);
    }
    }
    }
    private static void testAll(byte[] ba) {
    test(ba, new ManualWithInteger());
    test(ba, new ManualWithString());
    test(ba, new ManualWithCalc());
    test(ba, new JAXBTrick());
    test(ba, new BigIntegerHack());
    }
    public static void main(String[] args) {
    testAll(new byte[] { 1, 2, 3, 4, 5 });
    testAll(new byte[] { -1, -2, -3, -4, -5 });
    testAll(new byte[] { 0, 0, 0 });
    }
    }

    interface ByteArrayHex {
    public String encode(byte[] ba);
    public byte[] decode(String s);
    }

    class ManualWithInteger implements ByteArrayHex {
    public String encode(byte[] ba) {
    StringBuilder sb = new StringBuilder();
    for(int i = 0; i < ba.length; i++) {
    sb.append(Integer.toHexString((ba >> 4) & 0x0F));
    sb.append(Integer.toHexString(ba & 0x0F));
    }
    return sb.toString();
    }
    public byte[] decode(String s) {
    int n = s.length() / 2;
    byte[] res = new byte[n];
    for(int i = 0; i < n; i++) {
    res = (byte)(Integer.parseInt(s.substring(2 * i, 2 * i + 2),
    16));
    }
    return res;
    }
    }

    class ManualWithString implements ByteArrayHex {
    private static final String HEX = "0123456789abcdef";
    public String encode(byte[] ba) {
    StringBuilder sb = new StringBuilder();
    for(int i = 0; i < ba.length; i++) {
    sb.append(HEX.charAt((ba >> 4) & 0x0F));
    sb.append(HEX.charAt(ba & 0x0F));
    }
    return sb.toString();
    }
    public byte[] decode(String s) {
    int n = s.length() / 2;
    byte[] res = new byte[n];
    for(int i = 0; i < n; i++) {
    res = (byte)(HEX.indexOf(s.charAt(2 * i)) << 4 |
    HEX.indexOf(s.charAt(2 * i + 1)));
    }
    return res;
    }
    }

    class ManualWithCalc implements ByteArrayHex {
    private char encode(int v) {
    if(v < 10) {
    return (char)('0' + v);
    } else if(v < 16) {
    return (char)('a' + v - 10);
    } else {
    throw new RuntimeException("Invalid hex value");
    }
    }
    public String encode(byte[] ba) {
    StringBuilder sb = new StringBuilder();
    for(int i = 0; i < ba.length; i++) {
    sb.append(encode((ba >> 4) & 0x0F));
    sb.append(encode(ba & 0x0F));
    }
    return sb.toString();
    }
    private int decode(char c) {
    if('0' <= c && c <= '9') {
    return c - '0';
    } else if('a' <= c && c <= 'f') {
    return c - 'a' + 10;
    } else {
    throw new RuntimeException("Invalid hex value");
    }
    }
    public byte[] decode(String s) {
    int n = s.length() / 2;
    byte[] res = new byte[n];
    for(int i = 0; i < n; i++) {
    res = (byte)(decode(s.charAt(2 * i)) << 4 |
    decode(s.charAt(2 * i + 1)));
    }
    return res;
    }
    }

    class JAXBTrick implements ByteArrayHex {
    public String encode(byte[] ba) {
    return DatatypeConverter.printHexBinary(ba);
    }
    public byte[] decode(String s) {
    return DatatypeConverter.parseHexBinary(s);
    }
    }

    class BigIntegerHack implements ByteArrayHex {
    private String leftPad(String s, char c, int totlen) {
    StringBuilder sb = new StringBuilder();
    for(int i = s.length(); i < totlen; i++) sb.append(c);
    sb.append(s);
    return sb.toString();
    }
    private byte[] leftTrim(byte[] ba, int len) {
    return len < ba.length ? Arrays.copyOfRange(ba, ba.length - len,
    ba.length) : ba;
    }
    private byte[] leftPad(byte[] ba, byte b, int totlen) {
    byte[] res = new byte[totlen];
    int padlen = totlen - ba.length;
    for(int i = 0; i < padlen; i++) res = b;
    System.arraycopy(ba, 0, res, padlen, ba.length);
    return res;
    }
    public String encode(byte[] ba) {
    BigInteger bi = new BigInteger(1, ba);
    String s = bi.toString(16);
    return leftPad(s, '0', ba.length * 2);
    }
    public byte[] decode(String s) {
    BigInteger bi = new BigInteger(s, 16);
    return leftPad(leftTrim(bi.toByteArray(), s.length() / 2), (byte)0,
    s.length() / 2);
    }
    }
     
    Arne Vajhøj, Jun 27, 2013
    #13
  14. Laura Schmidt

    Arne Vajhøj Guest

    You will also get into problems with leading zero bytes.

    Arne
     
    Arne Vajhøj, Jun 27, 2013
    #14
  15. Laura Schmidt

    Arne Vajhøj Guest

    One of the problems is that toString(radix) and toHexString()
    are not the same method.

    Demo:

    public class SimpleHex {
    public static void main(String[] args) {
    for(int i = -1; i <= 1; i++) {
    System.out.println(Integer.toString(i, 16));
    System.out.println(Integer.toHexString(i));
    }
    }
    }


    Arne
     
    Arne Vajhøj, Jun 27, 2013
    #15
  16. Thank you! I agree. In general, I try not to duplicate code that already
    exists in standard libraries, but in this case there were too many hacks
    needed when using BigInteger.

    Laura
     
    Laura Schmidt, Jun 28, 2013
    #16
  17. Laura Schmidt

    Joerg Meier Guest

    Thankfully, as I pointed out in
    <16xd6kqk55a7s$>, you still don't need to
    duplicate code that already exists.

    Liebe Gruesse,
    Joerg
     
    Joerg Meier, Jun 28, 2013
    #17
  18. Wed, 26 Jun 2013 22:12:30 -0400, /Arne Vajhøj/:
    I may be dense, but I didn't got why using BigInteger is not good?
    What's wrong with the following, for example:

    byte[] val;
    ...
    String.format("%064x", new BigInteger(1, val));

    I guess one could come up with a method like:

    String toHexString(byte[] val, int len) {
    BigInteger num = new BigInteger(1, val);
    return (len > 0) ? String.format("%0" + len + "x", num)
    : String.format("%x", num);
    }

    Doesn't it appear sufficient?
     
    Stanimir Stamenkov, Jun 29, 2013
    #18
  19. Laura Schmidt

    Arne Vajhøj Guest

    You will not be able to get back to the same byte array with that.
    It seems to work.

    I would probably drop the len argument and use 2*val.length.

    But it is still not as simple as the DatatypeConverter.

    Arne
     
    Arne Vajhøj, Jul 4, 2013
    #19
  20. Hi
    I`m wondering, how I can implement java constructor in C++ and what private member I need ?
    Thanks Aydin
     
    aydin.k.abadi, Nov 24, 2013
    #20
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.