greatly differing processing time between java and Linux while calculating hashes?

Discussion in 'Java' started by qwertmonkey@syberianoutpost.ru, Sep 9, 2012.

  1. Guest

    ~
    > you want to make sure the size of the byte / char array given to the digest

    calculator is big so that you have the overhead of the call as infrequently
    as possible.
    ~
    is an increase from 512 (2^9) to 65536 (2^16) enough?
    ~
    I didn't see any differences whatsoever in the processing times
    ~
    > You might want to do a trace to be sure you are changing that, not just some

    other buffering.
    ~
    Well, I did trace the loaded classes using the java options:
    ~
    -verbose:jni -verbose:class
    ~
    and using jstack but I couldn't see anything explaining that much of a
    difference. From jstack I got:
    ~
    sun.jvm.hotspot.debugger.{UnmappedAddressException, UnalignedAddressException}
    ~
    which I couldn't quickly troubleshoot
    ~
    I still think those issues may relate in weird (not straight/explicit) ways in
    which the jvm may be chosing checksumming algorithms
    ~
    lbrtchx
    comp.lang.java.programmer:
    greatly differing processing time between java and Linux while calculating
    hashes?
    ~
    ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
    ~
    $ java -verbose:jni -verbose:class CheckSum00Test SHA-512 DQfUaXLk_sw.mp4 >
    `date +%Y%m%d%H%M%S`"_SHA-512.log"

    $ java -verbose:jni -verbose:class CheckSum00Test MD5 DQfUaXLk_sw.mp4 >
    `date +%Y%m%d%H%M%S`"_MD5.log"

    $ java -verbose:jni -verbose:class CheckSum00Test MD5 DQfUaXLk_sw.mp4 >
    `date +%Y%m%d%H%M%S`"_MD5.log"

    $ java -verbose:jni -verbose:class CheckSum00Test SHA-512 DQfUaXLk_sw.mp4 >
    `date +%Y%m%d%H%M%S`"_SHA-512.log"

    $ diff 20120909100112_MD5.log 20120909100215_MD5.log > diff_MD5.log

    $ wc -l *_MD5.log
    852 20120909100112_MD5.log
    852 20120909100215_MD5.log
    4 diff_MD5.log
    1708 total

    $ cat diff_MD5.log
    850c850
    < // __ MD5 encrypting "DQfUaXLk_sw.mp4":"ecbcbe3d0a2fda4ebfddc41836cee528"
    took: 5921 (ms)
    ---
    > // __ MD5 encrypting "DQfUaXLk_sw.mp4":"ecbcbe3d0a2fda4ebfddc41836cee528"

    took: 5830 (ms)

    $ diff 20120909095915_SHA-512.log 20120909100225_SHA-512.log > SHA-512.log

    $ wc -l *SHA-512.log
    853 20120909095915_SHA-512.log
    853 20120909100225_SHA-512.log
    4 SHA-512.log
    1710 total

    $ cat SHA-512.log
    851c851
    < // __ SHA-512 encrypting "DQfUaXLk_sw.mp4":"f6f2f2b88faf3cef5a5de0cb0bc8f1ed
    15969490ef785673d3f7e325ab947c8df8b30edf6636657e945b27eb5202300244b8acef874a11e
    630b2053f956982d8" took: 46852 (ms)
    ---
    > // __ SHA-512 encrypting "DQfUaXLk_sw.mp4":"f6f2f2b88faf3cef5a5de0cb0bc8f1ed

    15969490ef785673d3f7e325ab947c8df8b30edf6636657e945b27eb5202300244b8acef874a11e
    630b2053f956982d8" took: 45030 (ms)

    $ diff 20120909100225_SHA-512.log 20120909100112_MD5.log > SHA-512_MD5.log

    $ wc -l SHA-512_MD5.log
    9 SHA-512_MD5.log

    $ cat SHA-512_MD5.log
    848,849c848
    < [Loaded sun.security.provider.SHA5 from /media/sdb1/inst/sw/jdk1.7.0_07/jre/
    lib/rt.jar]
    < [Loaded sun.security.provider.SHA5$SHA512 from /media/sdb1/inst/sw/
    jdk1.7.0_07/jre/lib/rt.jar]
    ---
    > [Loaded sun.security.provider.MD5 from /media/sdb1/inst/sw/jdk1.7.0_07/jre/

    lib/rt.jar]
    851c850
    < // __ SHA-512 encrypting "DQfUaXLk_sw.mp4":"f6f2f2b88faf3cef5a5de0cb0bc8f1ed
    15969490ef785673d3f7e325ab947c8df8b30edf6636657e945b27eb5202300244b8acef874a11e
    630b2053f956982d8" took: 45030 (ms)
    ---
    > // __ MD5 encrypting "DQfUaXLk_sw.mp4":"ecbcbe3d0a2fda4ebfddc41836cee528"

    took: 5921 (ms)
    $
    ~
    ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
    ~
    #!/bin/bash

    echo "// __ java -version:"
    java -version

    _KDP="java CheckSum00Test"

    _DT=`date +%Y%m%d%H%M%S`

    _PID=`ps aux | grep "${_KDP}" | grep -v grep | awk '{print $2}'`

    echo "// __ process: "
    ps aux | grep "${_KDP}" | grep -v grep

    echo "// __ process id: "${_PID}

    _OFL=${_DT}"_jstack_-m_-l_"${_PID}".log"

    echo "// __ java -version:" >> ${_OFL} 2>&1
    java -version >> ${_OFL} 2>&1
    echo "~" >> ${_OFL} 2>&1


    echo "// __ readlink -f $(which jmap):" >> ${_OFL} 2>&1
    readlink -f $(which jmap) >> ${_OFL} 2>&1
    echo "~" >> ${_OFL} 2>&1


    echo "// __ process: " >> ${_OFL} 2>&1
    ps aux | grep "${_KDP}" | grep -v grep >> ${_OFL} 2>&1
    echo "~" >> ${_OFL} 2>&1


    echo "// __ process id: "${_PID} >> ${_OFL} 2>&1
    echo "~" >> ${_OFL} 2>&1


    echo "// __ jstack -m -l ${_PID} >> ${_OFL} 2>&1" >> ${_OFL} 2>&1
    echo "~" >> ${_OFL} 2>&1

    jstack -m -l ${_PID} >> ${_OFL} 2>&1
    ~
     
    , Sep 9, 2012
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.

Share This Page