Writing Binaries to and Reading Binaries from Disk

Discussion in 'Java' started by KevinSimonson, Sep 26, 2009.

  1. I'm planning on writing a program that stores a lot of binary informa-
    tion, and I'd like to be able to write it to disk when I'm done with
    it, so that later I can read it in from disk again. How do you do
    that in Java, write binary <short>s to disk and then later read in
    those <short>s from disk again?

    Kevin Simonson

    "You'll never get to heaven, or even to LA,
    if you don't believe there's a way."
    from _Why Not_
    KevinSimonson, Sep 26, 2009
    #1
    1. Advertising

  2. KevinSimonson wrote:
    > I'm planning on writing a program that stores a lot of binary informa-
    > tion, and I'd like to be able to write it to disk when I'm done with
    > it, so that later I can read it in from disk again. How do you do
    > that in Java, write binary <short>s to disk and then later read in
    > those <short>s from disk again?
    >
    > Kevin Simonson
    >
    > "You'll never get to heaven, or even to LA,
    > if you don't believe there's a way."
    > from _Why Not_


    I'm not sure what a binary <short> is but to write a Java short to a
    stream you can use the DataOutputStream class. Also you can convert it
    to bytes and use a regular OutputStream. Also see BufferedOutputStream.

    import java.io.*;

    public class test4 {
    public static void main(String[] args) throws IOException {
    FileOutputStream fos = new FileOutputStream("data.dat");
    DataOutputStream dos = new DataOutputStream(fos);
    short s1 = -32768;
    dos.writeShort(s1);
    dos.close();

    FileInputStream fis = new FileInputStream("data.dat");
    DataInputStream dis = new DataInputStream(fis);
    short s2 = dis.readShort();
    dis.close();

    System.out.print(s2);
    }
    }

    --

    Knute Johnson
    email s/nospam/knute2009/

    --
    Posted via NewsDemon.com - Premium Uncensored Newsgroup Service
    ------->>>>>>http://www.NewsDemon.com<<<<<<------
    Unlimited Access, Anonymous Accounts, Uncensored Broadband Access
    Knute Johnson, Sep 26, 2009
    #2
    1. Advertising

  3. KevinSimonson wrote:

    > I'm planning on writing a program that stores a lot of binary informa-
    > tion, and I'd like to be able to write it to disk when I'm done with
    > it, so that later I can read it in from disk again. How do you do
    > that in Java, write binary <short>s to disk and then later read in
    > those <short>s from disk again?


    IO-operations are byte-oriented (this is not java specific)
    so when writing shorts (2 bytes) there are two ways to do
    this and it depends if the data you write to disk should
    be read by other programs as well. So before you use
    DataOutputStream you should check if the format to be used
    is LSB least significant byte first or HSB (high significant
    byte first). HSB is used with DataOutputStream, if you
    want to write LSB you need to do the writing for yourself:

    outstream.write(myshort & 0xff);
    outstream.write((myshort >> 8) & 0xff);

    other direction:

    short myshort = instream.read() & 0xff;
    myshort |= (instream.read() << 8) & 0xff;


    Regards, Lothar
    --
    Lothar Kimmeringer E-Mail:
    PGP-encrypted mails preferred (Key-ID: 0x8BC3CD81)

    Always remember: The answer is forty-two, there can only be wrong
    questions!
    Lothar Kimmeringer, Sep 26, 2009
    #3
  4. KevinSimonson

    Eric Sosman Guest

    Lothar Kimmeringer wrote:
    > KevinSimonson wrote:
    >
    >> I'm planning on writing a program that stores a lot of binary informa-
    >> tion, and I'd like to be able to write it to disk when I'm done with
    >> it, so that later I can read it in from disk again. How do you do
    >> that in Java, write binary <short>s to disk and then later read in
    >> those <short>s from disk again?

    >
    > IO-operations are byte-oriented (this is not java specific)
    > so when writing shorts (2 bytes) there are two ways to do
    > this and it depends if the data you write to disk should
    > be read by other programs as well. So before you use
    > DataOutputStream you should check if the format to be used
    > is LSB least significant byte first or HSB (high significant
    > byte first). HSB is used with DataOutputStream, if you
    > want to write LSB you need to do the writing for yourself:
    >
    > outstream.write(myshort & 0xff);
    > outstream.write((myshort >> 8) & 0xff);
    >
    > other direction:
    >
    > short myshort = instream.read() & 0xff;
    > myshort |= (instream.read() << 8) & 0xff;


    Three of the 0xff masks are superfluous and the fourth
    is incorrect (assuming "outstream" is a java.io_OutputStream
    and "instream" a java.io.InputStream). Instead, try

    outstream.write(myshort);
    outstream.write(myshort >> 8);

    and

    short myshort = instream.read();
    myshort |= instream.read() << 8;

    or even

    short myshort = instream.read() | (instream.read() << 8);

    .... although as a dyed-in-the-wool C programmer this last version
    makes me cringe.

    --
    Eric Sosman
    lid
    Eric Sosman, Sep 26, 2009
    #4
  5. Patricia Shanahan wrote:

    > Lothar Kimmeringer wrote:
    >>
    >> outstream.write(myshort & 0xff);
    >> outstream.write((myshort >> 8) & 0xff);
    >>
    >> other direction:
    >>
    >> short myshort = instream.read() & 0xff;
    >> myshort |= (instream.read() << 8) & 0xff;

    >
    > What advantages does this have compared to wrapping the output stream in
    > a DataOutputStream and using its writeShort method?


    As I said (and you quoted it above):
    >> HSB is used with DataOutputStream, if you
    >> want to write LSB you need to do the writing for yourself:



    Regards, Lothar
    --
    Lothar Kimmeringer E-Mail:
    PGP-encrypted mails preferred (Key-ID: 0x8BC3CD81)

    Always remember: The answer is forty-two, there can only be wrong
    questions!
    Lothar Kimmeringer, Sep 26, 2009
    #5
  6. Eric Sosman wrote:

    > Lothar Kimmeringer wrote:
    >> outstream.write(myshort & 0xff);
    >> outstream.write((myshort >> 8) & 0xff);
    >>

    > Three of the 0xff masks are superfluous


    I know but it's a way of telling other that I really
    want to write only the bits 0 to 7.

    >> other direction:
    >>
    >> short myshort = instream.read() & 0xff;
    >> myshort |= (instream.read() << 8) & 0xff;

    > and the fourth
    > is incorrect


    True. Slipped my mind.

    > (assuming "outstream" is a java.io_OutputStream
    > and "instream" a java.io.InputStream). Instead, try
    >
    > outstream.write(myshort);
    > outstream.write(myshort >> 8);


    As I said, the & 0xff is my way of documenting and to make
    sure that I don't forget the parts where it's used.

    > and
    >
    > short myshort = instream.read();
    > myshort |= instream.read() << 8;
    >
    > or even
    >
    > short myshort = instream.read() | (instream.read() << 8);
    >
    > ... although as a dyed-in-the-wool C programmer this last version
    > makes me cringe.


    Of course it should always be checked if the end of stream
    hasn't been reached ;-)


    Regards, Lothar
    --
    Lothar Kimmeringer E-Mail:
    PGP-encrypted mails preferred (Key-ID: 0x8BC3CD81)

    Always remember: The answer is forty-two, there can only be wrong
    questions!
    Lothar Kimmeringer, Sep 26, 2009
    #6
  7. On Sep 26, 1:45 pm, Knute Johnson <>
    wrote:

    =I'm not sure what a binary <short> is but to write a Java short to a
    =stream you can use the DataOutputStream class. Also you can convert
    =it to bytes and use a regular OutputStream. Also see
    =BufferedOutputStream.

    Thanks for the example snippet of code; I'm going to start using it
    right away. But when I looked in the API for the description of
    <BufferedOutputStream> and <BufferedInputStream>, I couldn't figure
    out exactly how I'd use those classes to write Java <short>s to disk
    and read them from disk.

    I guess I could convert each <short> to a <byte> array and then call
    <write( toDiskArray, 0, 2)>, and then call
    <read( fromDiskArray, 0, 2)>, and then convert <fromDiskArray> back
    into a <short>, but this just seems like a really messy way to write
    and read a <short>. Is there a better way?

    Kevin Simonson

    "You'll never get to heaven, or even to LA,
    if you don't believe there's a way."
    from _Why Not_
    KevinSimonson, Sep 26, 2009
    #7
  8. Patricia Shanahan wrote:

    >>>> HSB is used with DataOutputStream, if you
    >>>> want to write LSB you need to do the writing for yourself:

    >
    > Remember the original question "How do you do that in Java, write binary
    > <short>s to disk and then later read in those <short>s from disk again?"
    >
    > What advantage to you see, given that requirement, to writing the data
    > in little-endian format?


    You don't like reading, do you? I also wrote

    >>>> So before you use
    >>>> DataOutputStream you should check if the format to be used
    >>>> is LSB least significant byte first or HSB (high significant
    >>>> byte first).


    The OP didn't say anything about other programs but there is nothing
    wrong in checking before if that is the case before you use a
    specific encoding and regret it later.


    Regards, Lothar
    --
    Lothar Kimmeringer E-Mail:
    PGP-encrypted mails preferred (Key-ID: 0x8BC3CD81)

    Always remember: The answer is forty-two, there can only be wrong
    questions!
    Lothar Kimmeringer, Sep 26, 2009
    #8
  9. KevinSimonson wrote:

    > Thanks for the example snippet of code; I'm going to start using it
    > right away. But when I looked in the API for the description of
    > <BufferedOutputStream> and <BufferedInputStream>, I couldn't figure
    > out exactly how I'd use those classes to write Java <short>s to disk
    > and read them from disk.
    >
    > I guess I could convert each <short> to a <byte> array and then call
    > <write( toDiskArray, 0, 2)>, and then call
    > <read( fromDiskArray, 0, 2)>, and then convert <fromDiskArray> back
    > into a <short>, but this just seems like a really messy way to write
    > and read a <short>. Is there a better way?


    I'm sure he meant it that way:

    FileOutputStream fos = new FileOutputStream(file);
    BufferedOutputStream bos = new BufferedOutputStream(fos);
    DataOutputStream dos = new DataOutputStream(bos);

    Then you can still use writeShort without writing every single
    directly to the disk which would lead to a drop in performance.


    Regards, Lothar
    --
    Lothar Kimmeringer E-Mail:
    PGP-encrypted mails preferred (Key-ID: 0x8BC3CD81)

    Always remember: The answer is forty-two, there can only be wrong
    questions!
    Lothar Kimmeringer, Sep 26, 2009
    #9
  10. KevinSimonson

    Qu0ll Guest

    "KevinSimonson" <> wrote in message
    news:...
    > I'm planning on writing a program that stores a lot of binary informa-
    > tion, and I'd like to be able to write it to disk when I'm done with
    > it, so that later I can read it in from disk again. How do you do
    > that in Java, write binary <short>s to disk and then later read in
    > those <short>s from disk again?


    The advice so far has been to use traditional java.io but I have found that
    you'll get better performance if you use nio and especially if you use a
    MappedByteBuffer which has methods to put shorts, ints etc.

    --
    And loving it,

    -Qu0ll (Rare, not extinct)
    _________________________________________________

    [Replace the "SixFour" with numbers to email me]
    Qu0ll, Sep 26, 2009
    #10
  11. KevinSimonson

    Lew Guest

    KevinSimonson wrote:
    > Thanks for the example snippet of code; I'm going to start using it
    > right away. But when I looked in the API for the description of
    > <BufferedOutputStream> and <BufferedInputStream>, I couldn't figure
    > out exactly how I'd use those classes to write Java <short>s to disk
    > and read them from disk.


    Dude, don't use angle brackets for Java conversations that way. Angle
    brackets have a semantic - they mean "generic base type". That you don't mean
    that is jarring. In your paragraph one wants to read "<BufferedOutputStream>"
    as "generic type BufferedOutputStream". That is confusing.

    I use single quotes around type names meant to be taken literally in an
    otherwise natural-language paragraph. Very often people just ignore special
    typography and let you understand that it's a Java type from context.

    --
    Lew
    Lew, Sep 27, 2009
    #11
  12. KevinSimonson

    Lew Guest

    Patricia Shanahan wrote:
    >> What advantage to you see, given that requirement, to writing the data
    >> in little-endian format?


    Lothar Kimmeringer wrote:
    > You don't like reading, do you?


    That was uncalled-for and very snotty. It also didn't work - you don't get to
    duck the question that way.

    Lothar Kimmeringer wrote:
    > I also wrote
    >>>>> So before you use
    >>>>> DataOutputStream you should check if the format to be used
    >>>>> is LSB least significant byte first or HSB (high significant
    >>>>> byte first).


    That doesn't answer Patricia's question. It is barely even related to her
    question. Now why don't you be a polite little boy and just answer what she
    asked?

    > The OP didn't say anything about other programs


    Exactly so.

    --
    Lew
    Lew, Sep 27, 2009
    #12
  13. KevinSimonson

    Roedy Green Guest

    On Sat, 26 Sep 2009 12:26:51 -0700 (PDT), KevinSimonson
    <> wrote, quoted or indirectly quoted someone who
    said :

    >I'm planning on writing a program that stores a lot of binary informa-
    >tion, and I'd like to be able to write it to disk when I'm done with
    >it, so that later I can read it in from disk again. How do you do
    >that in Java, write binary <short>s to disk and then later read in
    >those <short>s from disk again?


    For simple arrays, use DataOutputStream. For complex objects, use
    OutputObjectStream.

    For sample code see

    http://mindprod.com/applet/fileio.html
    --
    Roedy Green Canadian Mind Products
    http://mindprod.com

    "Civilisation advances by extending the number of important operations which we can perform without thinking about them."
    ~ Alfred North Whitehead (born: 1861-02-15 died: 1947-12-30 at age: 86)
    Roedy Green, Sep 27, 2009
    #13
  14. KevinSimonson

    Roedy Green Guest

    On Sat, 26 Sep 2009 23:51:57 +0200, Lothar Kimmeringer
    <> wrote, quoted or indirectly quoted someone
    who said :

    >
    >>>>> So before you use
    >>>>> DataOutputStream you should check if the format to be used
    >>>>> is LSB least significant byte first or HSB (high significant
    >>>>> byte first).


    There another thread about little-endian, big-endian which you might
    be conflating with this one. O.P. just wanted to write out some values
    and read them back. That would naturally be done in big-endian with
    DataOutputStream. DataOutputStream always writes big-endian,
    independent of the endianness of the platform.
    --
    Roedy Green Canadian Mind Products
    http://mindprod.com

    "Civilisation advances by extending the number of important operations which we can perform without thinking about them."
    ~ Alfred North Whitehead (born: 1861-02-15 died: 1947-12-30 at age: 86)
    Roedy Green, Sep 27, 2009
    #14
  15. KevinSimonson

    Eric Sosman Guest

    Patricia Shanahan wrote:
    > Lothar Kimmeringer wrote:
    >> Patricia Shanahan wrote:
    >>
    >>> Lothar Kimmeringer wrote:
    >>>> outstream.write(myshort & 0xff);
    >>>> outstream.write((myshort >> 8) & 0xff);
    >>>>
    >>>> other direction:
    >>>>
    >>>> short myshort = instream.read() & 0xff;
    >>>> myshort |= (instream.read() << 8) & 0xff;
    >>> What advantages does this have compared to wrapping the output stream in
    >>> a DataOutputStream and using its writeShort method?

    >>
    >> As I said (and you quoted it above):
    >>>> HSB is used with DataOutputStream, if you
    >>>> want to write LSB you need to do the writing for yourself:

    >
    > Remember the original question "How do you do that in Java, write binary
    > <short>s to disk and then later read in those <short>s from disk again?"
    >
    > What advantage to you see, given that requirement, to writing the data
    > in little-endian format?


    Elsethread, the O.P. has explained that the data will be
    consumed by a C program that expects to read a "native" format.

    --
    Eric Sosman
    lid
    Eric Sosman, Sep 27, 2009
    #15
  16. KevinSimonson

    Lew Guest

    Eric Sosman wrote:
    > Elsethread, the O.P. has explained that the data will be
    > consumed by a C program that expects to read a "native" format.


    I'm not finding that. I'm only finding:
    > How do you do that in Java,
    > write binary <short>s to disk and then later read in
    > those <short>s from disk again?


    and

    > I guess I could convert each <short> to a <byte> array and then call
    > <write( toDiskArray, 0, 2)>, and then call
    > <read( fromDiskArray, 0, 2)>, and then convert <fromDiskArray> back
    > into a <short>,


    after going back about a month's worth through clj.programmer and clj.help for
    posts by the OP in any thread.

    The only reference in this thread to reading by other programs was not from
    the OP but from Lothar Kimmeringer, which is why Patricia asked him about the
    advantages of little-endian storage, a question he has yet to answer.

    As Roedy Green said in this thread:
    > There['s] another thread about little-endian, big-endian which you might
    > be conflating with this one. O.P. just wanted to write out some values
    > and read them back. That would naturally be done in big-endian with
    > DataOutputStream. DataOutputStream always writes big-endian,
    > independent of the endianness of the platform.


    It looks like that other thread, the one you must be misremembering, Eric, is
    in clj.help from Alexandre Ferrieux, "How to (efficiently) write an int array
    into a file ?" [sic] 2009-09-25 at 21:14Z:
    > How do we accomplish this simple task of a direct write to disk of a
    > big (packed) array of ints, in native byte order ? [sic]


    --
    Lew
    Lew, Sep 27, 2009
    #16
  17. KevinSimonson

    Eric Sosman Guest

    Lew wrote:
    > Eric Sosman wrote:
    >> Elsethread, the O.P. has explained that the data will be
    >> consumed by a C program that expects to read a "native" format.

    >
    > I'm not finding that. [...]
    > It looks like that other thread, the one you must be misremembering,
    > Eric, is in clj.help from Alexandre Ferrieux, "How to (efficiently)
    > write an int array into a file ?" [sic] 2009-09-25 at 21:14Z:


    Yes: I have woven two threads into a tangle.

    --
    Eric Sosman
    lid
    Eric Sosman, Sep 27, 2009
    #17
  18. On Sep 26, 3:54 pm, Lothar Kimmeringer <>
    wrote:

    =I'm sure he meant it that way:
    =
    =FileOutputStream fos = new FileOutputStream(file);
    =BufferedOutputStream bos = new BufferedOutputStream(fos);
    =DataOutputStream dos = new DataOutputStream(bos);
    =
    =Then you can still use writeShort without writing every single
    =directly to the disk which would lead to a drop in performance.

    Thanks, Lothar; this is precisely what I wanted to know.

    Kevin Simonson

    "You'll never get to heaven, or even to LA,
    if you don't believe there's a way."
    from _Why Not_
    KevinSimonson, Sep 28, 2009
    #18
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jas Shultz
    Replies:
    0
    Views:
    936
    Jas Shultz
    Dec 3, 2003
  2. Replies:
    7
    Views:
    510
  3. Jeevan
    Replies:
    4
    Views:
    494
    Glen Herrmannsfeldt
    Oct 21, 2003
  4. Replies:
    12
    Views:
    506
    santosh
    Nov 15, 2006
  5. KevinSimonson
    Replies:
    12
    Views:
    787
    Paul N
    Oct 20, 2010
Loading...

Share This Page