Writing to file

Discussion in 'Java' started by sara, Nov 30, 2011.

  1. sara

    sara Guest

    Hi All,

    I have a program which generates many lines of data where each line
    includes an integer and two floats. I want to know the fastest way to
    write these generated lines to a file. Currently, I am writing to a
    binary file as follows:
    FileOutputStream fos = new FileOutputStream(fileName);
    DataOutputStream dos = new DataOutputStream(fos);
    for (n : N) {
    dos.writeInt(n.id);
    dos.writeFloat(x.floatValue());
    dos.writeFloat(y.floatValue());
    }

    However it seems that this approach is very slow. Can I use any kind
    of buffering technique to speed up writing to binary file?

    Best
    Sara
     
    sara, Nov 30, 2011
    #1
    1. Advertising

  2. sara

    Lew Guest

    On Wednesday, November 30, 2011 11:31:33 AM UTC-8, sara wrote:
    > Hi All,
    >
    > I have a program which generates many lines of data where each line
    > includes an integer and two floats. I want to know the fastest way to
    > write these generated lines to a file. Currently, I am writing to a
    > binary file as follows:
    > FileOutputStream fos = new FileOutputStream(fileName);
    > DataOutputStream dos = new DataOutputStream(fos);
    > for (n : N) {
    > dos.writeInt(n.id);
    > dos.writeFloat(x.floatValue());
    > dos.writeFloat(y.floatValue());
    > }
    >
    > However it seems that this approach is very slow. Can I use any kind
    > of buffering technique to speed up writing to binary file?


    Have you considered reading the API docs?
    <http://docs.oracle.com/javase/7/docs/api/java/io/BufferedOutputStream.html>

    Note that you will need to build the DataOutputStream on top of the BufferedOutputStream rather than the other way around.

    As to whether this helps performance, what measurements have you done, and how do you know that the result is "very slow"? Compared to what?

    What were the conditions of the measurements (other load on the system, hard drive configuration, etc.)?

    --
    Lew

    --
    Lew
     
    Lew, Nov 30, 2011
    #2
    1. Advertising

  3. sara

    Roedy Green Guest

    On Wed, 30 Nov 2011 11:31:33 -0800 (PST), sara <>
    wrote, quoted or indirectly quoted someone who said :

    >DataOutputStream dos = new DataOutputStream(fos);
    >for (n : N) {
    >dos.writeInt(n.id);
    >dos.writeFloat(x.floatValue());


    See http://mindprod.com/applet/fileio.html

    It will generate you code to do this. Make sure you ask for buffered
    if you want speed.

    Get rid of Float objects and just use pure float primitives if you
    want speed.

    There is not much overhead, just flipping the little endian Intel uses
    internally to big endian.

    --
    Roedy Green Canadian Mind Products
    http://mindprod.com
    For me, the appeal of computer programming is that
    even though I am quite a klutz,
    I can still produce something, in a sense
    perfect, because the computer gives me as many
    chances as I please to get it right.
     
    Roedy Green, Dec 2, 2011
    #3
  4. sara

    Arne Vajhøj Guest

    On 11/30/2011 2:31 PM, sara wrote:
    > I have a program which generates many lines of data where each line
    > includes an integer and two floats. I want to know the fastest way to
    > write these generated lines to a file. Currently, I am writing to a
    > binary file as follows:
    > FileOutputStream fos = new FileOutputStream(fileName);
    > DataOutputStream dos = new DataOutputStream(fos);
    > for (n : N) {
    > dos.writeInt(n.id);
    > dos.writeFloat(x.floatValue());
    > dos.writeFloat(y.floatValue());
    > }
    >
    > However it seems that this approach is very slow. Can I use any kind
    > of buffering technique to speed up writing to binary file?


    You can use a BufferedOutputStream between the DataOutputStream
    and the FileOutputStream.

    But I am a bit skeptical about the choice of format for
    persisting data.

    Arne
     
    Arne Vajhøj, Dec 3, 2011
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mark Kamoski
    Replies:
    1
    Views:
    2,453
  2. BLF
    Replies:
    4
    Views:
    1,282
    Mike Treseler
    Aug 7, 2004
  3. John Carnahan

    File Access error - writing to .txt file

    John Carnahan, Jul 18, 2003, in forum: ASP .Net
    Replies:
    2
    Views:
    803
    John Carnahan
    Jul 18, 2003
  4. JuneCleaver

    Writing to a log file with in an aspx file.

    JuneCleaver, Apr 8, 2004, in forum: ASP .Net
    Replies:
    1
    Views:
    1,048
    Juan Romero
    Apr 8, 2004
  5. HNguyen
    Replies:
    4
    Views:
    2,408
    HNguyen
    Dec 21, 2004
Loading...

Share This Page