PrintWriter woes

[Crouchez]

I create a PrintWriter with:
p = new PrintWriter(
new BufferedWriter(new OutputStreamWriter(
mystream,"UTF-8"),4096));

and capture the bytes in the write method of mystream that extends
OutputStream

when I p.print(somestring) a number of times it pushes the bytes to mystream
in 8192 chunks. How come when I've set BufferWriter to 4096?

 

[Lothar Kimmeringer]

when I p.print(somestring) a number of times it pushes the bytes to mystream
in 8192 chunks. How come when I've set BufferWriter to 4096?

A character has a length of 2 bytes. It's not very clear what
the API means with the size of the buffer but I assume it's
characters. But to make sure you should look into the sources.
Maybe there's another buffer (e.g. in OutputStreamWriter) leading
to the effect.


Regards, Lothar
--
Lothar Kimmeringer E-Mail: (e-mail address removed)
PGP-encrypted mails preferred (Key-ID: 0x8BC3CD81)

Always remember: The answer is forty-two, there can only be wrong
questions!

 

[Crouchez]

A character has a length of 2 bytes. It's not very clear what
the API means with the size of the buffer but I assume it's
characters. But to make sure you should look into the sources.
Maybe there's another buffer (e.g. in OutputStreamWriter) leading
to the effect.


Regards, Lothar
--
Lothar Kimmeringer E-Mail: (e-mail address removed)
PGP-encrypted mails preferred (Key-ID: 0x8BC3CD81)

Always remember: The answer is forty-two, there can only be wrong
questions!

It doesn't matter what size is given to BufferedWriter it still sends
buffers of 8192. Does sun.nio.cs.StreamEncoder buffer? And where is it?

 

[Roedy Green]

It doesn't matter what size is given to BufferedWriter it still sends
buffers of 8192. D

How did you discover this?

 

[Roedy Green]

I create a PrintWriter with:
p = new PrintWriter(
new BufferedWriter(new OutputStreamWriter(
mystream,"UTF-8"),4096));

and capture the bytes in the write method of mystream that extends
OutputStream

when I p.print(somestring) a number of times it pushes the bytes to mystream
in 8192 chunks. How come when I've set BufferWriter to 4096?

look at the code for the BufferedWriter constructor.

public BufferedWriter(Writer out, int sz) {
super(out);
if (sz <= 0)
throw new IllegalArgumentException("Buffer size <= 0");
this.out = out;
cb = new char[sz];

The size of the buffer is specified in chars.

See http://mindprod.com/applet/fileio.html
When it generates code for you it now documents the unit of measure
for buffer sizes.

 

[Lothar Kimmeringer]

It doesn't matter what size is given to BufferedWriter it still sends
buffers of 8192.

Can you post the source you wrote for testing this? With the
constructor Roedy was showing I would expect the number of
bytes being sent to the underlying stream is dependent on the
characters being written and the encoding to be used. With x
characters written and UTF-8 used as encoding the number of
bytes should vary between x (ASCII) and 3x (0x800 - 0xffff).
So with a buffer-size of 4096 the number of bytes in your example
could reach up to 12288.

I don't know how OutputStreamWriter works internally but if there
is a buffer as well this might be the reason, why you always
get the same number of bytes. But I'm not going to look into that
further until you posted your test-code.

Does sun.nio.cs.StreamEncoder buffer? And where is it?

In rt.jar or charset.jar, but I would expect it in the first one.


Regards, Lothar
--
Lothar Kimmeringer E-Mail: (e-mail address removed)
PGP-encrypted mails preferred (Key-ID: 0x8BC3CD81)

Always remember: The answer is forty-two, there can only be wrong
questions!

 

[Roedy Green]

when I p.print(somestring) a number of times it pushes the bytes to mystream
in 8192 chunks. How come when I've set BufferWriter to 4096?

In short BufferedWriter constructor takes the size in chars. 4096
chars = 8192 bytes.

 

[Crouchez]

In short BufferedWriter constructor takes the size in chars. 4096
chars = 8192 bytes.

It doesn't matter if you put a buffer of 1024 it still pushes a buffer
through of 8192

out = new AnOutputStream( 1024); //buf then set as 1024
p = new PrintWriter(
new BufferedWriter(new OutputStreamWriter(
out,"UTF-8"),1024));


p.print(string) x 4000 times eg.

result of public synchronized void write(byte b[], int off, int len) in out

len:8192

....

 

[Lew]

out = new AnOutputStream( 1024); //buf then set as 1024
p = new PrintWriter(
new BufferedWriter(new OutputStreamWriter(
out,"UTF-8"),1024));

len:8192

when I p.print(somestring) a number of times it pushes the bytes to mystream a.k.a. "out"
in 8192 chunks. How come when I've set BufferWriter to 4096?

To be clear - you are seeing writes to the underlying stream in 8 KiB chunks?
You aren't measuring the writes in either of the Writers?

Had you considered reading the API docs for OutputStreamWriter?

Each invocation of a write() method causes the encoding converter to be invoked on the given character(s).
The resulting bytes are accumulated in a buffer before being written to the underlying output stream. The size of this buffer may be specified, but by default it is large enough for most purposes.

Dollars to doughnuts the size of that OutputStreamWriter buffer on your system
is 8 KiB. Any takers?

Aren't the API docs wonderful?

 

[Crouchez]

To be clear - you are seeing writes to the underlying stream in 8 KiB
chunks? You aren't measuring the writes in either of the Writers?

Had you considered reading the API docs for OutputStreamWriter?


Dollars to doughnuts the size of that OutputStreamWriter buffer on your
system is 8 KiB. Any takers?

Aren't the API docs wonderful?

So you can't set this value unless you create your own version of it? So
sun.nio.cs.StreamEncoder buffers at 8192 potentially causing buffer
overflows at the end of the line?

"The size of this buffer may be specified, but by default it is large enough
for most purposes. "

Specified where exactly?

 

[Lew]

So you can't set this value unless you create your own version of it? So
sun.nio.cs.StreamEncoder buffers at 8192 potentially causing buffer

I'm not sure that's the class defining the buffer. How are you determining
that it is?

overflows at the end of the line?

There should be no risk of buffer overflow.

"The size of this buffer may be specified, but by default it is large enough
for most purposes. "

Specified where exactly?

Sometimes the API docs are not so wonderful. There is no apparent method or
constructor to set the value.

Regardless, you can always get the buffer to flush by calling flush(), so you
don't need to worry about crud being caught in the buffer. Just put a
finally
{
p.flush();
}
in the right place, and you should be fine.

Where the API docs are not so wonderful is in the documentation of these
Writers' buffer strategies, although we've seen that they at least document
that they have them. It makes it hard on us programmers to account for
multiple buffers and the concomitant inefficiencies.

Unless the Writers in question are smart enough to share their buffers? I
don't know - I am far too lazy to read through the publicly available source
code to find out for myself.

I do know that one can at least make sure, as you've done, that there are at
least some buffers. Your BufferedWriter of size 4096 will hold 8192 bytes in
its buffer, apparently a good match for the default (and ineluctable)
OutputStreamWriter buffer. Finish with a flush() and all should be well.

 

[Roedy Green]

It doesn't matter if you put a buffer of 1024 it still pushes a buffer
through of 8192

Again, how do you know this?

 

[Roedy Green]

I create a PrintWriter with:
p = new PrintWriter(
new BufferedWriter(new OutputStreamWriter(
mystream,"UTF-8"),4096));

OutputStreamWriter extends Writer which has a 1024 char buffer.
StreamEncoder has a default buffer of 8096 bytes.
These buffers cascade, so like a multistage colon would act like a
larger buffer.