what happens to buffer ?

[Chris Smith]

Gordon Beaton wrote:
...

Why the preference for powers of two?

Because they are more exciting. Many developers, myself included, are
quite reluctant to give up this remaining connection to our industry's
more technical past.

Seriously, I can't think of a reason.

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation

 

[Roedy Green]

Seriously, I can't think of a reason.

I am very surprised so many people did not immediately say:

if your buffer is not a multiple of 512, then the OS is going to have
to allocate its own buffer to read the multiple of 512 and copy the
bytes. If your buffer is a multiple of 512, there is a good chance it
can do the I/O directly into your buffer. Physical i/o is done in
terms of disk sectors.

 

[James McGill]

Why the preference for powers of two?

Systems that live close to the architecture often find performance
benefits, if not hard requirements, to align things on boundaries. It's
quite possible that you might find low-level disc seeks that are not
capable of seeking to an arbitrary address, but instead, deal in offsets
from some block boundary -- and that will invariably be divided into
some power of two.

But in this case, it's not at all clear, if it's even defined, whether
it matters, or if there's any performance implication at all, or if the
compiler or bytecode machine aligns them for you anyway, or if it would
be more efficient to use a prime number instead of a power of two, or
anything else about it. It's not a common thing to divide a buffer by
two, or to arrange buffers for best fit in a larger "power-of-two"
block, or to deal separately with "high and low half-buffers", or
anything of this nature.

It appears this is a historical idiom, not of the language, but of the
programmers. But it's hardly coincidental. Everything digital is
organized in finite quantities, every resource being bounded by some
power of two.

Maybe the next generation will revisit the merits of this whole "binary"
thing, and something better will emerge. When it does, do you think we
will have to throw away everything we know about discrete math?

In the meantime, I'll bet a dollar it does not matter whether you make
your buffers 2000, 2047, 2048, or 2049 bytes. (And I'll gladly pay up
if someone can show me metrics that show otherwise!)

 

[Roedy Green]

In the meantime, I'll bet a dollar it does not matter whether you make
your buffers 2000, 2047, 2048, or 2049 bytes. (And I'll gladly pay up
if someone can show me metrics that show otherwise!)

On the other paw, you might as well use powers of two since you have
no evidence that avoiding them is better. Those are the natural size
containers programmers think in.

Are you sure than even nio does not like buffers multiples of page
frames? It seems highly unlikely.

If I am going to take you up on your bet, I want to find out in
advance what you would consider "cheating". If I find even one OS
where it matters do I get your dollar?

And how much of a percentage in speed additional by using magic
multiples do I have to get to count as faster?

Am I allowed to use Jet, Java 1.6, -server?

 

[Patricia Shanahan]

I am very surprised so many people did not immediately say:

if your buffer is not a multiple of 512, then the OS is going to have
to allocate its own buffer to read the multiple of 512 and copy the
bytes. If your buffer is a multiple of 512, there is a good chance it
can do the I/O directly into your buffer. Physical i/o is done in
terms of disk sectors.

I've done that sort of thing where I knew enough about disk transfer
sizes and alignment requirements, and that the system would use it that way.

However, even java.nio buffers are only claimed to be suitable for
direct I/O if they are allocated by the ByteBuffer allocateDirect
factory method. Are you sure the JVM does direct I/O to ordinary byte
arrays?

Patricia

 

[Roedy Green]

Are you sure the JVM does direct I/O to ordinary byte
arrays?

The JVM likely has nothing to do with it. At the OS level you tell the
OS to deliver X bytes from offset X in the file to offset Y in your
buffer.

If the OS is clever it has prefetched those bytes and copied them to
your buffer.

A long time ago hardware insisted on reading block and plopping them
at 512 byte boundaries. I don't think that is still so, but I think
it may still be so that physical buffers need to be multiples of 512.
In Jet, buffers are automatically aligned on paragraph (16 byte
boundaries)

In benchmarking, you must watch out that you don't reuse the same
file since any reasonably decent OS will soon cache it.Once it is
cached magic buffer sizes would no longer apply, at least until we get
special hardware for copying pages around.

It is not just Windows you are talking about, but ancient old OS's
like IBM's that support Java.

 

[James McGill]

If I find even one OS
where it matters do I get your dollar?

If it's BSDI you only get a Canadian Dollar. If it's SCO, you owe me a
dollar and shame on you for having a SCO box