sysread and syswrite analogy

[xhoster]

For file handles that don't have O_NONBLOCK turned on, I see this
behavior:

sysread will return partial reads (fewer bytes than were requested), rather
than blocking, provided that the partial read is at least one byte.

syswrite will not perform partial writes, but rather it will block waiting
for the entire write to be accepted.

Based on sysread's behavior, I would have expected syswrite to return
immediately with a partial write, provided that at least one character was
written. Why the lack of analogy between the two?

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

 

[Jim Gibson]

For file handles that don't have O_NONBLOCK turned on, I see this
behavior:

sysread will return partial reads (fewer bytes than were requested), rather
than blocking, provided that the partial read is at least one byte.

Check "Advanced Programming in the Unix Environment", R. Stevens., p.
54. The Unix read system call operating in blocked mode will return
fewer than the number of bytes requested if:

1. File I/O: only the last record.
2. Terminal I/O: returns on end-of-line
3. Network I/O: returns if network can't supply bytes in reasonable
time.
4. Device I/O: device is record-oriented.

syswrite will not perform partial writes, but rather it will block waiting
for the entire write to be accepted.

Well you are operating in blocked mode, so it should block. So the
question should be "why does read not always block", and the exceptions
as listed above seem reasonable.

Based on sysread's behavior, I would have expected syswrite to return
immediately with a partial write, provided that at least one character was
written. Why the lack of analogy between the two?

I think that one basic difference is that for a read, you do not know
how many bytes are being made available. For a write, you know exactly
how many bytes are available for writing.

If the system were to only accept some of your bytes, you would have to
save the unwritten bytes and try to write them again, which is what
happens for non-blocking writes.

For input, it makes sense to return fewer bytes in those cases where
blocking would be unreasonable (stalled network I/O or record-oriented
data.)

For output, it doesn't make much sense to only accept a partial write
and return, because the program will usually just try to write the
unwritten bytes again, so the system call may as well block. It makes
more sense to use the select function to make sure that the output
stream is ready to accept output, and try to use a buffer size
sufficient to accept most output records.

 

[xhoster]

Check "Advanced Programming in the Unix Environment", R. Stevens., p.
54. The Unix read system call operating in blocked mode will return
fewer than the number of bytes requested if:

1. File I/O: only the last record.
2. Terminal I/O: returns on end-of-line
3. Network I/O: returns if network can't supply bytes in reasonable
time.
4. Device I/O: device is record-oriented.


Well you are operating in blocked mode, so it should block. So the
question should be "why does read not always block", and the exceptions
as listed above seem reasonable.

Sure, but (the analogy to) exception 3 seems reasonable for syswrite, as
well.

I think that one basic difference is that for a read, you do not know
how many bytes are being made available.

If the protocol decrees "You will send me 31,517 bytes" then I do know how
many will be available. Just not exactly when.

For a write, you know exactly
how many bytes are available for writing.

If the system were to only accept some of your bytes, you would have to
save the unwritten bytes and try to write them again, which is what
happens for non-blocking writes.

If I get a partial read, I need to save the partial read someplace,
come back later to get the rest, then concatenate them together. That
seems completely analogous to what I have to do with partial writes.

For input, it makes sense to return fewer bytes in those cases where
blocking would be unreasonable (stalled network I/O or record-oriented
data.)

For output, it doesn't make much sense to only accept a partial write
and return, because the program will usually just try to write the
unwritten bytes again, so the system call may as well block. It makes
more sense to use the select function to make sure that the output
stream is ready to accept output, and try to use a buffer size
sufficient to accept most output records.

But because syswrite will not do partial writes, the size of buffer which
it is safe to use is 1 byte. Which isn't much of a buffer.

The way sysread works, it makes sense to use select and sysread on a handle
that is in standard (blocking) mode. The way syswrite works, it doesn't
make much sense to use it with select in standard (blocking) mode, you
would have to syswrite one byte at a time. It seems like an odd
arrangement. But apparently it is an odd arrangement that Perl inherits
from the underlying system, and neglected to shield me from.

Does O_NONBLOCK (or $fh->blocking(0))work on all the systems which select
works on?

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

 

[sln]

For file handles that don't have O_NONBLOCK turned on, I see this
behavior:

sysread will return partial reads (fewer bytes than were requested), rather
than blocking, provided that the partial read is at least one byte.

syswrite will not perform partial writes, but rather it will block waiting
for the entire write to be accepted.

Based on sysread's behavior, I would have expected syswrite to return
immediately with a partial write, provided that at least one character was
written. Why the lack of analogy between the two?

Xho

Am I missing something here? Level I and level II with regard to i/o?
I thought syswrite/sysread were Level I i/o.

sln

 

[xhoster]

Am I missing something here? Level I and level II with regard to i/o?
I thought syswrite/sysread were Level I i/o.

What are "Level I" and "Level II" I/O? AFAICT, these terms are not used
in the Perl docs, or the Linux man pages. They seem to be part of your own
private language.


Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

 

[Peter J. Holzer]

For file handles that don't have O_NONBLOCK turned on, I see this
behavior:

sysread will return partial reads (fewer bytes than were requested), rather
than blocking, provided that the partial read is at least one byte.

syswrite will not perform partial writes, but rather it will block waiting
for the entire write to be accepted.

Based on sysread's behavior, I would have expected syswrite to return
immediately with a partial write, provided that at least one character was
written. Why the lack of analogy between the two?

I can see two reasons:

For a read operation, you typically cannot know how many bytes you need
to read (when Unix was developed, non-file input was from line oriented
terminals. Many network protocols are also line oriented). Blocking
until a fixed number of bytes are read is just impossible. (Imagine a
shell where you always need to type exactly 80 bytes for each command).
Reading single characters was considered too CPU intensive. Hence
read(2) returns as soon as possible. For write otoh, you always know
exactly, how many bytes need to be written. The program typically cannot
do anything else until that operation is finished (and it won't block
anyway because the write just copies the data into an OS buffer). So a
partial write wasn't considered desirable.

Both read and write are supposed to be atomic (they often aren't in
reality, but that's what the Unix programmer expects). For read a
partial read *helps* maintaining atomicity. If your read buffer is
larger than the largest write request, each read will normally return a
single (or an integral number) of records. And even if your buffer is
smaller, you can usually safely read the rest. But for write, it destroys
atomicity: You have just written a partial record, now you need to write
the rest and in the meantime somebody else could write another (partial)
record into the middle of your record. So partial writes are more
problematic than partial reads.



Of course some assumptions from the 1970's don't hold any more, and some
system calls which were added later don't fit nicely into this model.
Especially select(2) is problematic, because it just tells you that you
can write without blocking, but not how much you can write without
blocking. So either you need to write in 1-byte chunks or you need to
use a non-blocking file descriptor.


hp

 

[sln]

What are "Level I" and "Level II" I/O? AFAICT, these terms are not used
in the Perl docs, or the Linux man pages. They seem to be part of your own
private language.


Xho

You must be too young and inexperienced to know what Level I/II I/O is.
It was later changed to unbuffered/buffered I/O. The 'I' Roman Numberal being
'Lower' than the 'II' Roman Numeral. Hence 'Low-Level' I/O, conversely what,
'High-Level' I/O ??

What, there is no 'High-Level' I/O ?? Where did 'Low-Level' I/O come from ??
What the hell does 'Level' mean to you ??

And are you saying Perl does its own I/O ?? Well, I'm not suprised since I found
out Perl does its own floating point conversions.

Sounds like Perl justs RE-INVENTS the wheel all the time partner.
Grow up sonny !


sln

 

[xhoster]

Of course some assumptions from the 1970's don't hold any more, and some
system calls which were added later don't fit nicely into this model.
Especially select(2) is problematic, because it just tells you that you
can write without blocking, but not how much you can write without
blocking. So either you need to write in 1-byte chunks or you need to
use a non-blocking file descriptor.

Thanks for the useful discussion. I'm used to Perl protecting me from
such underlying system anachronisms. But in this case I guess I don't see
how it could unless it were to automatically set O_NONBLOCK for me, which
would probably be going a bit too far.

Thankfully IO::Handle->blocking() makes it almost unnecessary to manually
use the nightmare which is fcntl.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.