sysread and syswrite analogy

Discussion in 'Perl Misc' started by xhoster@gmail.com, Dec 4, 2008.

  1. Guest

    For file handles that don't have O_NONBLOCK turned on, I see this
    behavior:

    sysread will return partial reads (fewer bytes than were requested), rather
    than blocking, provided that the partial read is at least one byte.

    syswrite will not perform partial writes, but rather it will block waiting
    for the entire write to be accepted.

    Based on sysread's behavior, I would have expected syswrite to return
    immediately with a partial write, provided that at least one character was
    written. Why the lack of analogy between the two?

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    The costs of publication of this article were defrayed in part by the
    payment of page charges. This article must therefore be hereby marked
    advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
    this fact.
     
    , Dec 4, 2008
    #1
    1. Advertising

  2. Jim Gibson Guest

    In article <20081204145716.083$>, <>
    wrote:

    > For file handles that don't have O_NONBLOCK turned on, I see this
    > behavior:
    >
    > sysread will return partial reads (fewer bytes than were requested), rather
    > than blocking, provided that the partial read is at least one byte.


    Check "Advanced Programming in the Unix Environment", R. Stevens., p.
    54. The Unix read system call operating in blocked mode will return
    fewer than the number of bytes requested if:

    1. File I/O: only the last record.
    2. Terminal I/O: returns on end-of-line
    3. Network I/O: returns if network can't supply bytes in reasonable
    time.
    4. Device I/O: device is record-oriented.

    >
    > syswrite will not perform partial writes, but rather it will block waiting
    > for the entire write to be accepted.


    Well you are operating in blocked mode, so it should block. So the
    question should be "why does read not always block", and the exceptions
    as listed above seem reasonable.

    >
    > Based on sysread's behavior, I would have expected syswrite to return
    > immediately with a partial write, provided that at least one character was
    > written. Why the lack of analogy between the two?


    I think that one basic difference is that for a read, you do not know
    how many bytes are being made available. For a write, you know exactly
    how many bytes are available for writing.

    If the system were to only accept some of your bytes, you would have to
    save the unwritten bytes and try to write them again, which is what
    happens for non-blocking writes.

    For input, it makes sense to return fewer bytes in those cases where
    blocking would be unreasonable (stalled network I/O or record-oriented
    data.)

    For output, it doesn't make much sense to only accept a partial write
    and return, because the program will usually just try to write the
    unwritten bytes again, so the system call may as well block. It makes
    more sense to use the select function to make sure that the output
    stream is ready to accept output, and try to use a buffer size
    sufficient to accept most output records.

    --
    Jim Gibson
     
    Jim Gibson, Dec 4, 2008
    #2
    1. Advertising

  3. Guest

    Jim Gibson <> wrote:
    > In article <20081204145716.083$>, <>
    > wrote:
    >
    > > For file handles that don't have O_NONBLOCK turned on, I see this
    > > behavior:
    > >
    > > sysread will return partial reads (fewer bytes than were requested),
    > > rather than blocking, provided that the partial read is at least one
    > > byte.

    >
    > Check "Advanced Programming in the Unix Environment", R. Stevens., p.
    > 54. The Unix read system call operating in blocked mode will return
    > fewer than the number of bytes requested if:
    >
    > 1. File I/O: only the last record.
    > 2. Terminal I/O: returns on end-of-line
    > 3. Network I/O: returns if network can't supply bytes in reasonable
    > time.
    > 4. Device I/O: device is record-oriented.
    >
    > >
    > > syswrite will not perform partial writes, but rather it will block
    > > waiting for the entire write to be accepted.

    >
    > Well you are operating in blocked mode, so it should block. So the
    > question should be "why does read not always block", and the exceptions
    > as listed above seem reasonable.


    Sure, but (the analogy to) exception 3 seems reasonable for syswrite, as
    well.

    > > Based on sysread's behavior, I would have expected syswrite to return
    > > immediately with a partial write, provided that at least one character
    > > was written. Why the lack of analogy between the two?

    >
    > I think that one basic difference is that for a read, you do not know
    > how many bytes are being made available.


    If the protocol decrees "You will send me 31,517 bytes" then I do know how
    many will be available. Just not exactly when.

    > For a write, you know exactly
    > how many bytes are available for writing.
    >
    > If the system were to only accept some of your bytes, you would have to
    > save the unwritten bytes and try to write them again, which is what
    > happens for non-blocking writes.


    If I get a partial read, I need to save the partial read someplace,
    come back later to get the rest, then concatenate them together. That
    seems completely analogous to what I have to do with partial writes.


    > For input, it makes sense to return fewer bytes in those cases where
    > blocking would be unreasonable (stalled network I/O or record-oriented
    > data.)
    >
    > For output, it doesn't make much sense to only accept a partial write
    > and return, because the program will usually just try to write the
    > unwritten bytes again, so the system call may as well block. It makes
    > more sense to use the select function to make sure that the output
    > stream is ready to accept output, and try to use a buffer size
    > sufficient to accept most output records.


    But because syswrite will not do partial writes, the size of buffer which
    it is safe to use is 1 byte. Which isn't much of a buffer.

    The way sysread works, it makes sense to use select and sysread on a handle
    that is in standard (blocking) mode. The way syswrite works, it doesn't
    make much sense to use it with select in standard (blocking) mode, you
    would have to syswrite one byte at a time. It seems like an odd
    arrangement. But apparently it is an odd arrangement that Perl inherits
    from the underlying system, and neglected to shield me from.

    Does O_NONBLOCK (or $fh->blocking(0))work on all the systems which select
    works on?

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    The costs of publication of this article were defrayed in part by the
    payment of page charges. This article must therefore be hereby marked
    advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
    this fact.
     
    , Dec 5, 2008
    #3
  4. Guest

    On 04 Dec 2008 19:58:16 GMT, wrote:

    >For file handles that don't have O_NONBLOCK turned on, I see this
    >behavior:
    >
    >sysread will return partial reads (fewer bytes than were requested), rather
    >than blocking, provided that the partial read is at least one byte.
    >
    >syswrite will not perform partial writes, but rather it will block waiting
    >for the entire write to be accepted.
    >
    >Based on sysread's behavior, I would have expected syswrite to return
    >immediately with a partial write, provided that at least one character was
    >written. Why the lack of analogy between the two?
    >
    >Xho

    Am I missing something here? Level I and level II with regard to i/o?
    I thought syswrite/sysread were Level I i/o.

    sln
     
    , Dec 5, 2008
    #4
  5. Guest

    wrote:

    > Am I missing something here? Level I and level II with regard to i/o?
    > I thought syswrite/sysread were Level I i/o.


    What are "Level I" and "Level II" I/O? AFAICT, these terms are not used
    in the Perl docs, or the Linux man pages. They seem to be part of your own
    private language.


    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    The costs of publication of this article were defrayed in part by the
    payment of page charges. This article must therefore be hereby marked
    advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
    this fact.
     
    , Dec 5, 2008
    #5
  6. On 2008-12-04 19:58, <> wrote:
    > For file handles that don't have O_NONBLOCK turned on, I see this
    > behavior:
    >
    > sysread will return partial reads (fewer bytes than were requested), rather
    > than blocking, provided that the partial read is at least one byte.
    >
    > syswrite will not perform partial writes, but rather it will block waiting
    > for the entire write to be accepted.
    >
    > Based on sysread's behavior, I would have expected syswrite to return
    > immediately with a partial write, provided that at least one character was
    > written. Why the lack of analogy between the two?


    I can see two reasons:

    For a read operation, you typically cannot know how many bytes you need
    to read (when Unix was developed, non-file input was from line oriented
    terminals. Many network protocols are also line oriented). Blocking
    until a fixed number of bytes are read is just impossible. (Imagine a
    shell where you always need to type exactly 80 bytes for each command).
    Reading single characters was considered too CPU intensive. Hence
    read(2) returns as soon as possible. For write otoh, you always know
    exactly, how many bytes need to be written. The program typically cannot
    do anything else until that operation is finished (and it won't block
    anyway because the write just copies the data into an OS buffer). So a
    partial write wasn't considered desirable.

    Both read and write are supposed to be atomic (they often aren't in
    reality, but that's what the Unix programmer expects). For read a
    partial read *helps* maintaining atomicity. If your read buffer is
    larger than the largest write request, each read will normally return a
    single (or an integral number) of records. And even if your buffer is
    smaller, you can usually safely read the rest. But for write, it destroys
    atomicity: You have just written a partial record, now you need to write
    the rest and in the meantime somebody else could write another (partial)
    record into the middle of your record. So partial writes are more
    problematic than partial reads.



    Of course some assumptions from the 1970's don't hold any more, and some
    system calls which were added later don't fit nicely into this model.
    Especially select(2) is problematic, because it just tells you that you
    can write without blocking, but not how much you can write without
    blocking. So either you need to write in 1-byte chunks or you need to
    use a non-blocking file descriptor.


    hp
     
    Peter J. Holzer, Dec 6, 2008
    #6
  7. Guest

    On 05 Dec 2008 17:05:59 GMT, wrote:

    > wrote:
    >
    >> Am I missing something here? Level I and level II with regard to i/o?
    >> I thought syswrite/sysread were Level I i/o.

    >
    >What are "Level I" and "Level II" I/O? AFAICT, these terms are not used
    >in the Perl docs, or the Linux man pages. They seem to be part of your own
    >private language.
    >
    >
    >Xho


    You must be too young and inexperienced to know what Level I/II I/O is.
    It was later changed to unbuffered/buffered I/O. The 'I' Roman Numberal being
    'Lower' than the 'II' Roman Numeral. Hence 'Low-Level' I/O, conversely what,
    'High-Level' I/O ??

    What, there is no 'High-Level' I/O ?? Where did 'Low-Level' I/O come from ??
    What the hell does 'Level' mean to you ??

    And are you saying Perl does its own I/O ?? Well, I'm not suprised since I found
    out Perl does its own floating point conversions.

    Sounds like Perl justs RE-INVENTS the wheel all the time partner.
    Grow up sonny !


    sln
     
    , Dec 6, 2008
    #7
  8. Guest

    "Peter J. Holzer" <> wrote:
    ....

    > Of course some assumptions from the 1970's don't hold any more, and some
    > system calls which were added later don't fit nicely into this model.
    > Especially select(2) is problematic, because it just tells you that you
    > can write without blocking, but not how much you can write without
    > blocking. So either you need to write in 1-byte chunks or you need to
    > use a non-blocking file descriptor.


    Thanks for the useful discussion. I'm used to Perl protecting me from
    such underlying system anachronisms. But in this case I guess I don't see
    how it could unless it were to automatically set O_NONBLOCK for me, which
    would probably be going a bit too far.

    Thankfully IO::Handle->blocking() makes it almost unnecessary to manually
    use the nightmare which is fcntl.

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    The costs of publication of this article were defrayed in part by the
    payment of page charges. This article must therefore be hereby marked
    advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
    this fact.
     
    , Dec 6, 2008
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Weyoun the Dancing Borg

    I thought of an analogy...

    Weyoun the Dancing Borg, Jul 22, 2004, in forum: HTML
    Replies:
    21
    Views:
    852
  2. Mahmud Amin Mahi

    Function analogy

    Mahmud Amin Mahi, May 16, 2010, in forum: C Programming
    Replies:
    0
    Views:
    289
    Mahmud Amin Mahi
    May 16, 2010
  3. A. Farber
    Replies:
    1
    Views:
    130
    J. Gleixner
    Dec 29, 2009
  4. A. Farber
    Replies:
    2
    Views:
    145
    Uri Guttman
    May 14, 2010
  5. Richard Maher
    Replies:
    6
    Views:
    121
    Richard Maher
    Apr 19, 2007
Loading...

Share This Page