Re: Writev

Discussion in 'Python' started by Adam DePrince, Dec 21, 2004.

  1. On Sun, 2004-12-19 at 23:43, Jp Calderone wrote:
    > On Sun, 19 Dec 2004 23:12:27 -0500, Adam DePrince <> wrote:
    > > [snip]
    > >
    > > Of course, to take advantage of this requires that writev be exposed. I
    > > have an implementation of writev. This implementation is reasonably
    > > smart, it "unrolls" only so as many calls as necessary to
    > > fill the pointer list for the next invocation of writev. Of course, on
    > > many systems (Linux, BSD) writev's limit is 1024 items, so to
    > > accommodate users who don't want to peel off and hold in memory 1024
    > >'s, you can set an optional parameter to a smaller
    > > value.
    > >
    > > I'm not sure where to take this as a "next step." It seems too small a
    > > change for a PEP. Any ideas?
    > >
    > > You can download the patch from
    > >
    > >

    > writev() in the standard library would be nice. I have an
    > implementation too ;) I wonder how many other people do as
    > well.
    > Regarding the implementation, just one part confuses me.
    > What's the idea behind this check at the beginning of the
    > iterator unrolling loop?
    > + if ( !PyObject_CheckReadBuffer( buf ) )
    > + continue;
    > Is it me, or is that silently ignoring bad input?
    > Other miscellaneous feedback:
    > The use of alloca() isn't common in CPython. PyMem_Malloc
    > is probably what you want to be calling. At the very least,
    > alloca()'s return value should be checked to be sure it is
    > non-NULL. If you switch to PyMem_Malloc, you'll have to be
    > sure to free the memory, of course.
    > The support of iterators is a cool idea, but I'm not sure
    > it is actually useful. Consider the case where not all bytes
    > are written. How is an application supposed to handle writing
    > the rest of the bytes, if they have been consumed from an
    > iterator? Hopefully I'm missing the obvious answer, because
    > supporting arbitrary iterators would be cool, but it seems like
    > only sequences can really be usefully supported.

    Fixed this. The return value is a tuple of bytes written and a tuple of
    stuff that didn't get shipped. And yes, the first element is replaced
    with a string that represents the fraction of that element that didn't
    get sent. Here is simple sample code that illustrates how this works.

    data = [ .... lots of little strings in this list ... ]
    data = iter( data )
    import itertools
    import writev

    while 1:
    (count, leftovers ) = writev.writev( fd, data )
    if leftovers == None: break
    data = itertools.chain( iter( leftovers ), data )
    ... Of course, our loop would in a real example be controlled by a
    select. In this naive example we would want to sleep ...

    Adam DePrince
    Adam DePrince, Dec 21, 2004
    1. Advertisements

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Adam DePrince


    Adam DePrince, Dec 20, 2004, in forum: Python
    Mike Meyer
    Dec 20, 2004
  2. Adam DePrince

    Re: Writev

    Adam DePrince, Dec 20, 2004, in forum: Python
    Adam DePrince
    Dec 20, 2004
  3. kid joe

    writev() on a SOCK_STREAM socket

    kid joe, Jun 19, 2008, in forum: C Programming
  4. A. Farber

    Calling writev and readv from perl

    A. Farber, Aug 19, 2008, in forum: Perl Misc
    Leon Timmermans
    Aug 21, 2008

Share This Page