Re: Writev

Discussion in 'Python' started by Adam DePrince, Dec 21, 2004.

  1. On Sun, 2004-12-19 at 23:43, Jp Calderone wrote:
    > On Sun, 19 Dec 2004 23:12:27 -0500, Adam DePrince <> wrote:
    > > [snip]
    > >
    > > Of course, to take advantage of this requires that writev be exposed. I
    > > have an implementation of writev. This implementation is reasonably
    > > smart, it "unrolls" only so as many iteration.next calls as necessary to
    > > fill the pointer list for the next invocation of writev. Of course, on
    > > many systems (Linux, BSD) writev's limit is 1024 items, so to
    > > accommodate users who don't want to peel off and hold in memory 1024
    > > iteration.next()'s, you can set an optional parameter to a smaller
    > > value.
    > >
    > > I'm not sure where to take this as a "next step." It seems too small a
    > > change for a PEP. Any ideas?
    > >
    > > You can download the patch from
    > > http://deprince.net/software/writev/index.html
    > >

    >
    > writev() in the standard library would be nice. I have an
    > implementation too ;) I wonder how many other people do as
    > well.
    >
    > Regarding the implementation, just one part confuses me.
    > What's the idea behind this check at the beginning of the
    > iterator unrolling loop?
    >
    > + if ( !PyObject_CheckReadBuffer( buf ) )
    > + continue;
    >
    > Is it me, or is that silently ignoring bad input?
    >
    > Other miscellaneous feedback:
    >
    > The use of alloca() isn't common in CPython. PyMem_Malloc
    > is probably what you want to be calling. At the very least,
    > alloca()'s return value should be checked to be sure it is
    > non-NULL. If you switch to PyMem_Malloc, you'll have to be
    > sure to free the memory, of course.
    >
    > The support of iterators is a cool idea, but I'm not sure
    > it is actually useful. Consider the case where not all bytes
    > are written. How is an application supposed to handle writing
    > the rest of the bytes, if they have been consumed from an
    > iterator? Hopefully I'm missing the obvious answer, because
    > supporting arbitrary iterators would be cool, but it seems like
    > only sequences can really be usefully supported.
    >


    Fixed this. The return value is a tuple of bytes written and a tuple of
    stuff that didn't get shipped. And yes, the first element is replaced
    with a string that represents the fraction of that element that didn't
    get sent. Here is simple sample code that illustrates how this works.

    data = [ .... lots of little strings in this list ... ]
    data = iter( data )
    import itertools
    import writev

    while 1:
    (count, leftovers ) = writev.writev( fd, data )
    if leftovers == None: break
    data = itertools.chain( iter( leftovers ), data )
    ... Of course, our loop would in a real example be controlled by a
    select. In this naive example we would want to sleep ...

    Adam DePrince
    Adam DePrince, Dec 21, 2004
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Adam DePrince

    Writev

    Adam DePrince, Dec 20, 2004, in forum: Python
    Replies:
    6
    Views:
    583
    Mike Meyer
    Dec 20, 2004
  2. Adam DePrince

    Re: Writev

    Adam DePrince, Dec 20, 2004, in forum: Python
    Replies:
    0
    Views:
    391
    Adam DePrince
    Dec 20, 2004
  3. kid joe

    writev() on a SOCK_STREAM socket

    kid joe, Jun 19, 2008, in forum: C Programming
    Replies:
    5
    Views:
    384
  4. A. Farber

    Calling writev and readv from perl

    A. Farber, Aug 19, 2008, in forum: Perl Misc
    Replies:
    6
    Views:
    136
    Leon Timmermans
    Aug 21, 2008
Loading...

Share This Page