Kqueue interface module implementation

Discussion in 'Perl Misc' started by U. A. R. Ruirarchzatrea, Aug 4, 2004.

  1. Hello,

    I have been trying to get a Kqueue module written for Perl with XS
    that would allow the Kqueue event notification facility to be accessed
    from Perl. Kqueue is a scalable event notification facility often
    found on BSD OSs that allows you to recieve events from sockets for
    instance, and corrects the performance limitations of select, allowing
    for multiplexed servers to be written which can scale to very large
    numbers of connections. It does the same thing as epoll on Linux but
    does have a broader range of applications and features than epoll.

    Having a module avialable to access Kqueue would be of use to anyone
    wanting to write scalable network code on BSD OSs.


    Has anyone out there looked into implemented a Kqueue module?
    I was looking into doing it myself, but got confused how to do it. The goal
    is for a Kqueue module written which can be contributed to CPAN so
    other people who need to use Kqueue do not have to reinvent the wheel.

    The Kqueue module should i think basically a straightforward
    mapping of the kqueue and kevent functions and the kevent structures
    to Perl. Later I can writing a higher level OO IO::Kqueue interface
    around the Kqueue module, which would be no problem for me, since that
    wouldnt directly use XS.

    Of course, the Kqueue module involves writing an XS wrapper and that
    is where I am completely confused as to how to properly implement the
    interface.

    I am sure for a Perl XS guru this would probably take a short bit to
    implement with a few lines of code, but despite my reading the xs
    documentation it is still beyond my understanding how to to implement
    the wrapper.

    Documentation for kqueue can be found in the kqueue man page "man
    kqueue", the man pages can also be read online if you search for
    kqueue.

    Is there anyone out there who might be interested and know how to
    write the code to do this?

    I do have some of the files from my attempt including an XS wrapper
    and and a .pm file in the following archive:
    http://www.geocities.com/millueradfa/Kqueue.tar.gz. The archive also
    includes the kqueue manual page and a sample kqueue echo server I
    found. The man pages explains the kqueue API. A pointer to an array of
    kevent structures is passed to kevent().

    Perhaps this would map to
    perl as a reference to an array containing arrays whos elements would
    contain the values in the kevent structure. I have written some
    documentation of what I was thinking of for the API for the module at
    the end of the .pm file. Here is a synopsis of the API i was
    considering, this is the low level API and will be wrapper with a
    higher level interface later in IO::Kqueue:


    use Kqueue;

    ($kq, $errno)=kqueue();

    $timeout=12;
    $filter='EVFILT_READ';
    $flags='EV_ADD';
    $fflags='';
    $data='';
    $udata='';

    @kevent=($sockid, $filter, $flags, $fflags, $data, $udata);
    @changelist=(\@kevent);
    $changelist=\@changelist;

    @eventlist=();
    $eventlist=\@eventlist;

    ($errorval, $errno)=kevent($kq, $changelist, $eventlist, $timeout);

    I think Changelist contains any changes to the configuration, eventlist, is used
    to return pending events.

    I am posting some of the documentation for Kqueue here and the
    incomplete XS file I need help with getting to work. For the full
    files (including .pm) you can look at the archive download I
    previously referenced.


    part of Kqueue man page (for full manpage look at :
    http://people.freebsd.org/~jmg/kqueue.man.html)

    SYNOPSIS
    #include <sys/types.h>
    #include <sys/event.h>
    #include <sys/time.h>

    int
    kqueue(void);

    int
    kevent(int kq, const struct kevent *changelist, int nchanges,
    struct kevent *eventlist, int nevents,
    const struct timespec *timeout);

    EV_SET(&kev, ident, filter, flags, fflags, data, udata);

    The kevent structure is defined as:

    struct kevent {
    uintptr_t ident; /* identifier for this event */
    short filter; /* filter for event */
    u_short flags; /* action flags for kqueue */
    u_int fflags; /* filter flag value */
    intptr_t data; /* filter data value */
    void *udata; /* opaque user data identifier */
    };

    RETURN VALUES
    kqueue() creates a new kernel event queue and returns a file
    descriptor.
    If there was an error creating the kernel event queue, a value of
    -1 is
    returned and errno set.

    kevent() returns the number of events placed in the eventlist, up
    to the
    value given by nevents. If an error occurs while processing an
    element
    of the changelist and there is enough room in the eventlist, then
    the
    event will be placed in the eventlist with EV_ERROR set in flags
    and the
    system error in data. Otherwise, -1 will be returned, and errno
    will be
    set to indicate the error condition. If the time limit expires,
    then
    kevent() returns 0.


    Kqueue.xs:

    #include "ppport.h"

    #include <sys/event.h>
    #include <sys/types.h>
    #include <sys/time.h>

    #include "const-c.inc"

    MODULE = Kqueue PACKAGE = Kqueue

    INCLUDE: const-xs.inc

    int
    kqueue ()


    int
    kevent (kq, changelist, nchanges, eventlist, nevents, timeout)
    int kq
    const struct kevent *changelist
    int nchanges
    struct kevent *eventlist
    int nevents
    const struct timespec *timeout


    Thank you for any assistance in this.

    U. A. R. Ruirarchzatrea
    U. A. R. Ruirarchzatrea, Aug 4, 2004
    #1
    1. Advertising

  2. U. A. R. Ruirarchzatrea

    Uri Guttman Guest

    >>>>> "UARR" == U A R Ruirarchzatrea <> writes:

    UARR> I have been trying to get a Kqueue module written for Perl with
    UARR> XS that would allow the Kqueue event notification facility to be
    UARR> accessed from Perl. Kqueue is a scalable event notification
    UARR> facility often found on BSD OSs that allows you to recieve
    UARR> events from sockets for instance, and corrects the performance
    UARR> limitations of select, allowing for multiplexed servers to be
    UARR> written which can scale to very large numbers of connections. It
    UARR> does the same thing as epoll on Linux but does have a broader
    UARR> range of applications and features than epoll.

    UARR> Having a module avialable to access Kqueue would be of use to
    UARR> anyone wanting to write scalable network code on BSD OSs.


    UARR> Has anyone out there looked into implemented a Kqueue module? I
    UARR> was looking into doing it myself, but got confused how to do
    UARR> it. The goal is for a Kqueue module written which can be
    UARR> contributed to CPAN so other people who need to use Kqueue do
    UARR> not have to reinvent the wheel.

    check out using libevent (at http://monkey.org/~provos/libevent/)
    instead and wrapping that in perl via xs/swig/inline. that will be more
    effective in that it gives you /dev/poll, kqueue(2), select(2), poll(2)
    and epoll(4) support already and with one api wrapper you get all in
    perl. i would be willing to help out with this as i have written and
    wrapped event loops and done some (small) xs work (actually in swig).

    you shouldn't post the kqueue api here as it can be found on the
    net. and proposing a perl api for it is too early. first check out
    libevent and think about wrapping that. then get the extending and
    embedding perl book and read that. then work on the api (i can help
    there for sure). also look at event.pm (on cpan) and its api. there is
    plenty to do before you create the api and the actual wrapper code.

    uri

    --
    Uri Guttman ------ -------- http://www.stemsystems.com
    --Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
    Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
    Uri Guttman, Aug 4, 2004
    #2
    1. Advertising

  3. Uri Guttman <> wrote in message news:<>...

    > check out using libevent (at http://monkey.org/~provos/libevent/)
    > instead and wrapping that in perl via xs/swig/inline. that will be more
    > effective in that it gives you /dev/poll, kqueue(2), select(2), poll(2)
    > and epoll(4) support already and with one api wrapper you get all in
    > perl. i would be willing to help out with this as i have written and
    > wrapped event loops and done some (small) xs work (actually in swig).
    >


    Thank you for the advice. I think doing a libevent interface i an
    excellent idea. But I also think that there is still is a need for a
    interface to kqueue, and that is on my list of things to study doing.
    There are actually many event engines written in Perl that could
    benefit from it, such as POE. Doing an engine in Perl has its
    advantages, as it is eisier to understand the code and to improve upon
    it, and there is less chance of memory buffer overflows.
    U. A. R. Ruirarchzatrea, Aug 4, 2004
    #3
  4. U. A. R. Ruirarchzatrea

    Uri Guttman Guest

    >>>>> "UARR" == U A R Ruirarchzatrea <> writes:

    UARR> Uri Guttman <> wrote in message news:<>...
    >> check out using libevent (at http://monkey.org/~provos/libevent/)
    >> instead and wrapping that in perl via xs/swig/inline. that will be more
    >> effective in that it gives you /dev/poll, kqueue(2), select(2), poll(2)
    >> and epoll(4) support already and with one api wrapper you get all in
    >> perl. i would be willing to help out with this as i have written and
    >> wrapped event loops and done some (small) xs work (actually in swig).
    >>


    UARR> Thank you for the advice. I think doing a libevent interface i an
    UARR> excellent idea. But I also think that there is still is a need for a
    UARR> interface to kqueue, and that is on my list of things to study doing.
    UARR> There are actually many event engines written in Perl that could
    UARR> benefit from it, such as POE. Doing an engine in Perl has its
    UARR> advantages, as it is eisier to understand the code and to improve upon
    UARR> it, and there is less chance of memory buffer overflows.

    i don't think you got what i meant. wrap libevent in perl and then stem
    and poe and others can use libevent which already wraps kqueue and
    several other things. for the amount of work to just wrap kqueue in
    perl, you get several event interfaces in perl by wrapping libevent.

    focus your effort where it will do the most good and libevent is that
    for you.

    uri

    --
    Uri Guttman ------ -------- http://www.stemsystems.com
    --Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
    Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
    Uri Guttman, Aug 5, 2004
    #4
  5. Uri Guttman <> wrote in message news:<>...
    > >>>>> "UARR" == U A R Ruirarchzatrea <> writes:

    >
    > UARR> Uri Guttman <> wrote in message news:<>...


    > i don't think you got what i meant. wrap libevent in perl and then stem
    > and poe and others can use libevent which already wraps kqueue and
    > several other things. for the amount of work to just wrap kqueue in
    > perl, you get several event interfaces in perl by wrapping libevent.
    >
    > focus your effort where it will do the most good and libevent is that
    > for you.
    >
    > uri



    I think that doing a libevent wrapper is a good idea, however, there
    still needs to be a direct interface to Kqueue for various reasons,
    especially if a Perl programmer wants to be able to write the event
    loop in Perl. It should be a purely trivial task to wrap Kqueue in a
    wrapper for an XS expert, I see no reason not to.

    Furthermore, it does not seem that libevent supports all of Kqueue
    features.
    U. A. R. Ruirarchzatrea, Aug 6, 2004
    #5
  6. U. A. R. Ruirarchzatrea

    Rocco Caputo Guest

    On 4 Aug 2004 09:46:25 -0700, U. A. R. Ruirarchzatrea wrote:
    > Uri Guttman <> wrote in message news:<>...
    >
    >> check out using libevent (at http://monkey.org/~provos/libevent/)
    >> instead and wrapping that in perl via xs/swig/inline. that will be more
    >> effective in that it gives you /dev/poll, kqueue(2), select(2), poll(2)
    >> and epoll(4) support already and with one api wrapper you get all in
    >> perl. i would be willing to help out with this as i have written and
    >> wrapped event loops and done some (small) xs work (actually in swig).
    >>

    >
    > Thank you for the advice. I think doing a libevent interface i an
    > excellent idea. But I also think that there is still is a need for a
    > interface to kqueue, and that is on my list of things to study doing.
    > There are actually many event engines written in Perl that could
    > benefit from it, such as POE. Doing an engine in Perl has its
    > advantages, as it is eisier to understand the code and to improve upon
    > it, and there is less chance of memory buffer overflows.


    Have a look at the POE::Loop documentation if you're considering
    interfacing POE to BSD's kqueue. For portability's sake, all it really
    requires is I/O events and a timeout.

    --
    Rocco Caputo - http://poe.perl.org/
    Rocco Caputo, Aug 7, 2004
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Bill

    closing a kqueue

    Bill, Jan 25, 2007, in forum: C Programming
    Replies:
    4
    Views:
    371
  2. Bill

    Stopping a kqueue

    Bill, Jan 25, 2007, in forum: C++
    Replies:
    3
    Views:
    331
    Dizzy
    Jan 26, 2007
  3. Ritesh Nadhani

    Re: Using select.kqueue()

    Ritesh Nadhani, Sep 3, 2009, in forum: Python
    Replies:
    0
    Views:
    493
    Ritesh Nadhani
    Sep 3, 2009
  4. Ritesh Nadhani

    select.kqueue.control changelist argument

    Ritesh Nadhani, Sep 3, 2009, in forum: Python
    Replies:
    0
    Views:
    291
    Ritesh Nadhani
    Sep 3, 2009
  5. harijay
    Replies:
    0
    Views:
    301
    harijay
    Jan 25, 2011
Loading...

Share This Page