Threads and sockets

Discussion in 'Perl Misc' started by Martijn Lievaart, Nov 12, 2006.

  1. Hi,

    Sorry, no code. Normally I just code something up and work it out from
    there. I want to get a grasp on the basic concepts before starting to code
    for this one.

    I've written a small server that listens for UDP packets or TCP
    connections, computes an answer and returns the result. The trouble is
    that computing the answer may be almost instantanious, or may take several
    seconds. I want to make the server concurrent by using ithreads, creating
    n threads and distributing the requests over the threads using
    Threads::Queue. The thread itself should send back the result to the
    client.

    Now I understand the basics of :shared. I just cannot wrap this around to
    sockets.

    For TCP: Can I just enqueue a socket and use it in the thread that
    dequeues it? Or should the socket be marked shared? Or will this not work
    at all? Or depending on the OS (I'm using Linux).

    What would be an appropriate model for UDP? I probably can just create a
    UDP socket per thread and use that to send the answer, am I on the right
    track?

    Thanks in advance for any insight anyone can give on this.
    M4
    --
    Redundancy is a great way to introduce more single points of failure.
     
    Martijn Lievaart, Nov 12, 2006
    #1
    1. Advertising

  2. Martijn Lievaart

    Guest

    Martijn Lievaart <> wrote:
    > Hi,
    >
    > Sorry, no code. Normally I just code something up and work it out from
    > there. I want to get a grasp on the basic concepts before starting to
    > code for this one.
    >
    > I've written a small server that listens for UDP packets or TCP
    > connections, computes an answer and returns the result. The trouble is
    > that computing the answer may be almost instantanious, or may take
    > several seconds.


    Do you know which ahead of time? If so, I would just answer directly if it
    is instantaneous and fork if it is not, rather than using threads
    (assuming linux or similar system).

    > I want to make the server concurrent by using ithreads,
    > creating n threads and distributing the requests over the threads using
    > Threads::Queue. The thread itself should send back the result to the
    > client.


    Why not just use another Thread::Queue to put the answer in, and have the
    main thread send all the responses?

    > Now I understand the basics of :shared. I just cannot wrap this around to
    > sockets.
    >
    > For TCP: Can I just enqueue a socket and use it in the thread that
    > dequeues it?


    I don't think so. You can't enqueue objects. You could enqueue some kind
    of index or key that could be used to point to the socket residing in some
    other variable.

    > Or should the socket be marked shared?


    I get an error message "Cannot share globs yet" when I try.

    > Or will this not work
    > at all? Or depending on the OS (I'm using Linux).


    In my experience on linux, file handles (including sockets) don't need to
    be shared. When you create a new thread, a new Perl variable is created
    for the handle, but it points to the same underlying C/OS structure as the
    old one, so the socket is implicitly shared by all threads which are
    "downstream" from the thread that initiated the socket. You may need to
    create a shared (and hence lockable) sentinel variable to control access to
    the real socket, if such serialized control is necessary.

    >
    > What would be an appropriate model for UDP? I probably can just create a
    > UDP socket per thread and use that to send the answer,


    To the extent that that would work in unthreaded code, I don't see why it
    would stop working in threaded code.

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
     
    , Nov 12, 2006
    #2
    1. Advertising

  3. On Sun, 12 Nov 2006 23:38:29 +0000, xhoster wrote:

    >> I've written a small server that listens for UDP packets or TCP
    >> connections, computes an answer and returns the result. The trouble is
    >> that computing the answer may be almost instantanious, or may take
    >> several seconds.

    >
    > Do you know which ahead of time? If so, I would just answer directly if it
    > is instantaneous and fork if it is not, rather than using threads
    > (assuming linux or similar system).


    No I don't. Otherwise there would be no problem.

    >> I want to make the server concurrent by using ithreads,
    >> creating n threads and distributing the requests over the threads using
    >> Threads::Queue. The thread itself should send back the result to the
    >> client.

    >
    > Why not just use another Thread::Queue to put the answer in, and have the
    > main thread send all the responses?


    Because I don't see how I can both select() and dequeue() in the main
    thread. Or do you see another solution?

    I thought of another solution. I could use fork instead of threads and
    communicate over sockets or pipes. That way I can simply use select in the
    main thread to get both new requests and answers.

    This has another advantage. It makes signal handling much simpler. I think
    signals and threads cannot be used (simple) together so this is probably a
    better solution overall.

    The problem here is that I also want some kind of caching, that is why I
    looked at threads in the first place. Oh well, I can use an on disk cache
    as well.

    >> Now I understand the basics of :shared. I just cannot wrap this around to
    >> sockets.
    >>
    >> For TCP: Can I just enqueue a socket and use it in the thread that
    >> dequeues it?

    >
    > I don't think so. You can't enqueue objects. You could enqueue some kind


    I could not find that in the docs, do you have a pointer?

    > of index or key that could be used to point to the socket residing in some
    > other variable.


    Yes. Of course. Slaps forehead.

    >> Or will this not work
    >> at all? Or depending on the OS (I'm using Linux).

    >
    > In my experience on linux, file handles (including sockets) don't need to
    > be shared. When you create a new thread, a new Perl variable is created
    > for the handle, but it points to the same underlying C/OS structure as the
    > old one, so the socket is implicitly shared by all threads which are
    > "downstream" from the thread that initiated the socket. You may need to
    > create a shared (and hence lockable) sentinel variable to control access to
    > the real socket, if such serialized control is necessary.


    Slaps forehead again. Of course. I should not have searched the perl docs
    but the Linux docs.

    >> What would be an appropriate model for UDP? I probably can just create
    >> a UDP socket per thread and use that to send the answer,

    >
    > To the extent that that would work in unthreaded code, I don't see why
    > it would stop working in threaded code.


    Slaps forehead yet again.

    Thanks. This makes it much clearer.

    M4
    --
    Redundancy is a great way to introduce more single points of failure.
     
    Martijn Lievaart, Nov 13, 2006
    #3
  4. Martijn Lievaart

    Guest

    Martijn Lievaart <> wrote:
    >
    > >> I want to make the server concurrent by using ithreads,
    > >> creating n threads and distributing the requests over the threads
    > >> using Threads::Queue. The thread itself should send back the result to
    > >> the client.

    > >
    > > Why not just use another Thread::Queue to put the answer in, and have
    > > the main thread send all the responses?

    >
    > Because I don't see how I can both select() and dequeue() in the main
    > thread. Or do you see another solution?


    You could open a pipe from socket to itself. Have the slaves print one byte
    to it when they have just enqueued something. Then use select on both the
    accept handle and on the semaphor handle--when you get the semaphor thread
    you know you should check the queue.

    >
    > I thought of another solution. I could use fork instead of threads and
    > communicate over sockets or pipes. That way I can simply use select in
    > the main thread to get both new requests and answers.


    Yes, or that. For some reason I like using the pipe just as a semaphor.
    I guess because with Thread::Queue your messages are self-delimited, so
    you don't have to screw around with that.

    > This has another advantage. It makes signal handling much simpler. I
    > think signals and threads cannot be used (simple) together so this is
    > probably a better solution overall.
    >
    > The problem here is that I also want some kind of caching, that is why I
    > looked at threads in the first place. Oh well, I can use an on disk cache
    > as well.


    Checking the cache should be fast, right? With the forking method, you
    could only fork if the thing wasn't in the cache, and then use a pipe so
    the child can tell the parent how to update it's cache. Of course, if your
    cache is going to be (or likely become in the forseeable future) bigger
    than memory, then there is no reason to go to lengths to avoid a disk
    cache.

    > >> Now I understand the basics of :shared. I just cannot wrap this around
    > >> to sockets.
    > >>
    > >> For TCP: Can I just enqueue a socket and use it in the thread that
    > >> dequeues it?

    > >
    > > I don't think so. You can't enqueue objects. You could enqueue some
    > > kind

    >
    > I could not find that in the docs, do you have a pointer?


    Mostly I just got it from the error messages. Actually, you can enqueue
    objects, as long as they are "shared" before you try to enqueue them. But
    sharing objects isn't necessarily well supported, especially file handles,
    it seems

    >
    > > of index or key that could be used to point to the socket residing in
    > > some other variable.

    >
    > Yes. Of course. Slaps forehead.


    Actually, I'm not sure how this would work, either. See below.

    > >> Or will this not work
    > >> at all? Or depending on the OS (I'm using Linux).

    > >
    > > In my experience on linux, file handles (including sockets) don't need
    > > to be shared. When you create a new thread, a new Perl variable is
    > > created for the handle, but it points to the same underlying C/OS
    > > structure as the old one, so the socket is implicitly shared by all
    > > threads which are "downstream" from the thread that initiated the
    > > socket. You may need to create a shared (and hence lockable) sentinel
    > > variable to control access to the real socket, if such serialized
    > > control is necessary.


    However, this is all irrelevant to you. Since you are using pre-created
    threads, the relevant sockets will not exist at the time of thread
    creation, and so won't be shared through this automatic method. Maybe you
    can use something like File::FDpasser (I've never used it) to do it, or
    just get the file descriptor number using fileno and enqueue that, then
    reconstruct it at the end (I've never done that, either.)

    > Slaps forehead again. Of course. I should not have searched the perl docs
    > but the Linux docs.


    I don't this is documented there, either, as it is a result of the an
    interaction between how linux implements files and how Perl implements
    threads. My ideas in this area are mostly based on experience and
    experiments, not on the docs.

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
     
    , Nov 13, 2006
    #4
  5. Martijn Lievaart

    Guest

    wrote:
    > Martijn Lievaart <> wrote:
    > >
    > > >> I want to make the server concurrent by using ithreads,
    > > >> creating n threads and distributing the requests over the threads
    > > >> using Threads::Queue. The thread itself should send back the result
    > > >> to the client.
    > > >
    > > > Why not just use another Thread::Queue to put the answer in, and have
    > > > the main thread send all the responses?

    > >
    > > Because I don't see how I can both select() and dequeue() in the main
    > > thread. Or do you see another solution?

    >
    > You could open a pipe from socket to itself. Have the slaves print one
    > byte to it when they have just enqueued something.


    Obviously you could open a pipe from the *process* to itself. I don't
    how "socket" got in there. You could either have one pipe per slave,
    or one pipe total that all the saves share. In the latter case, it may
    get corrupted, but I don't think that will matter, as it only used
    as a semaphore. But I think it would still work, as long as each time you
    wake up, you would have to read from Thread::Queue till it is is empty
    (i.e. until it would block) just in case concurrent "posts" to the pipe got
    corrupted and so don't look like they are more than one.


    > Then use select on
    > both the accept handle and on the semaphor handle--when you get the
    > semaphor thread you know you should check the queue.


    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
     
    , Nov 13, 2006
    #5
  6. On Mon, 13 Nov 2006 13:36:28 +0000, zentara wrote:

    > On 12 Nov 2006 23:38:29 GMT, wrote:
    >
    >>Martijn Lievaart <> wrote:

    >
    >>> Or should the socket be marked shared?

    >>
    >>I get an error message "Cannot share globs yet" when I try.
    >>
    >>> Or will this not work
    >>> at all? Or depending on the OS (I'm using Linux).

    >
    > A couple of quick ideas( I may be missing your problem entirely) :)


    (snip)

    Thanks. Very informative.

    M4
    --
    Redundancy is a great way to introduce more single points of failure.
     
    Martijn Lievaart, Nov 13, 2006
    #6
  7. On Mon, 13 Nov 2006 15:06:59 +0000, xhoster wrote:

    > Martijn Lievaart <> wrote:
    >>
    >> >> I want to make the server concurrent by using ithreads,
    >> >> creating n threads and distributing the requests over the threads
    >> >> using Threads::Queue. The thread itself should send back the result to
    >> >> the client.
    >> >
    >> > Why not just use another Thread::Queue to put the answer in, and have
    >> > the main thread send all the responses?

    >>
    >> Because I don't see how I can both select() and dequeue() in the main
    >> thread. Or do you see another solution?

    >
    > You could open a pipe from socket to itself. Have the slaves print one byte
    > to it when they have just enqueued something. Then use select on both the
    > accept handle and on the semaphor handle--when you get the semaphor thread
    > you know you should check the queue.


    (seen your other post, got the correction).

    Yes, that's an idea. A very good idea at that, as I can pass perl data
    through the queue, but should Data::Dumper it to get it across a socket.

    >> This has another advantage. It makes signal handling much simpler. I
    >> think signals and threads cannot be used (simple) together so this is
    >> probably a better solution overall.


    And this promisses to be the killer. Threads and signals don't go
    together. I need timeouts.

    >> The problem here is that I also want some kind of caching, that is why I
    >> looked at threads in the first place. Oh well, I can use an on disk cache
    >> as well.

    >
    > Checking the cache should be fast, right? With the forking method, you
    > could only fork if the thing wasn't in the cache, and then use a pipe so
    > the child can tell the parent how to update it's cache. Of course, if your
    > cache is going to be (or likely become in the forseeable future) bigger
    > than memory, then there is no reason to go to lengths to avoid a disk
    > cache.


    Well I was thinking about preforking, not forking, so the penalty
    shouldn't be to bad. Using a cache only in the main thread would also
    work but shifts some work back to the main thread, so this would mean some
    re-coding the existing worker routine. Oh well, it actually is somewhat
    cleaner, even if somewhat less efficient on a cache miss (which will be
    frequent, the pattern is most of the time no cache hit, but sometimes a
    lot of cache hits).

    >> >> Now I understand the basics of :shared. I just cannot wrap this around
    >> >> to sockets.
    >> >>
    >> >> For TCP: Can I just enqueue a socket and use it in the thread that
    >> >> dequeues it?
    >> >
    >> > I don't think so. You can't enqueue objects. You could enqueue some
    >> > kind

    >>
    >> I could not find that in the docs, do you have a pointer?

    >
    > Mostly I just got it from the error messages. Actually, you can enqueue
    > objects, as long as they are "shared" before you try to enqueue them. But
    > sharing objects isn't necessarily well supported, especially file handles,
    > it seems


    Right. Noted. Thanks. Sucks.

    This is getting way to complex (see also the note about signals). I'll
    settle for a preforked server, send responses back to the client in the
    parent proces and somehow get the data over a socket from parent to
    children and back.

    Actually the rewrite I mentioned above makes this more possible.

    The server is a very simple DNS server. It should be driven from bind
    trough a forward statement. The questions are alway a simple string. The
    answers are always A or TXT records, or an errorcode. No glue records,
    SOAs or other complications. These are faily simple to pass along a socket
    using a line oriented protocol which also makes debugging much simpler.
    The parent can then encode the RR and send it to the client.

    /me switches to desktop 5 and starts coding.

    Thanks for all the pointers

    M4
    --
    Redundancy is a great way to introduce more single points of failure.
     
    Martijn Lievaart, Nov 13, 2006
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jupiter5F

    Threads and Sockets

    Jupiter5F, Nov 14, 2003, in forum: C++
    Replies:
    4
    Views:
    2,336
    Jupiter5F
    Nov 14, 2003
  2. Julia Goolia

    tkinter, sockets and threads together

    Julia Goolia, Sep 11, 2003, in forum: Python
    Replies:
    4
    Views:
    554
    Steve Holden
    Sep 12, 2003
  3. Gonçalo Rodrigues

    Help needed in problem with Threads and sockets.

    Gonçalo Rodrigues, Dec 29, 2003, in forum: Python
    Replies:
    0
    Views:
    292
    Gonçalo Rodrigues
    Dec 29, 2003
  4. Rod Stephenson
    Replies:
    0
    Views:
    467
    Rod Stephenson
    Jun 22, 2004
  5. Ajay

    threads and sockets

    Ajay, Oct 13, 2004, in forum: Python
    Replies:
    3
    Views:
    311
    Elbert Lev
    Oct 14, 2004
Loading...

Share This Page