Threads and sockets

[Martijn Lievaart]

Hi,

Sorry, no code. Normally I just code something up and work it out from
there. I want to get a grasp on the basic concepts before starting to code
for this one.

I've written a small server that listens for UDP packets or TCP
connections, computes an answer and returns the result. The trouble is
that computing the answer may be almost instantanious, or may take several
seconds. I want to make the server concurrent by using ithreads, creating
n threads and distributing the requests over the threads using
Threads::Queue. The thread itself should send back the result to the
client.

Now I understand the basics of :shared. I just cannot wrap this around to
sockets.

For TCP: Can I just enqueue a socket and use it in the thread that
dequeues it? Or should the socket be marked shared? Or will this not work
at all? Or depending on the OS (I'm using Linux).

What would be an appropriate model for UDP? I probably can just create a
UDP socket per thread and use that to send the answer, am I on the right
track?

Thanks in advance for any insight anyone can give on this.
M4

 

[xhoster]

Hi,

Sorry, no code. Normally I just code something up and work it out from
there. I want to get a grasp on the basic concepts before starting to
code for this one.

I've written a small server that listens for UDP packets or TCP
connections, computes an answer and returns the result. The trouble is
that computing the answer may be almost instantanious, or may take
several seconds.

Do you know which ahead of time? If so, I would just answer directly if it
is instantaneous and fork if it is not, rather than using threads
(assuming linux or similar system).

I want to make the server concurrent by using ithreads,
creating n threads and distributing the requests over the threads using
Threads::Queue. The thread itself should send back the result to the
client.

Why not just use another Thread::Queue to put the answer in, and have the
main thread send all the responses?

Now I understand the basics of :shared. I just cannot wrap this around to
sockets.

For TCP: Can I just enqueue a socket and use it in the thread that
dequeues it?

I don't think so. You can't enqueue objects. You could enqueue some kind
of index or key that could be used to point to the socket residing in some
other variable.

Or should the socket be marked shared?

I get an error message "Cannot share globs yet" when I try.

Or will this not work
at all? Or depending on the OS (I'm using Linux).

In my experience on linux, file handles (including sockets) don't need to
be shared. When you create a new thread, a new Perl variable is created
for the handle, but it points to the same underlying C/OS structure as the
old one, so the socket is implicitly shared by all threads which are
"downstream" from the thread that initiated the socket. You may need to
create a shared (and hence lockable) sentinel variable to control access to
the real socket, if such serialized control is necessary.

What would be an appropriate model for UDP? I probably can just create a
UDP socket per thread and use that to send the answer,

To the extent that that would work in unthreaded code, I don't see why it
would stop working in threaded code.

Xho

 

[Martijn Lievaart]

Do you know which ahead of time? If so, I would just answer directly if it
is instantaneous and fork if it is not, rather than using threads
(assuming linux or similar system).

No I don't. Otherwise there would be no problem.

Why not just use another Thread::Queue to put the answer in, and have the
main thread send all the responses?

Because I don't see how I can both select() and dequeue() in the main
thread. Or do you see another solution?

I thought of another solution. I could use fork instead of threads and
communicate over sockets or pipes. That way I can simply use select in the
main thread to get both new requests and answers.

This has another advantage. It makes signal handling much simpler. I think
signals and threads cannot be used (simple) together so this is probably a
better solution overall.

The problem here is that I also want some kind of caching, that is why I
looked at threads in the first place. Oh well, I can use an on disk cache
as well.

I don't think so. You can't enqueue objects. You could enqueue some kind

I could not find that in the docs, do you have a pointer?

of index or key that could be used to point to the socket residing in some
other variable.

Yes. Of course. Slaps forehead.

In my experience on linux, file handles (including sockets) don't need to
be shared. When you create a new thread, a new Perl variable is created
for the handle, but it points to the same underlying C/OS structure as the
old one, so the socket is implicitly shared by all threads which are
"downstream" from the thread that initiated the socket. You may need to
create a shared (and hence lockable) sentinel variable to control access to
the real socket, if such serialized control is necessary.

Slaps forehead again. Of course. I should not have searched the perl docs
but the Linux docs.

To the extent that that would work in unthreaded code, I don't see why
it would stop working in threaded code.

Slaps forehead yet again.

Thanks. This makes it much clearer.

M4

 

[xhoster]

Because I don't see how I can both select() and dequeue() in the main
thread. Or do you see another solution?

You could open a pipe from socket to itself. Have the slaves print one byte
to it when they have just enqueued something. Then use select on both the
accept handle and on the semaphor handle--when you get the semaphor thread
you know you should check the queue.

I thought of another solution. I could use fork instead of threads and
communicate over sockets or pipes. That way I can simply use select in
the main thread to get both new requests and answers.

Yes, or that. For some reason I like using the pipe just as a semaphor.
I guess because with Thread::Queue your messages are self-delimited, so
you don't have to screw around with that.

This has another advantage. It makes signal handling much simpler. I
think signals and threads cannot be used (simple) together so this is
probably a better solution overall.

The problem here is that I also want some kind of caching, that is why I
looked at threads in the first place. Oh well, I can use an on disk cache
as well.

Checking the cache should be fast, right? With the forking method, you
could only fork if the thing wasn't in the cache, and then use a pipe so
the child can tell the parent how to update it's cache. Of course, if your
cache is going to be (or likely become in the forseeable future) bigger
than memory, then there is no reason to go to lengths to avoid a disk
cache.

I could not find that in the docs, do you have a pointer?

Mostly I just got it from the error messages. Actually, you can enqueue
objects, as long as they are "shared" before you try to enqueue them. But
sharing objects isn't necessarily well supported, especially file handles,
it seems

Yes. Of course. Slaps forehead.

Actually, I'm not sure how this would work, either. See below.

However, this is all irrelevant to you. Since you are using pre-created
threads, the relevant sockets will not exist at the time of thread
creation, and so won't be shared through this automatic method. Maybe you
can use something like File::FDpasser (I've never used it) to do it, or
just get the file descriptor number using fileno and enqueue that, then
reconstruct it at the end (I've never done that, either.)

Slaps forehead again. Of course. I should not have searched the perl docs
but the Linux docs.

I don't this is documented there, either, as it is a result of the an
interaction between how linux implements files and how Perl implements
threads. My ideas in this area are mostly based on experience and
experiments, not on the docs.

Xho

 

[xhoster]

You could open a pipe from socket to itself. Have the slaves print one
byte to it when they have just enqueued something.

Obviously you could open a pipe from the *process* to itself. I don't
how "socket" got in there. You could either have one pipe per slave,
or one pipe total that all the saves share. In the latter case, it may
get corrupted, but I don't think that will matter, as it only used
as a semaphore. But I think it would still work, as long as each time you
wake up, you would have to read from Thread::Queue till it is is empty
(i.e. until it would block) just in case concurrent "posts" to the pipe got
corrupted and so don't look like they are more than one.

Then use select on
both the accept handle and on the semaphor handle--when you get the
semaphor thread you know you should check the queue.

Xho

 

[Martijn Lievaart]

A couple of quick ideas( I may be missing your problem entirely)

(snip)

Thanks. Very informative.

M4

 

[Martijn Lievaart]

You could open a pipe from socket to itself. Have the slaves print one byte
to it when they have just enqueued something. Then use select on both the
accept handle and on the semaphor handle--when you get the semaphor thread
you know you should check the queue.

(seen your other post, got the correction).

Yes, that's an idea. A very good idea at that, as I can pass perl data
through the queue, but should Data:umper it to get it across a socket.

And this promisses to be the killer. Threads and signals don't go
together. I need timeouts.

Checking the cache should be fast, right? With the forking method, you
could only fork if the thing wasn't in the cache, and then use a pipe so
the child can tell the parent how to update it's cache. Of course, if your
cache is going to be (or likely become in the forseeable future) bigger
than memory, then there is no reason to go to lengths to avoid a disk
cache.

Well I was thinking about preforking, not forking, so the penalty
shouldn't be to bad. Using a cache only in the main thread would also
work but shifts some work back to the main thread, so this would mean some
re-coding the existing worker routine. Oh well, it actually is somewhat
cleaner, even if somewhat less efficient on a cache miss (which will be
frequent, the pattern is most of the time no cache hit, but sometimes a
lot of cache hits).

Mostly I just got it from the error messages. Actually, you can enqueue
objects, as long as they are "shared" before you try to enqueue them. But
sharing objects isn't necessarily well supported, especially file handles,
it seems

Right. Noted. Thanks. Sucks.

This is getting way to complex (see also the note about signals). I'll
settle for a preforked server, send responses back to the client in the
parent proces and somehow get the data over a socket from parent to
children and back.

Actually the rewrite I mentioned above makes this more possible.

The server is a very simple DNS server. It should be driven from bind
trough a forward statement. The questions are alway a simple string. The
answers are always A or TXT records, or an errorcode. No glue records,
SOAs or other complications. These are faily simple to pass along a socket
using a line oriented protocol which also makes debugging much simpler.
The parent can then encode the RR and send it to the client.

/me switches to desktop 5 and starts coding.

Thanks for all the pointers

M4