Queue in C99?

[cerr]

Hi There,

I'm writing a multithreaded logging client. My main app. would call e
log function, the log function would take the string, copy it into a
queue and a thread running on the other side would fetch data from the
queue and send them off to the server.
How do I accomplish that best in C99 if I don't have the std namespace
available?
Thank you,
Ron

 

[Antoninus Twink]

I'm writing a multithreaded logging client. My main app. would call e
log function, the log function would take the string, copy it into a
queue and a thread running on the other side would fetch data from the
queue and send them off to the server.

The simplest arrangement would just be to have a single mutex protecting
the queue.

To add a message to be logged, the first thread acquires the mutex,
enqueues the string, increments number_of_things_in_queue, and then
releases the mutex.

Meanwhile, the logger thread periodically tries to acquire the mutex.
When it's holding the mutex, it checks to see whether
number_of_things_in_queue is > 0. If so, it dequeues a string and
decrements number_of_things_in_queue. Then it releases the mutex.

Of course, more complicated things may be needed in practise depending
on how frequently your program generates messages to be logged.

If you wanted to avoid multithreading but still eliminate I/O latency
when writing log messages to disk, you could also look into the
asynchronous I/O facilities provided by your operating system.

 

[Keith Thompson]

I'm writing a multithreaded logging client. My main app. would call e
log function, the log function would take the string, copy it into a
queue and a thread running on the other side would fetch data from the
queue and send them off to the server.
How do I accomplish that best in C99 if I don't have the std namespace
available?

What does the std namespace (a C++ feature) have to do with it? Do
you mean that you don't have the C++ standard library (STL) available?

Write or find software in C that manages queues. (Or you might
consider implementing part or all of the program in C++.)

Since you're using threads, comp.programming.threads might be a
better place to ask.

 

[cerr]

What does the std namespace (a C++ feature) have to do with it?  Do
you mean that you don't have the C++ standard library (STL) available?

Exactly, i wanna avoid using elements from the stl.

Write or find software in C that manages queues.  (Or you might
consider implementing part or all of the program in C++.)

Since you're using threads, comp.programming.threads might be a
better place to ask.

--
Keith Thompson (The_Other_Keith) (e-mail address removed)  <http://www.ghoti.net/~kst>
Nokia
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

My actual problem isn't multi threading related but i'm wondering how
i can best have a queue available where i can push in data on on one
side and fetch them on the other. I'm aware of the fact that i'll need
to use mutexes, that's not of concern.

Thanks,
Ron

 

[Flash Gordon]

Hi There,

I'm writing a multithreaded logging client. My main app. would call e
log function, the log function would take the string, copy it into a
queue and a thread running on the other side would fetch data from the
queue and send them off to the server.
How do I accomplish that best in C99 if I don't have the std namespace
available?

Namespaces are nothing to do with this problem as far as I can see,
which is good since C does not have the concept (in the sense that C++
does for the std namespace etc).

One potentially simple(ish) option is to use a simple(ish) database for
your queue. I've used database tables to hold queues, and for what I
wanted (which was a different problem) it works well. Mind you, my
application was already linked against the database software.

The other solution is to investigate the logging facilities provided by
your OS, I know some of the "standard" Linux logging daemons support
remote logging, and I know there is similar stuff already available for
Windows as well.

Threads, and the issues with them, if you want to implement this stuff
yourself, are not topical here since they are not part of C but rather
part of the OS (or OS family) that you are using. So you would need to
ask on an appropriate OS specific group.

 

[Chris M. Thomasson]

The simplest arrangement would just be to have a single mutex protecting
the queue.

To add a message to be logged, the first thread acquires the mutex,
enqueues the string, increments number_of_things_in_queue, and then
releases the mutex.

Meanwhile, the logger thread periodically tries to acquire the mutex.
When it's holding the mutex, it checks to see whether
number_of_things_in_queue is > 0. If so, it dequeues a string and
decrements number_of_things_in_queue. Then it releases the mutex.

[...]

FWIW, you don't actually "need" any kind of counter in there unless you are
enforcing a limit on the number of items that can be in the queue.


Something simple like this will suffice:

<pseudo-code typed in news reader>
_____________________________________________________________
struct node
{
struct node* next;
};


struct queue
{
struct node* head; /* = NULL */
struct node* tail;
};


void
queue_push(struct queue* const self,
struct node* node)
{
node->next = NULL;

if (! self->head)
{
self->head = node;
}

else
{
self->tail->node;
}

self->tail = node;
}


struct node*
queue_pop(struct queue* const self)
{
struct node* node = self->head;

if (node)
{
self->head = node->next;
}

return node;
}




/* thread-safe queue */
struct tsqueue
{
struct queue queue;
pthread_mutex_t mutex;
pthread_cond_t cond;
};


void
tsqueue_push(struct tsqueue* const self,
struct node* node)
{
pthread_mutex_lock(&self->mutex);

queue_push(&self->queue, node);

pthread_mutex_unlock(&self->mutex);

pthread_cond_signal(&self->cond);
}


struct node*
tsqueue_try_pop(struct tsqueue* const self)
{
struct node* node;

pthread_mutex_lock(&self->mutex);

node = queue_pop(&self->queue);

pthread_mutex_unlock(&self->mutex);

return node;
}


struct node*
tsqueue_wait_pop(struct tsqueue* const self)
{
struct node* node;

pthread_mutex_lock(&self->mutex);

while (! (node = queue_pop(&self->queue)))
{
pthread_cond_wait(&self->cond, &self->mutex);
}

pthread_mutex_unlock(&self->mutex);

return node;
}
_____________________________________________________________




Of course you don't need any mutexs if you create a distributed design. One
that I personally like for logging is each application thread has a
single-producer/consumer queue, and the logger thread
periodically/episodically polls those queues. Another way is to use a
multi-producer/single-consumer queue, which does not need locks.

 

[Flash Gordon]

Exactly, i wanna avoid using elements from the stl.

If you are using C++ but not the STL this would still be the wrong place
to ask. If you are using C then it is implicit that you are not using
the STL as it is part of C++ and NOT C.

<snip>

Please don't quote peoples sig's, the bit typically after the "-- "
unless you are responding to the signature.

My actual problem isn't multi threading related but i'm wondering how
i can best have a queue available where i can push in data on on one
side and fetch them on the other. I'm aware of the fact that i'll need
to use mutexes, that's not of concern.

How best to implement a queue depends. If there is a reasonable fixed
known upper bound on on the size you can do a circular list. Or you can
do a linked list. Or a block of malloc'd memory you realloc as required.
Or a database table (either pre-existing or one you write yourself),
with that database either being entirely in memory, basically on disk,
or in memory but falling back to disk if required due to size, or a
facility provided by your OS or environment, or... each method has
advantages and disadvantages.