[25% OT] C library integral part of OS/kernel???

[Romeo Colacitti]

Is the C library of most OSes (i.e, unix type OSes) implemented at the
very low kernel or just outside kernel level?

Looking through the source tree of Linux/BSDs it seems like the C
library is intertwined with the OS (string.h and other headers are
there). So does this mean that when I program in C and use standard
library functions, it's similar to calling the OS specific APIs (same
type of performance)?

Seeing all these Standard Library functions being implemented at the
kernel/OS level makes me want to use C more, because it obviously means
other languages are implemented at a level "more distant" from the
system (possibly calling the C library functions????).

Is my thinking correct or am I way off? Am I getting too excited by
seeing standard library functions being implemented in OS kernel code?

Clarification will be appreciated.

 

[Alexei A. Frounze]

Is the C library of most OSes (i.e, unix type OSes) implemented at the
very low kernel or just outside kernel level?

Looking through the source tree of Linux/BSDs it seems like the C
library is intertwined with the OS (string.h and other headers are
there). So does this mean that when I program in C and use standard
library functions, it's similar to calling the OS specific APIs (same
type of performance)?

Seeing all these Standard Library functions being implemented at the
kernel/OS level makes me want to use C more, because it obviously means
other languages are implemented at a level "more distant" from the
system (possibly calling the C library functions????).

Is my thinking correct or am I way off? Am I getting too excited by
seeing standard library functions being implemented in OS kernel code?

Clarification will be appreciated.

The thing is, the standard C library MUST be based on something. This
something is the OS kernel (actually, this is true not only of C, it's like
that with any other language and its standard library). So, it doesn't
matter how tight and explicit the relationship between the two is, it is
there by definition.
On the other hand, lots of the standard C functions are generic and useful
enough to be used inside the OS kernel itself. Yes, those string functions,
conversion functions, I/O functions, whatever. So, the kernel MAY and often
DOES use them too.
The key difference is that the kernel provides some base functionality,
usually very low-level. The standard C library and the applications are
built on top of that, these are higher-level. BUT (!) there's nothing that
prevents you from using the higher-level functions in the kernel. I mean,
the kernel does use its own low level functionality and may use the high
level functionality, which is based on its low level funcs... You see, if
you have sprintf() and want to use it in the kernel, you can do that -- you
don't need to have a special version of sprintf() just for the kernel
itself. The 2nd would contribute to the bloat. You could use open()/fopen()
in the kernel, you should be able to because the kernel (plus the drivers)
will provide you with file access anyway, even though it's on low level. If
high level is more convenient for use yet doesn't compromise the kernel, do
use it in the kernel.

Alex

 

[Eric Sosman]

Is the C library of most OSes (i.e, unix type OSes) implemented at the
very low kernel or just outside kernel level?

"Yes."

From the C Standard's point of view (you've posted to
comp.lang.c), the library is part of "the implementation"
and the internal arrangements of the implementation are
whatever the implementor chooses. The Standard describes
what the implementation (including the library) must do,
but not how it must be done.

In most implementations, some parts of the library are
entirely "user-land code" but others require some amount of
help from the O/S. It's likely that qsort() runs entirely
in user land, but it's likely that fopen() is a mixture of
user and kernel code.

P.J. Plauger's "The Standard C Library" is a good
exposition of a typical (C90) library implementation. (Yes:
I seem to have plugged this book quite a lot recently, but no:
I don't get a commission ...)

Looking through the source tree of Linux/BSDs it seems like the C
library is intertwined with the OS (string.h and other headers are
there). So does this mean that when I program in C and use standard
library functions, it's similar to calling the OS specific APIs (same
type of performance)?

Different systems will have different performance. That's
about all the typing I care to do on this very complex subject;
others may have more stamina.

Seeing all these Standard Library functions being implemented at the
kernel/OS level makes me want to use C more, because it obviously means
other languages are implemented at a level "more distant" from the
system (possibly calling the C library functions????).

Non sequitur: the fact that something is "in the kernel"
doesn't in and of itself give a speed advantage. On most
systems, crossing and recrossing the user/kernel boundary
exacts a time penalty (part of the job of <stdio.h> is to
reduce the number of crossings). Even if the in-kernel code
for this or that function is faster than a user-land version,
a program that needs to hop in and out of the kernel to use
that faster code may actually run more slowly. Don't drive
ten kilometers to avoid one traffic signal.

Also, if the performance of individual micro-operations is
all-important to you, you should not use ANY high-level language,
be it C, Lisp, Java, Ada, or COBOL. For any given programming
task, the O/S is too general and hence too slow; discard it and
implement your own specialized version. And don't use assembly
language: use hand-crafted numeric machine code. You'll have
the fastest imaginable implementation -- for a machine that will
have become obsolete before you finish your code ... Performance
is far from the be-all and end-all of good programming.

Is my thinking correct or am I way off? Am I getting too excited by
seeing standard library functions being implemented in OS kernel code?

I think you are. YMMV.

 

[Keith Thompson]

Non sequitur: the fact that something is "in the kernel"
doesn't in and of itself give a speed advantage. On most
systems, crossing and recrossing the user/kernel boundary
exacts a time penalty (part of the job of <stdio.h> is to
reduce the number of crossings). Even if the in-kernel code
for this or that function is faster than a user-land version,
a program that needs to hop in and out of the kernel to use
that faster code may actually run more slowly. Don't drive
ten kilometers to avoid one traffic signal.

Some C library functions can be implemented straightforwardly in C.
strlen() is a good example. If the OS kernel is written in C, it's
going to need some sort of C library (though it may not be a fully
conforming hosted implementation).

My guess is that there's only one implementation of the strlen()
function, and that it's callable either from the kernel or from user
code without crossing the user/kernel boundary. It's just a function
call. The same is going to be true for a lot of other C library
functions.

 

[Romeo Colacitti]

snip


My guess is that there's only one implementation of the strlen()
function, and that it's callable either from the kernel or from user
code without crossing the user/kernel boundary. It's just a function
call. The same is going to be true for a lot of other C library
functions.

I've even seen some "redefinitions" of library functions in kernel code
for unix-type operating systems. Something like:

#include <stdio.h>
int getchar()
{
return getchar();
}

So this suggests that these kernel writers are trying to get these C
functions to "propogate" in future versions of the kernel - I think.
So I guess compiling the code for the kernel, creates the standard
library in the kernel itself. It seems like the standard library is as
much important to these OSes as POSIX functions, etc. I guess this is
another reason why C is considered efficient - it's library is closer
to kernel (or part of it).

 

[Maxim S. Shatskih]

kernel/OS level makes me want to use C more, because it obviously means

other languages are implemented at a level "more distant" from the
system (possibly calling the C library functions????).

Yes. Usually, it is so.

 

[Eric Sosman]

I've even seen some "redefinitions" of library functions in kernel code
for unix-type operating systems. Something like:

#include <stdio.h>
int getchar()
{
return getchar();
}

I hope "something like" means "rather different."
There are two invocations of undefined behavior in the
above, and if they don't bite you the remaining problem
surely will ...

Recursion (n): see "recursion."

 

[Dan Pop]

My guess is that there's only one implementation of the strlen()
function, and that it's callable either from the kernel or from user
code without crossing the user/kernel boundary. It's just a function
call. The same is going to be true for a lot of other C library
functions.

Your guess is typically wrong: a kernel is a freestanding program
and it doesn't rely on any external library, not even the standard
C library. It defines all the standard C library functions it uses,
which is OK for an application translated in freestanding mode.

The source code of the kernel's strlen() may be the same as the
source code of the C library's strlen(), but they're still completely
separate entities, that can be maintained independently (kernel people
and library people are not necessarily the same). Nontrivial
functions, like malloc and friends may have completely different
implementations.

Dan

 

[Luke Wu]

Is the C library of most OSes (i.e, unix type OSes) implemented at the
very low kernel or just outside kernel level?

Looking through the source tree of Linux/BSDs it seems like the C
library is intertwined with the OS (string.h and other headers are
there). So does this mean that when I program in C and use standard
library functions, it's similar to calling the OS specific APIs (same
type of performance)?

The C library is usally integrated into the OS, but not the kernel part
of it. The code you saw in the kernel code is for use by the kernel
developers. But part of what you're saying is right (see below).

Seeing all these Standard Library functions being implemented at the
kernel/OS level makes me want to use C more, because it obviously means
other languages are implemented at a level "more distant" from the
system (possibly calling the C library functions????).

Is my thinking correct or am I way off? Am I getting too excited by
seeing standard library functions being implemented in OS kernel code?

Clarification will be appreciated.

The library is implemented as part of the userland of the OS (which is
GNU not linux) - libc. That's why all the standard C library functions
are in the man pages, where as the C++, python, perl are not. The
standard C library is considered part of the "OS's SYSTEM CALLS." So
it is in the 'OS' , but not the kernel (the userland part of the OS).
This can only be said for the C library, no the C++, perl, python etc
(which incidentally could be calling/using the C library system calls)
- outside the userland.

 

[Luke Wu]

Romeo Colacitti wrote:

The library is implemented as part of the userland of the OS (which is
GNU not linux) - libc. That's why all the standard C library functions
are in the man pages, where as the C++, python, perl are not. The
standard C library is considered part of the "OS's SYSTEM CALLS." So
it is in the 'OS' , but not the kernel (the userland part of the OS).
This can only be said for the C library, no the C++, perl, python etc
(which incidentally could be calling/using the C library system calls)
- outside the userland.

This diagram better compares the API provided by other languages, with
the API provided by C (for unix/linux). As you can see, LIBC is
"closer" to the kernel, but not "in" the kernel.

JAVA API
|
\/
JVM
/ \
POSIX LIBC(c api)
| |
\/ \/
SYSTEM CALLS
|
_____ \/_____
| KERNEL |
|___________|

 

[Minti]

Is the C library of most OSes (i.e, unix type OSes) implemented at the
very low kernel or just outside kernel level?

That would depend on what your definition of _most_ is. If by most you
mean Operating Systems that are _USED_ the most, then yes you are
right, but these days I see a lot of {new} Operating systems that are
trying to move away from the monolithic design paradigm. In these
{[micro][nano][pico]}kernel architecture there is very little need of
adding a large part of the C library {seperately} in the Operating
System Kernel. They could very well implement a substantial portion the
C library at the _user_ level. Under which case the Operating System's
C library and yours {c/w}ould be same.

Looking through the source tree of Linux/BSDs it seems like the C
library is intertwined with the OS (string.h and other headers are
there).

That the string functions were implemented in <linux/string.h> is
nothing _but_ a mere coincidence the developers could have named it

So does this mean that when I program in C and use standard
library functions, it's similar to calling the OS specific APIs (same
type of performance)?

Most of the C libary calls that you make, printf, scanf, malloc, free,
fopen, fclose etc. require _at-least_ 2 levels

a) Your [gcc or MS] C libary.
b) Operating System underneath.

Though in case of Operating Systems like Linux, if you are using
functions like kprintf, kmalloc and strlen etc you are directly linking
your code with library. Which in a sense could be considered to be
static linking of your calls.

Seeing all these Standard Library functions being implemented at the
kernel/OS level makes me want to use C more, because it obviously means
other languages are implemented at a level "more distant" from the
system (possibly calling the C library functions????).

In way you are quite right, there are people who consider .NET and Java
to be _mere_ wrappers. I don't[%].

Is my thinking correct or am I way off? Am I getting too excited by
seeing standard library functions being implemented in OS kernel

code?



The very important point that favors C being used in Operating Systems
is that no other compiler-for-language-other-than-c has been able to
been able to beat code-generated-by-c-compilers. It is very much
possible to write an Operating System within a programming language
like Java, with a pinch of salt actually, and then generating the
native code for a platform like i386, again with a pinch of salt. There
is at least one Operating System Project that attempts to do so.



[%] An interesting point is that .NET's CLR was implemented in C++,
actually first in LISP the translated to C++. I am quite sure that JRE
is also implemented in a similar fashion. Why this is so, is left as an
exercise for the interested reader.



--
Imanpreet Singh Arora
If I am given 6 hours to chop a tree, I will spend the
first 4 sharpening my axe.
-- A.L.
It's been 10, Can I stop now?
-- Nisha

 

[Ravi Uday]

Your guess is typically wrong: a kernel is a freestanding program
and it doesn't rely on any external library, not even the standard
C library. It defines all the standard C library functions it uses,
which is OK for an application translated in freestanding mode.

So, when an executable gets built, then there are two copies of C
library functions (one copy of functions used by kernel and other by
user/app level) !! An app might end up using user level c functions but
dont you think this approach increases the executable size :-(
in the end ?

- Ravi

 

[Mark McIntyre]

So, when an executable gets built, then there are two copies of C
library functions

Maybe, maybe not. Dan's guess is probably as wrong as Keiths, and both are
just as possibly right. Indeed if you read what Keith said, its not even
incompatible with Dan's comment.

For example the kernel may provide a service to allow userland apps to call
its version of the C library functions. Or the two may both use an
physically external library and there may be one shared copy in memory, or
two separate copies one in userspace and one in kernelspace, or there may
be two or more physical copies of the library on disk - consider that you
probably have executables built with microsoft, gnu and borland C, each
providing their own library, possibly more than one, if different versions
of the compiler were used.

 

[Maxim S. Shatskih]

So, when an executable gets built, then there are two copies of C

library functions (one copy of functions used by kernel and other by
user/app level) !!

Yes. Otherwise, you would need to cross the kernel-user boundary for each
strlen(). This is a bad idea.

An app might end up using user level c functions but
dont you think this approach increases the executable size :-(

Usually such increase is not noticeable anyway.

 

[Minti]

So, when an executable gets built, then there are two copies of C
library functions (one copy of functions used by kernel and other by
user/app level) !! An app might end up using user level c functions but
dont you think this approach increases the executable size :-(
in the end ?

- Ravi

What executable? When you create your daily-day executables all it does
is interact with about 250 calls on Linux and around 1500(?) calls in
case of Windows environment. There isn't essentially a sort of thing
like there is a printf in user enviroment or user library and a printf
in Kernel. Both exist seperately for a very good reason.


As regards to increasing the size it is quite possible for your code to
directly call "strlen"[%] of kernel but there is a very good reason
this approach is not followed. Also, most executables now use shared C
libraries and since most programs in Linux, or at least the ones I use,
are command line printf is shared among all of these programs.


That said you make a very good point for micro-kernel based Operating
Systems.


[%] At least in Linux this could done by editing entry.S and couple of
other hacks.

 

[DHOLLINGSWORTH2]

Very simply put.

The OS needs no C. and as such, niether is integrated with the other.

C compilers link low level code to interact with the OS.

In the "early days" most of this was done using the BIOS standard interface.
Wich exists without an OS. So you could develop an OS with C that does not
interact with any OS resources. Or you could say that the BIOS is an
integral componant of Every OS.

Keep them seperate in your mind. C is C, and OS is OS.
no integration.

Now, Before someone corrects me.
some OS's do have C compilers as an integral componant of that particular
OS. These are the exceptions, and some of the better OS's.

But C, will never, (dare I say never), integrate any particular OS.

So, when an executable gets built, then there are two copies of C
library functions (one copy of functions used by kernel and other by
user/app level) !! An app might end up using user level c functions but
dont you think this approach increases the executable size :-(
in the end ?

- Ravi

What executable? When you create your daily-day executables all it does
is interact with about 250 calls on Linux and around 1500(?) calls in
case of Windows environment. There isn't essentially a sort of thing
like there is a printf in user enviroment or user library and a printf
in Kernel. Both exist seperately for a very good reason.


As regards to increasing the size it is quite possible for your code to
directly call "strlen"[%] of kernel but there is a very good reason
this approach is not followed. Also, most executables now use shared C
libraries and since most programs in Linux, or at least the ones I use,
are command line printf is shared among all of these programs.


That said you make a very good point for micro-kernel based Operating
Systems.


[%] At least in Linux this could done by editing entry.S and couple of
other hacks.

 

[Dan Pop]

So, when an executable gets built, then there are two copies of C
library functions (one copy of functions used by kernel and other by
user/app level) !!

Nope. When an executable gets built, it contains at most one copy of
any given standard library function. With dynamic linking, it contains
no copy at all.

An app might end up using user level c functions but
dont you think this approach increases the executable size :-(
in the end ?

As long as applications don't contain the kernel code, I can't see how
the existence of two implementations of standard library functions
(one for kernel usage, one for hosted applications usage) is going to
affect the size of any application executable code.

OTOH, it is not uncommon for compilers to inline certain standard
library function calls. This does increase the executable size, but
it also accelerates the program execution.

Dan

 

[Eric Sosman]

Nope. When an executable gets built, it contains at most one copy of
any given standard library function. With dynamic linking, it contains
no copy at all.

Well, that's not always true. A fairly common counter-
example is the inline expansion of some "simple" Standard
library functions like sqrt() or memcpy(). An implementation
that performs such inlining may generate an executable with
as many "copies" of sqrt() as there are calls to it -- and
perhaps yet another, to be called via a function pointer!

(Are these inline expansions "functions?" Probably not,
from the point of view of the hardware: they won't have the
argument marshalling, call-and-return, and so forth that are
associated with ordinary subroutines. But from the point of
view of the C Standard they most certainly are "functions;"
the inline expansions are merely an implementation detail,
and all the Standard's language about "the fmod() function"
still governs the behavior of the code.)

 

[CBFalconer]

.... snip ...

OTOH, it is not uncommon for compilers to inline certain standard
library function calls. This does increase the executable size,
but it also accelerates the program execution.

Welcome back. I trust nothing especially evil has happened to you
in the interim.

 

[Dan Pop]

Well, that's not always true. A fairly common counter-
example is the inline expansion of some "simple" Standard
library functions like sqrt() or memcpy(). An implementation
that performs such inlining may generate an executable with
as many "copies" of sqrt() as there are calls to it -- and
perhaps yet another, to be called via a function pointer!

A possibility I have *explicitly* mentioned myself in the text you
have not included.

Dan