C runtime library for Unix

[Paul Edwards]

I would like to port PDPCLIB:

http://sourceforge.net/projects/pdos/

my public domain C runtime library for DOS, OS/2,
Win32, MVS to Unix now.

I am a bit perplexed as to how to do this. Unix/Posix
systems provide open() with which I can implement
fopen(). But then Posix is also required to define fopen(),
so this is not the level at which I need to implement it.

Although I don't think the CRT should be polluting the
namespace by using open() in the first place.

If the lower level is platform-dependent, then the platform
I wish to target is Linux.

Can someone get me started?

Thanks. Paul.

 

[santosh]

I would like to port PDPCLIB:

http://sourceforge.net/projects/pdos/

my public domain C runtime library for DOS, OS/2,
Win32, MVS to Unix now.

I am a bit perplexed as to how to do this. Unix/Posix
systems provide open() with which I can implement
fopen(). But then Posix is also required to define fopen(),
so this is not the level at which I need to implement it.

Although I don't think the CRT should be polluting the
namespace by using open() in the first place.

If the lower level is platform-dependent, then the platform
I wish to target is Linux.

Can someone get me started?

As a system library, you'll need to only rely on the APIs exported by
the OS. Linux aims towards POSIX compliance and thus implements many
APIs defined by POSIX. The system C library often also provides POSIX
interface.

In general you should try to do as much as possible within standard C
and write specific functions when they're called for. If you want the
library to port to most UNIXes, it's structure and compilation will get
more involved. If it's specifically for Linux, then it'll be much
simpler.

At a minimum you'll need to refer to the C standard, the POSIX
standards and the man 2 pages for the OS API. The tricky part will be
satisfying the OS without sacrificing too much portability.

Post Linux API related questions to one of the Linux groups. Details
within standard C portions can be discussed here or in comp.std.c, (but
I doubt that those areas would be much of a problem for you).

The Linux newsgroups will probably help you best. Also browse the
sources of C libraries like dietlibc etc.

 

[Paul Edwards]

Also browse the sources of C libraries like dietlibc etc.

Thanks. I found that, and their fopen() calls __libc_open()
which in turn calls __syscall_open().

The __syscall_open() is presumably what the OS exports,
and what I need to use to avoid polluting the user's namespace
by a direct call to open().

I haven't found where __syscall_open() is defined yet,
but I found a sys_open() in syscalls.h. Maybe that's the
proper one to use.

BFN. Paul.

 

[Tor Rustad]

Paul Edwards skrev:

[...]

I haven't found where __syscall_open() is defined yet,
but I found a sys_open() in syscalls.h. Maybe
that's the proper one to use.

sys_open()
sys_read()
sys_write()
sys_close()

BUT, why not look up the Linux kernel documentation or
ask in a forum where Linux is on-topic?

comp.os.linux.misc
comp.os.linux.development.system
comp.os.linux.development.apps
etc.

<gd&r>

 

[Eric Sosman]

Paul Edwards wrote On 11/17/06 15:20,:

Thanks. I found that, and their fopen() calls __libc_open()
which in turn calls __syscall_open().

The __syscall_open() is presumably what the OS exports,
and what I need to use to avoid polluting the user's namespace
by a direct call to open().

I haven't found where __syscall_open() is defined yet,
but I found a sys_open() in syscalls.h. Maybe that's the
proper one to use.

Why is it better to pollute the user's namespace
with sys_open() than with open()?

 

[santosh]

Thanks. I found that, and their fopen() calls __libc_open()
which in turn calls __syscall_open().

The __syscall_open() is presumably what the OS exports,
and what I need to use to avoid polluting the user's namespace
by a direct call to open().

I haven't found where __syscall_open() is defined yet,
but I found a sys_open() in syscalls.h. Maybe that's the
proper one to use.

No. This is still a feature of the existing C library. If you want your
CRT to be independant of other libraries, you must directly call the
kernel by means of assembler, (either INT, SYSCALL, SYSENTER etc). The
API exported by the kernel is available in the 'asm/unistd.h' header
for UNIX like kernels. For each defined syscall, refer to it's man 2
page for the official documentation, but they're often not good enough
to enable you to write robust code, (especially for newer system
calls). As the ultimate reference, the kernel's source may have to
consulted.

To enable the rest of your library to be portable, you'll probably want
to create wrappers around system calls, just like existing sys_open()
sys_write() etc.

 

[Paul Edwards]

Why is it better to pollute the user's namespace
with sys_open() than with open()?

Technically it isn't, but since no-one actually uses sys_open(),
I didn't think anyone would notice.

I have the same problem with Win32. It is polluting the user's
namespace with CreateFile(). I don't know how to get around
that. I wonder what other C compilers do?

BFN. Paul.

 

[Richard Heathfield]

Paul Edwards said:

Technically it isn't, but since no-one actually uses sys_open(),
I didn't think anyone would notice.

I have the same problem with Win32. It is polluting the user's
namespace with CreateFile(). I don't know how to get around
that. I wonder what other C compilers do?

Don't worry about it. Since no-one actually uses CreateFile() - except as a
Win32 API call - I don't think anyone will notice.

--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: normal service will be restored as soon as possible. Please do not
adjust your email clients.

 

[Tor Rustad]

Eric Sosman skrev:

Paul Edwards wrote On 11/17/06 15:20,:
[...]

The __syscall_open() is presumably what the OS exports,
and what I need to use to avoid polluting the user's namespace
by a direct call to open().

Why is it better to pollute the user's namespace
with sys_open() than with open()?

I don't get it. Why does the standard library pollute user's
namespace?

Internally, the library will have e.g. <open.c>, which calls the
OS functions. So those OS calls shouldn't be visable to the
library caller... so is it a link problem we talk about here?

 

[Paul Edwards]

Eric Sosman skrev:

Paul Edwards wrote On 11/17/06 15:20,:
[...]

The __syscall_open() is presumably what the OS exports,
and what I need to use to avoid polluting the user's namespace
by a direct call to open().

Why is it better to pollute the user's namespace
with sys_open() than with open()?

I don't get it. Why does the standard library pollute user's
namespace?

Internally, the library will have e.g. <open.c>, which calls the
OS functions. So those OS calls shouldn't be visable to the
library caller... so is it a link problem we talk about here?

Yes, the problem will occur at link time. The C library is
not meant to be dependent on functions which are in the
user's namespace. The C library is allowed to use names
such as __open() or _Open() (on a case-sensitive system),
but not open().

BFN. Paul.

 

[Eric Sosman]

Technically it isn't, but since no-one actually uses sys_open(),
I didn't think anyone would notice.

They might be *more* likely to notice than if you'd just
used open(). Programs written with portability in mind are
quite likely to avoid making native O/S calls directly, but
to route them through "wrapper" functions instead. On many
systems the wrappers will simply call the native interface
without further ado, but they provide a convenient place to
insert special handling for a system that requires it or can
take advantage of it. For example, a wrapper for open() might
just take its three arguments and call open() on most systems,
but on Frobozz Unix there's extra code to set a special mode
bits when opening anything under the "/var" directory -- for
some sort of esoteric FUX-specific reason you and I don't
understand.

Now: The wrapper can't be called open(), obviously. As
a developer of this portable program, what name are you likely
to choose? Actual examples I've seen include OSopen() and
OsOpen(), but it wouldn't surprise me in the least to find that
someone had decided to form wrapper names by prefixing sys_ to
everything ...

I have the same problem with Win32. It is polluting the user's
namespace with CreateFile(). I don't know how to get around
that. I wonder what other C compilers do?

Special games with linkers, if they bother at all. There'll
be some sort of magic that allows the user to write and use an
open() function while library denizens like fopen() can still
somehow call the system's open(). There'll be "weak symbols"
and possibly a lot of renaming going on -- or, sometimes, they'll
just say "Tough: You bought our system, you bought our names."
It's a jungle out there.

 

[Keith Thompson]

Eric Sosman skrev:

Paul Edwards wrote On 11/17/06 15:20,:
[...]

The __syscall_open() is presumably what the OS exports,
and what I need to use to avoid polluting the user's namespace
by a direct call to open().

Why is it better to pollute the user's namespace
with sys_open() than with open()?

I don't get it. Why does the standard library pollute user's
namespace?

Internally, the library will have e.g. <open.c>, which calls the
OS functions. So those OS calls shouldn't be visable to the
library caller... so is it a link problem we talk about here?

Yes, the problem will occur at link time. The C library is
not meant to be dependent on functions which are in the
user's namespace. The C library is allowed to use names
such as __open() or _Open() (on a case-sensitive system),
but not open().

I just tried a small test program that defines an external function
called "open". It worked with no problem. The systems I tried it on
also have a system function called "open"; you have to include a
non-standard header to make it visible. There was problem linking the
program. I don't know (or much care) how the system does this; it
just works.

 

[Spiros Bousbouras]

Eric Sosman skrev:
Paul Edwards wrote On 11/17/06 15:20,:

[...]

The __syscall_open() is presumably what the OS exports,
and what I need to use to avoid polluting the user's namespace
by a direct call to open().

Why is it better to pollute the user's namespace
with sys_open() than with open()?

I don't get it. Why does the standard library pollute user's
namespace?

Internally, the library will have e.g. <open.c>, which calls the
OS functions. So those OS calls shouldn't be visable to the
library caller... so is it a link problem we talk about here?

Yes, the problem will occur at link time. The C library is
not meant to be dependent on functions which are in the
user's namespace. The C library is allowed to use names
such as __open() or _Open() (on a case-sensitive system),
but not open().

I just tried a small test program that defines an external function
called "open". It worked with no problem. The systems I tried it on
also have a system function called "open"; you have to include a
non-standard header to make it visible. There was problem linking the
program. I don't know (or much care) how the system does this; it
just works.

Do you mean there was NO problem linking ?

 

[Keith Thompson]

Keith Thompson wrote: [...]

I just tried a small test program that defines an external function
called "open". It worked with no problem. The systems I tried it on
also have a system function called "open"; you have to include a
non-standard header to make it visible. There was problem linking the
program. I don't know (or much care) how the system does this; it
just works.

Do you mean there was NO problem linking ?

Yes.

% cat foo.h
#ifndef FOO_H
#define FOO_H
void open(void);
#endif

% cat foo.c
#include <stdio.h>
#include "foo.h"
void open(void)
{
printf("In open()\n");
}

% cat main.c
#include "foo.h"

int main(void)
{
open();
return 0;
}
% gcc foo.c main.c -o main
% ./main
In open()

 

[CBFalconer]

.... snip ...

I just tried a small test program that defines an external function
called "open". It worked with no problem. The systems I tried it
on also have a system function called "open"; you have to include a
non-standard header to make it visible. There was problem linking
the program. I don't know (or much care) how the system does this;
it just works.

This will normally be because the run time library is searched
last, and only for items that are so far undefined. This is not
mandated by the standard, but is in practice the methodology. The
success is also affected by how finely the library is sub-divided,
for example the code block for malloc will usually include both
free and realloc, so they can only be replaced as a complete triad.

 

[Keith Thompson]

This will normally be because the run time library is searched
last, and only for items that are so far undefined. This is not
mandated by the standard, but is in practice the methodology. The
success is also affected by how finely the library is sub-divided,
for example the code block for malloc will usually include both
free and realloc, so they can only be replaced as a complete triad.

And the result of this is that a C program can use any identifier it
likes as long as it's not reserved by the standard, even if it's one
like "open" (or "sys_open", or whatever) that might be used by the
operating system.

 

[Tor Rustad]

Paul Edwards skrev:

Yes, the problem will occur at link time. The C library is
not meant to be dependent on functions which are in the
user's namespace. The C library is allowed to use names
such as __open() or _Open() (on a case-sensitive system),
but not open().

I still don't get it...

On Linux you go from user space to kernel space via interrupt
0x80, using inline assembler... ref. <asm/unistd.h> and
_syscallN() macros... right?

On Win32 you load kernel32.dll and call the functions via
functions pointers... right?

So exactly how do the linker see any function names which
is invading the user name space?

What am I missing???

 

[Paul Edwards]

Paul Edwards skrev:

I still don't get it...

On Linux you go from user space to kernel space via interrupt
0x80, using inline assembler... ref. <asm/unistd.h> and
_syscallN() macros... right?

Well I didn't know that until you told me just now.

On Win32 you load kernel32.dll and call the functions via
functions pointers... right?

Well, not the way I'm doing it. I'm using the published
interface, which means I have coded CreateFile() in my
implementation of fopen(), which means I am polluting
the namespace.

So exactly how do the linker see any function names which
is invading the user name space?

What am I missing???

You're missing the fact that I thought it was a good idea to
use the OS's published interface, at least for Win32 and
OS/2. I didn't have this problem in DOS because the only
interface they provided was assembler interrupts. And
now I am looking into Unix and wondering what I should
use as the "official interface".

BFN. Paul.

 

[Paul Edwards]

I just tried a small test program that defines an external function
called "open". It worked with no problem. The systems I tried it on
also have a system function called "open";

Your test is inadequate because you wouldn't see the
problem unless you did an fopen() and suddenly your
code got executed by the C library.

BFN. Paul.

 

[Tor Rustad]

Paul Edwards skrev:

[...]

On Linux you go from user space to kernel space via interrupt
0x80, using inline assembler... ref. <asm/unistd.h> and
_syscallN() macros... right?

Well I didn't know that until you told me just now.

No problem.

Well, not the way I'm doing it. I'm using the published
interface, which means I have coded CreateFile() in my
implementation of fopen(), which means I am polluting
the namespace.

With the C compilers I have used at least, unless using
function pointers... a dynamicly linked Win32 function
was prefixed with:

__declspec(dllimport)

which translate e.g. CreateFile(...) into

__imp__CreateFile(...)

i.e. no polluting of user's name space at link time. I have
never used gcc on Win32... so if this is really a problem
in that environment, you can solve it by using another
C compiler.

You're missing the fact that I thought it was a good idea to
use the OS's published interface, at least for Win32 and
OS/2. I didn't have this problem in DOS because the only
interface they provided was assembler interrupts.

Methinks, I'm not the one missing something here... ;-)