WHY doesn't C know anything about directories?

[Mark McIntyre]

On Tue, 5 Jun 2007 17:50:40 +0000 (UTC), in comp.lang.c ,

So your case either proves that the standard C library should not have any
file-related functions, or that there would be no harm in adding
directory-related functions since they would have exactly the same status on
your platform as the file-related functions. Which is it?

Neither. IMHO it proves only that the ISO committee felt that files
were commonplace enough, and yet sufficiently in requirement of
standardisation, that adding support via streams was useful. Whereas
directories were neither sufficiently supported, not sufficiently in
need of a common API.

So you're lobbying to have stdio removed from standard C too, because what's
not supported by your platform has no place in the core?

Presumably you have a reason for making that statement up, but its not
clear what it is.
--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan

 

[Tor Rustad]

David Mathog said:


How are you going to deal *portably* with directories on a system that
doesn't have them? Examples include VM/CMS and OS390, both of which are
currently used around the world in the production environments of many
large corporations.

To that list, I can add *HP NonStopKernel*, I guess the installation at
AOL uses the Unix like interface, but many still run under Guardian,
which means no directories.

NSK is used all over world, for critical systems:

http://h20223.www2.hp.com/NonStopComputing/cache/76385-0-0-0-121.html

Guess what, NSK has a C compiler.

 

[Keith Thompson]

The same way you deal with fopen() on a system that doesn't have any files.
By allowing the standardized opendir() to be an always-return-NULL function
on a system that doesn't have anything matching the directory abstraction.

Systems that don't have files are typically embedded systems, and
typically have freestanding C implementations (as opposed to hosted
implementations). A conforming freestanding implementation needn't
support most of the standard headers at all. In other words, the
typical way to handle fopen() for systems that don't have files is not
to provide it at all, not to have it always return NULL.

It turns out to be a lot easier to define a portable file interface
than to define a portable directory interface. For example, you'd
need to define a way to construct file names from directory names. On
Unix-like systems, you can join the directory path elements together
with '/' characters. On VMS the syntax is much more complex
("SYS$FOO::[DIR1.DIR2.DIR3]FILE.TXT", or something like that). And,
as RH pointed out, some mainframe systems support files, but don't
support directories in any meaningful sense.

 

[user923005]

There have been a series of questions about directory operations, all of
which have been answered with "there is no portable way to do this".

This raises the perfectly reasonable question, why, in this day and age,
does the C standard have no abstract and portable method for
dealing with directories? It doesn't seem like a particularly
difficult problem. For instance, this

int show_current_directory(struct DIRSTRUCT *current_directory);

could return status values like these:

VALID_DIRECTORY
success, and the structure is filled in
PLATFORM_HAS_NO_DIRECTORIES
self evident, what you'd see on an embedded controller, for
instance.
CURRENT_DIRECTORY_DOES_NOT_EXIST
like after "cd /newdirectory", when
the directory has not been created yet. Structure
is filled in. Some bit in that structure indicates
that the directory doesn't actually exist.
ERROR
self evident

The organization of DIRSTRUCT could be platform dependent, so
long as key operations (navigation, create directory, delete
directory, compare directory structures [am I in this directory?]) are
fully supported by functions operating on this type.

So, what's the real scoop. Why doesn't the standard support
portable directory operations????

This thingy has a directory abstraction, and is useful for C projects:
http://legacy.imatix.com/html/sfl/

For C++, I prefer this:
http://www.cs.wustl.edu/~schmidt/ACE.html

When you code to one of those standard APIs, your code does not have
to change from system to system.

So why not just fold stuff like the above into the C and C++
standards?

While we could ignore directory services for things like toaster ICs
and other embedded systems, there is still a problem.
The SFL works for POSIX and Windows type systems. No soap on IBM
Mainframes running MVS or VSE and many other systems.
Same for ACE. So it only works on a subset of hosted systems. The
whole point of a nationwide or worldwide standard is that it gives you
a guarantee of how things are supposed to work. But since we can't
make it work that way, we would have to invent a new fork in the
standard:
Hosted, hosted pathless, non-hosted. A bad precedent I think. So why
not just use an existing toolkit that will do what you want to do in
the environment you need to do it in? (After all, you don't usually
need to navigate folders on z/OS except on hosted subsystem operating
systems).

 

[SM Ryan]

# So, what's the real scoop. Why doesn't the standard support
# portable directory operations????

Inertia.

 

[Dik T. Winter]

> If a program could be guaranteed that *if* the
> implementation provided directory services, they would be provided in
> a standard way, a number of new functional possibilities could be
> available to implementors.

But it remains highly non-portable. Even from stdio, fopen is not
really portable. The only really portable parts of stdion are concerned
with stdin, stdout and stderr. Consider:
fopen("c:\\file", "w");
which works perfectly on the system I am now using, but probably will
not do what the writer of the code intended (it will create the file
named "c:\file" in the current directory). And will work only on some
other system that I am using when there is a directory named "c" in the
current directory.

 

[Keith Thompson]

But it remains highly non-portable. Even from stdio, fopen is not
really portable. The only really portable parts of stdion are concerned
with stdin, stdout and stderr. Consider:
fopen("c:\\file", "w");
which works perfectly on the system I am now using, but probably will
not do what the writer of the code intended (it will create the file
named "c:\file" in the current directory). And will work only on some
other system that I am using when there is a directory named "c" in the
current directory.

Sure, but hard-wiring a file name into a program is bound to be
non-portable.

As far as stdio is concerned, a file name is an arbitrary string to be
passed to fopen(). Programs usually obtain file names from
command-line arguments or by reading other files.

 

[Christopher Benson-Manica]

Sure, but hard-wiring a file name into a program is bound to be
non-portable.

I think it's worth making the distinction between absolute and
relative paths; the latter is more likely to be generally useful than
the former. The primary difficulty with supporting relative paths
would seem to be the identity of the path delimiter.

 

[Keith Thompson]

I think it's worth making the distinction between absolute and
relative paths; the latter is more likely to be generally useful than
the former. The primary difficulty with supporting relative paths
would seem to be the identity of the path delimiter.

And the difficulty with relative paths is that the syntax of a file
name varies from one system to another. Some systems may place an
upper bound on the length of a file name, or disallow certain
characters, or ignore distinctions between upper and lower case, or
require exactly one '.' character, and so forth.

The C standard's requirements for header names (C99 6.10.2p5):

The implementation shall provide unique mappings for sequences
consisting of one or more letters or digits (as defined in 5.2.1)
followed by a period (.) and a single letter. The first character
shall be a letter. The implementation may ignore the distinctions
of alphabetical case and restrict the mapping to eight significant
characters before the period.

are intended to be achievable on all systems -- but even then the
specified strings need only be mapped to actual file names (they
needn't actually be file names).

 

[Christopher Benson-Manica]

And the difficulty with relative paths is that the syntax of a file
name varies from one system to another.

I do suppose that's rather a thornier issue for an implementation to
resolve.

The C standard's requirements for header names (C99 6.10.2p5):
(snip)

In an only tenuously related question, does this paragraph mean that
header includes such as <sys/unistd.h> are disallowed, or merely that
the implementation need not provide a "unique mapping" to the header
so identified?

 

[David Mathog]

How would you handle the fact that different systems represent
directory trees differently in filenames? One system may have
something like "/hd1/appl/foobar/data.txt" and another one may
have "disk$1:[appl.foobar]data.txt", and perhaps yet another
may have "data.txt:1".

That was the whole point of having an abstract platform specific
struct contain the directory specification and defining
functions to operate on this structure.

The primary purpose of directories, and also some of the file
organization methods on systems without directories, is just that: file
organization. Directories allow a person (and program) to place and
later find files. What I was thinking of was something more or less
like this pseudocode:

struct DIRSTRUCT *adirstruct=NULL;
char **list;
char buffer[SIZEBUFFER];
int status;

/* Handle platforms without directory support */
status = directory_support();
if(!status)exit(EXIT_FAILURE);

adirstruct = directory_create_dirstruct();
/*fill it with current directory info*/
status=directory_set_current(&adirstruct);
/*list upward path(s), number found returned in status */
status=directory_navigable(adirstruct,&list,UP);
if(status==1){ /* unix, vms, and other single up organizations */
status=directory_navigate(&adirstruct,0,UP);
}
else if(status>0)
/* insert code to pick one super directory from list as index J */
status=directory_navigate(&adirstruct,J,UP);
}
else {
/* handle no such concept as UP on platform or other error */
}
/*list downward path(s). [not shown here, ACROSS] */
status=directory_navigable(adirstruct,&list,DOWN);
/* insert code to pick one subdirectory from list as index J */
status=directory_navigate(&adirstruct,J,DOWN);
/* chdir() equivalent */
status=directory_change(adirstruct);
status=directory_make_fullpath(adirstruct,buffer,SIZEBUFFER,filename);
/* or after the directory_change, this equivalent
status=directory_make_fullpath(NULL,buffer,SIZEBUFFER,filename);
*/
/* check the status (maybe buffer was too small) NOT SHOWN */
status=fopen(buffer,"w");
/* ... other code, list files in directory */
status=directory_listfiles(adirstruct,&list);

/* create a new directory specification, then create it */
status = directory_new(&adirstruct,DOWN,"newsubdir");
status = directory_create(adirstruct);

And so forth.

The point being that the string syntax used for a directory
specification is never explicitly accessed within the code.
When we need to generate one for use in fopen (for instance),
we use the directory_makepath function to construct something
platform specific, without the programmer seeing, knowing, or
caring about the details.

One could argue that a similar set of functions should also exist for
file name manipulation, so that filename syntax variations could
be similarly abstracted.

Regards,

David Mathog

 

[user923005]

Directory structures are antiquated loads of crap.

We should put everything into a database (including making the file
system be a database).

For file systems, a network model or hierarchy is probably better than
a relational model.

At any rate, the C language itself is the wrong place to reform
things.

IMO-YMMV.

 

[Mark McIntyre]

Directory structures are antiquated loads of crap.

We should put everything into a database (including making the file
system be a database).

You've been reading
http://en.wikipedia.org/wiki/WinFS
.....

--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan

 

[user923005]

You've been readinghttp://en.wikipedia.org/wiki/WinFS
....

The AS/400 has a database for a file system. The idea is very old.

 

[Rob Morris]

There have been a series of questions about directory operations, all of
which have been answered with "there is no portable way to do this".

I suspect the reasons are historical, due to the 'standard I/O' library
predating standard C by some years. As far as I can tell (these events
happening some years before my birth) stdio was developed really quite
early on, providing convenience and portability over and above the UNIX
syscalls.

Ah, here we are, from: http://cm.bell-labs.com/cm/cs/who/dmr/chist.html -
"Also during this period, the compiler was retargeted to other nearby
machines, particularly the Honeywell 635 and IBM 360/370; because the
language could not live in isolation, the prototypes for the modern
libraries were developed. In particular, Lesk wrote a `portable I/O
package' [Lesk 72] that was later reworked to become the C `standard
I/O' routines."

So, further to other answers here, the historical reasons might
specifically be due to the capabilities of the first few systems C was
ported to.

 

[Richard Bos]

*Brrrr*

You've been reading
http://en.wikipedia.org/wiki/WinFS

*Brrrrrrrrrrrrr*

Sends my back a-creeping, that does.

But at least it does illustrate why putting directory support into C is
not as simple as it sounds at first.

Richard

 

[Mark McIntyre]

The AS/400 has a database for a file system. The idea is very old.

I didn't say that the authors of WinFS had an /original/ thought.
(obviously....)
--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan

 

[David Mathog]

Directory structures are antiquated loads of crap.

We should put everything into a database (including making the file
system be a database).

A. Every computer I work with (locally numbering into the hundreds)
has a directory structure. It would be nice if the language supported
these machines.

B. If there is no way to navigate the database (for instance, no way
to recover a list of the keys it accepts) then

directory_support()

would return false, as it would on an embedded controller.

C. How many keys do these proposed database filing systems require?
The proposed navigation calls already support UP, ACROSS, and DOWN
navigation directions. That should be sufficient to navigate within
a database so long as a list of valid keys can be retrieved and
the number of keys needed to index a file location is finite.

Example, if a group of files is indexed by:

PRIMARY=R_AND_D (division)
SECONDARY=OPTOELECTRONICS (section)
TERTIARY=FLEXIBLE_DISPLAYS (subsection)
QUATERNARY=ORGANIC_LED (sub-subsection)

the navigation functions should work fine, so long as one can
first retrieve all the primary keys, and then for one primary key,
all secondaries, for that primary and secondary all tertiary, and for
that primary, secondary, and tertiary all quaternary.

If one CANNOT retrieve the database index keys how the heck would
one ever locate a file by any sort of search criteria?

Regards,

David Mathog

 

[Dave Vandervies]

A. Every computer I work with (locally numbering into the hundreds)
has a directory structure.

And every computer in the world is a VAX^WSun^WWintel^Wx64.

Therefore, if a VAX^WSun^WWintel^Wx64 can do it, it should be in every
programming langauge in the world.


dave

 

[Bart van Ingen Schenau]

In an only tenuously related question, does this paragraph mean that
header includes such as <sys/unistd.h> are disallowed, or merely that
the implementation need not provide a "unique mapping" to the header
so identified?

The paragraph does not state that characters other than letters and
digits are disallowed.
Therefor, an implementation does not have to diagnose a #include
directive of
#include <sys/unistd.h>
but it also does not have to do anything meaningful with it.
Technically, such a directive results in UB, but most implementations
will try to do something sensible (like asking the OS if something by
such a name can be located).

Bart v Ingen Schenau