The threading specs in the standard: a new catastrophe

[Wojtek Lerch]

Is really an appropriate analogy? Using fopen or open is an
implementation detail whereas the choice of thread implementation is a
pretty fundamental design decision.

Really? I have written a lot of code using vanilla pthread_create() etc
without setting any attributes to override any defaults of my threads,
mutexes, or condvars; converting that code to the new C API would be
purely a matter of mechanically renaming things. On some other
occassions, I did use some of the POSIX features that are absent int the
C1X draft, and probably in many of those cases converting my code to C1X
would require some non-trivial redesigning. But I can say all the above
about the C99 and POSIX file APIs -- choosing an API that supports
directories, allows accessing unlinked files, and lets you control
permissions can for sure impace the design compared to having to
restrict your code to using the simplistic C99 file API.

I agree there are similarities. Both filesystem I/O and threading do
require some degree of awareness of the operating environment, but I'd
argue that dependency is stronger with threading.

Could it be that the source of that feeling is the fact that the C1X
threading API is new and unfamiliar? If it had been around for a while,
we would be used to the distinction between the basics covered by the C
standard and the fancy stuff that requires you to give up some
portability, just like we are in the file I/O area. But for now, all
the bits and pieces of the pthreads API feel equally "standard", and the
ones absent from C1X feel "missing" as opposed to "less portable".

From a pthreads user's perspective the proposed API is close enough to
pthreads with default options it be usable. Provided implementations
provide a means of mixing native and C1x thread, there shouldn't be many
barriers to their adoption. For example, on Solaris older native threads
are compatible with pthreads, so the two can be mixed in the same
application.

Yeah, I'm sure POSIX will define a mapping, just like today it does for
mixing the C99 FILE API with the underlying file descriptor-based POSIX API.

 

[Ian Collins]

Really? I have written a lot of code using vanilla pthread_create() etc
without setting any attributes to override any defaults of my threads,
mutexes, or condvars; converting that code to the new C API would be
purely a matter of mechanically renaming things. On some other
occassions, I did use some of the POSIX features that are absent int the
C1X draft, and probably in many of those cases converting my code to C1X
would require some non-trivial redesigning. But I can say all the above
about the C99 and POSIX file APIs -- choosing an API that supports
directories, allows accessing unlinked files, and lets you control
permissions can for sure impace the design compared to having to
restrict your code to using the simplistic C99 file API.

Yes I agree with the above as I wrote later in my post. In most cases
the new API will save passing a NULL for the default action! The
pthreads feature I'll miss most is process shared, which is why I said
it is important for standard and native threads to be compatible under
the hood.

 

[MikeP]

In general, I am not at all sure of WHY the C standard must decide
this. Thread libraries are working OK, and POSIX pthreads is widely
used.
It would be better that the C standard would scrap all this and add
a line:

"The threads specifications follows the POSIX standard."

And then C could bow (curtsy?) and exit stage left, never to return again
to the MS theater. That may even be a better suggestion than dropping ALL
of the standard libraries. Ugly-like-Unix systems then uniquely have (the
also ugly) POSIX and Windows will uniquely have Win32. Then the POSIX
folks could FOCUS on something like a Dot-Not-for-Unix and stop worrying
so much about minutia.

 

[Shao Miller]

The committee has decided, for reasons unknown to me (there is no
explanation why is this necessary anywhere) to go on with the flawed
Dinkum library specs and proclaim that library as the new standard specs
for the whole language, what will surely please Plauger but leaves the
rest of us with a pile of errors.

Uh oh.

I posted in this group on June 8th, 2009, a detailed critique of that
library. Two years later all the original errors are there, together
with new ones. I will repost again then, the most glaring errors with
the hope that the committee corrects them before is too late and stops
this whole idea.

But ought a future C Standard to cover threads _somehow_? As I
mentioned before, the sequencing model is really striking in C1X. It
has "before" and "after" (and un- and indeterminately, etc.)... Thread
discussion would seem to cover "simultaneously"! (Roughly.)

1) The specs use concepts from C++ without furnishing the slightest
explanation of how those concepts should be understood within the C
language.

Examples:
--------
7.25.1.3
TSS_DTOR_ITERATIONS
which expands to an integer constant expression representing the
maximum number of times that destructors will be called when a
thread terminates.

The committee thinks apparently that the term "destructors" is
common in C so that doesn't need any definition or explanation of what
that should be in the context of the C language. The term doesn't even
appear in the index nor is explained anywhere.

Hmmm.


7.25.1.4
cnd_t
which is a complete object type that holds an identifier for a
condition variable;

How can a type "hold an identifier" ??? The committee defines in
6.4.2.1.2 an identifier as a sequence of letters that designates
one or more entities.
Now, we have the opposite: a type holds an identifier and nowhere
is explained what that should mean!

Hmmm again.

2) The proposed functions are absurd.

Take, for instance, nothing less than the thread creation function
thrd_create:

int thrd_create(thrd_t *thr, thrd_start_t func,void *arg);

As you can see, there are *no arguments that can be passed to the
underlying operating system* to further specify the newly created
thread. (The "args" arument is an argument for the thread).

I wouldn't use the proposed C1X threads with such expectations. I'd use
libraries and interfaces which meet my expectations.

The pthreads standard (POSIX) proposes an "attributes" argument. The
Microsoft threads propose a "security attributes" and a set of flags
that further specify the newly created thread. The C API proposes
NOTHING so it is impossible to use.

Why, there's even "thread impersonation" in at least Windows. There're
even "fibers". So I would assume that for C1X threads, no such
considerations are portable and thus don't belong in a C Standard.

Since there is no way to specify the attributes, this function MUST
use the default attributes that are NOWHERE specified. This means
that two compiler systems could very well produce two different sets
of attributes to be passed to the underlying thread creation
functions so that we would have a differnt behavior for the same
source compiled in the same machine. This is completely ridiculous!

Why is it ridiculous? Let's pretend for a moment that C1X (or later)
thread support gives only a minimal and mundane set of capabilities.
Something to the effect of "now my program doesn't need to follow a
strictly sequential order of execution." That seems worth having on any
platform that could support it (which itself is a concern). For
anything else, don't use C threads.

The mutex creation functions do not specify any security/ownership
specifications either.
Anybody can use anything. This will force all
developers that run in a security sensitive environment to reject
this specs. C stayed in the single user era.

I disagree. I think it's worth noting that the term "threads" might not
have a meaning of great value on its own ("_its_", Mr. J. Kuyper, not
"_it's_"!). I don't see a reason why "C threads" must encompass "POSIX
thread" details or "Windows thread" details. I _certainly_ don't see
why a developer would attempt to use "portable C threads" with security
in mind when security itself is platform-dependent!

A microwave doesn't need to worry about carrying access control lists
with Active Directory SIDs! But it might like to distract you with an
LCD dance while it monitors for temperature dangers!

The specifications are completely unusable in many other functions:
For instance:
The thrd_detach function tells the operating system to dispose of
any resources allocated to the thread identified by thr when that
thread terminates.

Such as a thread descriptor whose storage the kernel might enjoy
repossessing, perhaps.

That's new to me. The OS should free all memory allocated by the
thread? Close all files? This is highly system dependent and
many systems could very well not do the cleanup until the process
that created those ressources stops.

I agree that that'd seem strange, if that's what the authors' intentions
are. Not impossible, mind you. I would agree that clarification is
warranted, given your interpretation.

3) New types are defined without any specification.
Example:
-------
The "xtime" data type is defined as
struct xtime { time_t sec; long nanoseconds; };

What is the epoch of the "sec" member? Nobody knows. When you call
a function using an "xtime" as argument with
(struct xtime) {5,0}
means:
A)
I call this function with a timeout of five seconds, i.e. the "sec"
epoch starts running at the instant of function call.

B)
I call it with a timeout of January first 1970 and five seconds,
i.e. since the "sec" member is a time_t and time_t variables are
used to store CALENDAR TIMES (as per difftime specs in 7.26.2.1)
I specify a timeout that has already expired.

Nowhere in the standard is the "xtime" explained or further described.
The committee supposes that it suffices to name a member "sec" for
its meaning to be crystal clear.

I fear that is not the case.

Hmmm yet again.

[...more about "time"...]

5) Error analysis is very difficult.

The proposed functions return the enum value "thrd_error" if
SOMETHING goes wrong. Impossible to know WHAT did go wrong since
nowhere is specified if errno should have a value set or how
to figure out WHAT HAPPENED!!! There is NO WAY that the running
program can communicate the reasons of failure so that we are
left in the dark as to why each error happens. The Microsoft API
specifies GetLastError() and pthreads have errno and other
mechanisms to further speicify what kind of error happened. This
specs ignore all that. It doesn't work, now go figure what happens
during HOURS of debugging!

For simple threads, a simple error might be sufficient. If an
implementation documents that their "C threads" map to some subset of
"OS threads," perhaps they'd also provide additional functions. Else,
don't try to write portable code that cares about platform-specific errors.

I am repeating here in an expanded form the critique that I wrote
in comp.std.c in a message on June 8th 2009. There was no answer then,
and there has been no change since. ALl errors I pointed out then are
still in the specs.

D'oh.

I hope this time they will listen or at least justify what they are
doing. Copy/Paste of Plauger's library docs is not enough to specify
a change to the C language, excuse me.

But wouldn't it be nice to have _some_ kind of thread support in C? I
would not appreciate a very sophisticated thread system, and would not
appreciate injecting POSIX threads into the C Standard. Please keep it
Cimple. It'd be nice to write a simple, portable program with multiple
threads of execution.

 

[Ian Collins]

The committee has decided, for reasons unknown to me (there is no
explanation why is this necessary anywhere) to go on with the flawed
Dinkum library specs and proclaim that library as the new standard specs
for the whole language, what will surely please Plauger but leaves the
rest of us with a pile of errors.

1) The specs use concepts from C++ without furnishing the slightest
explanation of how those concepts should be understood within the C
language.

Examples:
--------
7.25.1.3
TSS_DTOR_ITERATIONS
which expands to an integer constant expression representing the
maximum number of times that destructors will be called when a
thread terminates.

This term is quite clear in the latest draft, see 7.26.1.4.

7.25.1.4
cnd_t
which is a complete object type that holds an identifier for a
condition variable;

How can a type "hold an identifier" ??? The committee defines in
6.4.2.1.2 an identifier as a sequence of letters that designates
one or more entities.
Now, we have the opposite: a type holds an identifier and nowhere
is explained what that should mean!

The type is an implementation detail, just like pthread_t is in POSIX.

2) The proposed functions are absurd.

Take, for instance, nothing less than the thread creation function
thrd_create:

int thrd_create(thrd_t *thr, thrd_start_t func,void *arg);

As you can see, there are *no arguments that can be passed to the
underlying operating system* to further specify the newly created
thread. (The "args" arument is an argument for the thread).

The pthreads standard (POSIX) proposes an "attributes" argument. The
Microsoft threads propose a "security attributes" and a set of flags
that further specify the newly created thread. The C API proposes
NOTHING so it is impossible to use.

In many (most) cases, the default values get passed in pthreads at least.

Since there is no way to specify the attributes, this function MUST
use the default attributes that are NOWHERE specified. This means
that two compiler systems could very well produce two different sets
of attributes to be passed to the underlying thread creation
functions so that we would have a differnt behavior for the same
source compiled in the same machine. This is completely ridiculous!

Agreed, there should be something there, even if it says they are
implementation defined!

The mutex creation functions do not specify any security/ownership
specifications either.

Neither does POSIX.

The specifications are completely unusable in many other functions:
For instance:
The thrd_detach function tells the operating system to dispose of
any resources allocated to the thread identified by thr when that
thread terminates.

That's new to me. The OS should free all memory allocated by the
thread? Close all files? This is highly system dependent and
many systems could very well not do the cleanup until the process
that created those ressources stops.

Not *by* the thread *to* the thread.

3) New types are defined without any specification.
Example:

Fixed (gone) in current draft.

5) Error analysis is very difficult.

The proposed functions return the enum value "thrd_error" if
SOMETHING goes wrong. Impossible to know WHAT did go wrong since
nowhere is specified if errno should have a value set or how
to figure out WHAT HAPPENED!!! There is NO WAY that the running
program can communicate the reasons of failure so that we are
left in the dark as to why each error happens. The Microsoft API
specifies GetLastError() and pthreads have errno and other
mechanisms to further speicify what kind of error happened. This
specs ignore all that. It doesn't work, now go figure what happens
during HOURS of debugging!

At least in the current draft, the returns are no more or less
informative than pthread_create (which does not set errno).

I am repeating here in an expanded form the critique that I wrote
in comp.std.c in a message on June 8th 2009. There was no answer then,
and there has been no change since.

But there have, you quoted form an out of date draft!

 

[Juha Niskanen]

["Followup-To:" header set to comp.std.c.]

The committee has decided, for reasons unknown to me (there is no
explanation why is this necessary anywhere) to go on with the flawed
Dinkum library specs and proclaim that library as the new standard specs
for the whole language, what will surely please Plauger but leaves the
rest of us with a pile of errors.

Let's continue adding to the pile. From N1570:

1)
7.26.1.5: "The enumeration constants are
mtx_plain
which is passed to mtx_init to create a mutex object that supports neither
timeout nor test and return;"

What is "test and return" and where is it specified? Do mtx_recursive or
mtx_timed or both support "test and return functions"?

2)
7.26.4.1.2: "No threads can be blocked waiting for the mutex pointed to by mtx."
What does this mean? What happens if a thread is waiting to lock a mutex when
it is destroyed? Or is the intent that future mutex operations on destroyed
mutex are guaranteed to return thrd_error? Can destroyed mutex be reinitialized?

3)
7.26.4.2.2: The mtx_init function: "which must have one of the six values:"
The list has four values. What if type argument is not one of the
valid values? Is the behavior undefined or is mtx_init() guaranteed to return
thrd_error?

Is it UB to initialize an already initialized mutex?

Also, I believe this is a conforming implementation of mtx_init():

int
mtx_init(mtx_t *mtx, int type)
{
return thrd_error; // "quality of implementation" issue here
}

4)
7.26.4.3.2: "Prior calls to mtx_unlock on the same mutex shall synchronize
with this operation."

What does this mean exactly? What if mtx_unlock() fails? Shoudn't this read
"...calls to successful mtx_unlock..." What does "prior" mean here, in context
of concurrently executing threads?

5)
7.27.1.4: The sentence "The semantics of the members and their normal ranges are
expressed in the comments." appears as part of normative text for struct tm but
in footnote 317 for struct timespec. Is this intentional or just editing error?

What if I initialize struct timespec like this

struct timespec ts = { .tv_sec = 0, .tv_nsec = 999999999+1 };

and pass &ts to any library function that accepts struct timespec arguments?
Is the behavior undefined or am I guaranteed to get thrd_error return value?

Pthreads is a better specification than this because it is precise about when
to return EINVAL and other error codes. With C1X threads there is no way to
find out what went wrong when library returns an error indication and there
are too many possibilities for undefined behavior due to lack of explicit
specification.

 

[Jens Gustedt]

Am 07.07.2011 23:14, schrieb Ian Collins:

From a pthreads user's perspective the proposed API is close enough to
pthreads with default options it be usable.

I find two important features missing though,
PTHREAD_MUTEX_INITIALIZER and PTHREAD_COND_INITIALIZER. Not having
them will make the initialization of simple critical sections quite
clumbsy.

Or is it tacidly implied in the standard that the default
initialization of static mutexes and conditions will to the right
thing?

Jens

 

[James Kuyper]

On 07/ 7/11 10:33 PM, jacob navia wrote: ....

The type is an implementation detail, just like pthread_t is in POSIX.

Even if it's an implementation detail, it's description should still be
meaningful in the standard; "hold an identifier" doesn't satisfy that
requirement.

 

[Ian Collins]

Even if it's an implementation detail, it's description should still be
meaningful in the standard; "hold an identifier" doesn't satisfy that
requirement.

Well the SUS describes pthread_t as a type "Used to identify a thread"
and pthread_cond_t as a type "Used for condition variables". It qualify
them as types that are not required to be arithmetic types, but that's all.

 

[James Kuyper]

Well the SUS describes pthread_t as a type "Used to identify a thread"
and pthread_cond_t as a type "Used for condition variables". It qualify
them as types that are not required to be arithmetic types, but that's all.

The key point is that "identifier" is a term with a very specific
meaning in the C standard, a meaning that doesn't work in this context.
Replace "that holds an identifier for" with "the value of which
identifies", and I think the problem would be solved.

 

[jacob navia]

Le 08/07/11 14:23, James Kuyper a écrit :

The key point is that "identifier" is a term with a very specific
meaning in the C standard, a meaning that doesn't work in this context.
Replace "that holds an identifier for" with "the value of which
identifies", and I think the problem would be solved.

Definitely. Now I understand what they wanted to say: the "identifiers"
weren't C identifiers but tokens returned by the underlying system
that identify a thread.


That should be explicit as you propose.

 

[lawrence.jones]

"They do not read anything, or care about anyone."

Is that what you want to say?

Maybe you are just wrong, and they do care about what the community
thinks or proposes.

Or maybe you are right, and they live in limbo. Time will tell.

A more accurate description is that they have quite enough to do reading
things that come in through official channels and caring about the
community they're required to care about to go looking for other things
to read or other groups to care about.

 

[lawrence.jones]

"The threads specifications follows the POSIX standard."

Microsoft has implemented it under windows, and some companies
have ported it to windows.

Microsoft has *not*, to my knowledge, implemented pthreads under
Windows. They have included it as part of their "POSIX Subsystem", which
is a POSIX emulation environment much like cygwin. It's great for
porting POSIX code to a Windows machine, but it's completely useless for
native Windows programs.

 

[Shao Miller]

Microsoft has *not*, to my knowledge, implemented pthreads under
Windows. They have included it as part of their "POSIX Subsystem", which
is a POSIX emulation environment much like cygwin. It's great for
porting POSIX code to a Windows machine, but it's completely useless for
native Windows programs.

A complication might be using "Windows" instead of "Windows NT" and
"Windows API". Yes, they have implemented it for "Windows NT" via the
POSIX subsystem. No, they have not implemented it as or via the
"Windows API" (but another project has).

Nit-pick: Interix isn't suppose to be an "emulation" of anything, to my
knowledge; it's a peer to the Windows subsystem. "Much like Cygwin":
Sure. A difference: Cygwin lies on top of the Windows subsystem and
Interix doesn't.

 

[Seebs]

Well the SUS describes pthread_t as a type "Used to identify a thread"
and pthread_cond_t as a type "Used for condition variables". It qualify
them as types that are not required to be arithmetic types, but that's all.

"Hold a thing which identifies" and "hold an identifier" are very different
things when you're talking about the C standard, which has a very precise
definition for "identifier" which trumps the standard English usage.

-s

 

[Todd Carnes]

"They do not read anything, or care about anyone."

Is that what you want to say?

No, more like I think they're probably too busy to worry about reading a
bunch of newsgroups for suggestions.

Most newsgroups are high on noise & low on real content.

Todd

 

[Chris M. Thomasson]

[...]

Assuming you use a CS as a mutex, then the only nontrivial thing to
have is PTHREAD_MUTEX_INITIALIZER. Fortunately, recent versions of
Microsoft windows allow static initialization of these via the
definition:
#define PTHREAD_MUTEX_INITIALIZER {(void*)-1,-1,0,0,0,0}

Can you point me to some documentation please?

Thanks.

[...]

 

[sfuerst]

[...]

Assuming you use a CS as a mutex, then the only nontrivial thing to
have is PTHREAD_MUTEX_INITIALIZER.  Fortunately, recent versions of
Microsoft windows allow static initialization of these via the
definition:
#define PTHREAD_MUTEX_INITIALIZER {(void*)-1,-1,0,0,0,0}

Can you point me to some documentation please?

Thanks.

[...]

The CRITICAL_SECTION structure is defined in Winnt.h

Description of what the fields do is described here:
http://msdn.microsoft.com/en-us/magazine/cc164040.aspx
Basically, the above initializes things in the unlocked state, with an
invalid DebugInfo field. This corresponds to the effects of
InitializeCriticalSection() when it cannot allocate memory.

Older versions of Microsoft Windows raise exceptions instead of using
the "invalid field" trick, so the above will not work for them.

Steven

 

[Ian Collins]

The key point is that "identifier" is a term with a very specific
meaning in the C standard, a meaning that doesn't work in this context.
Replace "that holds an identifier for" with "the value of which
identifies", and I think the problem would be solved.

Ah I see the point now and yes I agree.

 

[Chris M. Thomasson]

[...]

Description of what the fields do is described here:
http://msdn.microsoft.com/en-us/magazine/cc164040.aspx
Basically, the above initializes things in the unlocked state, with an
invalid DebugInfo field. This corresponds to the effects of
InitializeCriticalSection() when it cannot allocate memory.

Older versions of Microsoft Windows raise exceptions instead of using
the "invalid field" trick, so the above will not work for them.

Thank you.