Are system calls sometimes costlier than library calls?

[Richard Tobin]

Hence in above example its clear that printf() is cheaper that write()
contrary to what we might think. Why is that? Who is telling our
program to combine to calls to printf() in a single wrtie() ? Is it
the libc?

One of the main purposes of the standard i/o library is to provide
buffering in order to reduce the overhead of system calls. You
can control buffering with the setbuf() and setvbuf() functions.

-- Richard

 

[omkar pangarkar]

Hi all,
I have two simple hello world programs one using printf()[a
library func] and other using write() [syscall]

--prog 1--
#include<stdio.h>
#include<stdlib.h>

int main() {
printf("Hello"); /* up to here write() isn't called, if u
* give \n here then two write()s will
* be called (?)*/
printf("World\n");
return 0;
}

--end--

--prog2--
#include<unistd.h>

int main() {
write(1, "Hello", 5);
write(1, "world\n", 6);
return 0;
}

--end--

If u see the output of strace for prog1

--truncated--
....
munmap(0xb7fa0000, 115973) = 0
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 2), ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb7fbc000
write(1, "HelloWorld\n", 11) = 11
exit_group(0) = ?

-- end --

Its only making ONE write() system call

for prog2 if u see strace
--truncated--
....
mprotect(0xb7f50000, 4096, PROT_READ) = 0
munmap(0xb7f56000, 115973) = 0
write(1, "Hello", 5) = 5
write(1, "world\n", 6) = 6
exit_group(0) = ?

--end--

Its making TWO syscalls.

Hence in above example its clear that printf() is cheaper that write()
contrary to what we might think. Why is that? Who is telling our
program to combine to calls to printf() in a single wrtie() ? Is it
the libc?

Its certainly not compiler optimization as I compiled it with -O0
option.

Thanks
Omkar.

 

[Walter Roberson]

Hence in above example its clear that printf() is cheaper that write()
contrary to what we might think. Why is that? Who is telling our
program to combine to calls to printf() in a single wrtie() ? Is it
the libc?

I suggest you read about setbuf()

 

[santosh]

Hi all,
I have two simple hello world programs one using printf()[a
library func] and other using write() [syscall]

--prog 1--
#include<stdio.h>
#include<stdlib.h>

int main() {
printf("Hello"); /* up to here write() isn't called, if u
* give \n here then two write()s will
* be called (?)*/
printf("World\n");
return 0;
}

--end--

--prog2--
#include<unistd.h>

int main() {
write(1, "Hello", 5);
write(1, "world\n", 6);
return 0;
}

--end--

If u see the output of strace for prog1

--truncated--
...
munmap(0xb7fa0000, 115973) = 0
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 2), ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb7fbc000
write(1, "HelloWorld\n", 11) = 11
exit_group(0) = ?

-- end --

Its only making ONE write() system call

for prog2 if u see strace
--truncated--
...
mprotect(0xb7f50000, 4096, PROT_READ) = 0
munmap(0xb7f56000, 115973) = 0
write(1, "Hello", 5) = 5
write(1, "world\n", 6) = 6
exit_group(0) = ?

--end--

Its making TWO syscalls.

Hence in above example its clear that printf() is cheaper that write()
contrary to what we might think.

Why are you worried about microoptimisation like these?

Why is that? Who is telling our
program to combine to calls to printf() in a single wrtie() ? Is it
the libc?

No. It's the compiler's optimiser.

Its certainly not compiler optimization as I compiled it with -O0
option.

BTW, write per se isn't likely to be a system call, but a wrapper to
one.

 

[Rob Kendrick]

I have two simple hello world programs one using printf()[a
library func] and other using write() [syscall]

Hence in above example its clear that printf() is cheaper that write()
contrary to what we might think.

You're thinking too simply. printf() may well be doing a lot of
processing before it calls write() (in fact, it is) which may be more
expensive than an additional system call. You can't know which is faster
by simply monitoring what is called.

Why is that? Who is telling our program
to combine to calls to printf() in a single wrtie() ? Is it the libc?

Yes. Standard IO under almost all self-respecting C libraries is
buffered.

Its certainly not compiler optimization as I compiled it with -O0
option.

Indeed not. I'd be upset if a compiler merged function calls like this.

Why are you playing about with such a trivial low-gain optimisation?
It's doubly pointless, as you're not even measuring the costs properly.
Perhaps similar reasons to your faulty optimisation of the word "you" is
at work?

B.

 

[Eric Sosman]

santosh wrote On 11/06/07 15:32,:

[... two printf() calls generate just one write() ...]

Why is that? Who is telling our
program to combine to calls to printf() in a single wrtie() ? Is it
the libc?

No. It's the compiler's optimiser.

What compiler does this? Libraries have been
buffering output since time immemorial, but I've
never seen a compiler merge two successive printf()
calls into one call. (I've seen a compiler replace
printf("Hello!\n") with puts("Hello!"), but never
the transformation you describe.)

 

[omkar pangarkar]

I have two simple hello world programs one using printf()[a
library func] and other using write() [syscall]

Hence in above example its clear that printf() is cheaper that write()
contrary to what we might think.

You're thinking too simply. printf() may well be doing a lot of
processing before it calls write() (in fact, it is) which may be more
expensive than an additional system call. You can't know which is faster
by simply monitoring what is called.

Why is that? Who is telling our program
to combine to calls to printf() in a single wrtie() ? Is it the libc?

Yes. Standard IO under almost all self-respecting C libraries is
buffered.

Its certainly not compiler optimization as I compiled it with -O0
option.

Indeed not. I'd be upset if a compiler merged function calls like this.

Why are you playing about with such a trivial low-gain optimisation?
It's doubly pointless, as you're not even measuring the costs properly.
Perhaps similar reasons to your faulty optimisation of the word "you" is
at work?

B.

This is not an attempt to optimize any code. I just need to know why
such thing is happening and who is behind it?

 

[cr88192]

I have two simple hello world programs one using printf()[a
library func] and other using write() [syscall]
<snip>

Indeed not. I'd be upset if a compiler merged function calls like this.

Why are you playing about with such a trivial low-gain optimisation?
It's doubly pointless, as you're not even measuring the costs properly.
Perhaps similar reasons to your faulty optimisation of the word "you" is
at work?

B.

This is not an attempt to optimize any code. I just need to know why
such thing is happening and who is behind it?

CRTL...

as others have noted, it does buffering, and may choose to write output in
larger chunks than what is initially passed to it.

I suspect, at least in the case of printf, that this buffering is usually
done until either:
the buffer is full;
a newline or similar is encountered.


for file IO, it is less clear if/when exactly contents are written, but we
do know at least that fflush and fclose cause contents to be written.

I am not sure if in general fflush has lower-level effects (for example, if
it also flushes OS-level file caching or commits changes to the underlying
filesystem, or if it only ensures that changes are commited to the OS
proper). this would mostly effect cases like, say, one calls fflush, and
then very shortly following, the kernel crashes or the power fails or
something...

 

[Tzy-Jye Daniel Lin]

What compiler does this? Libraries have been
buffering output since time immemorial, but I've never seen a compiler
merge two successive printf() calls into one call. (I've seen a
compiler replace printf("Hello!\n") with puts("Hello!"), but never the
transformation you describe.)

stdio flushes its output buffer on every '\n'.

#include <stdio.h>
int main() {
char c, s[2] = "";
for (s[0] = 'A'; s[0] <= 'Z'; s[0]++)
puts(c);
for (c = 'a'; c <= 'z'; c++)
putchar(c);
putchar('\n');
return 0;
}

You will see that 26 write("A\n")..write("Z\n") syscalls are made, but
only one write("abcdefghijklmnopqrstuvwxyz\n") syscall is made.

 

[Martien Verbruggen]

stdio flushes its output buffer on every '\n'.

Not necessarily. A stream can be unbuffered, fully buffered, or line
buffered (which is presumably what you think is always the case).
setvbuf() can be used to control this behaviour.

The C standard tells us that stderr is (initially) not fully buffered
(i.e. line buffered or unbuffered), and stdin and stdout are fully
buffered when a stream is determined not to be an interactive device.

\begin{offtopic}
On the OP's system, assuming it is some Unix-like system, stdout
will be line buffered when connected to an interactive terminal, and
fully buffered when connected to a non-interactive terminal.
\end{offtopic}

Martien

 

[Kenny McCormack]

\begin{offtopic}
On the OP's system, assuming it is some Unix-like system, stdout
will be line buffered when connected to an interactive terminal, and
fully buffered when connected to a non-interactive terminal.
\end{offtopic}

\begin{compulsoryCLCnitpickMode (*)}
What's a "non-interactive terminal"?
\end{compulsoryCLCnitpickMode}

(*) Also known as "vying for points".

 

[Kenny McCormack]

Why are you playing about with such a trivial low-gain optimisation?
It's doubly pointless, as you're not even measuring the costs properly.
Perhaps similar reasons ...

You would do well to Google the words "test", "experiment", and "learning".

 

[Richard]

Not necessarily. A stream can be unbuffered, fully buffered, or line
buffered (which is presumably what you think is always the case).
setvbuf() can be used to control this behaviour.

The C standard tells us that stderr is (initially) not fully buffered
(i.e. line buffered or unbuffered), and stdin and stdout are fully
buffered when a stream is determined not to be an interactive device.

\begin{offtopic}
\end{offtopic}

WTF is all this stuff? Postscript?

 

[Richard]

I have two simple hello world programs one using printf()[a
library func] and other using write() [syscall]

Hence in above example its clear that printf() is cheaper that write()
contrary to what we might think.

You're thinking too simply. printf() may well be doing a lot of
processing before it calls write() (in fact, it is) which may be more
expensive than an additional system call. You can't know which is faster
by simply monitoring what is called.

Yes you can. No one cares how long the "printf part is" - clearly the
value of importance is the total time to return.

Yes. Standard IO under almost all self-respecting C libraries is
buffered.


Indeed not. I'd be upset if a compiler merged function calls like this.

Why are you playing about with such a trivial low-gain optimisation?
It's doubly pointless, as you're not even measuring the costs properly.
Perhaps similar reasons to your faulty optimisation of the word "you" is
at work?

I would imagine he is interested. And some people do run on systems
where every clock cycle gained is important. if more people thought like
that each and every time they check something in then maybe Word would
open faster on a dual core duo with 3 gigs of ram than it did 10 years
ago on a p3 with 128 megs.

 

[Chris Torek]

Before I get any further, just to answer the question in the subject
line (i.e., "are system calls sometimes costlier than library
calls"), the short answer is "yes". Of course, we also have to
define "system" vs "library" calls, and the short answer is useless,
because "sometimes" is not "always" and this does not really tell
us anything about any *particular* calls: a call to foo() may be
100 times slower than one to bar(), regardless of whether foo()
and/or bar() are "system calls" (whatever those are!).

On Tue, 06 Nov 2007 12:26:25 -0800, omkar pangarkar wrote:
I have two simple hello world programs one using printf()[a
library func] and other using write() [syscall] ...

then later added this clarification:

Well, if you code one call to printf(), and one to write(), then
*you* are behind it, and it is happening because that is what you
wrote.

If the question is "when and why does printf() choose to call the
underlying OS's `write' system call"...

as others have noted, [the system's stdio library] does buffering,
and may choose to write output in larger chunks than what is
initially passed to it.

I suspect, at least in the case of printf, that this buffering is usually
done until either:
the buffer is full;
a newline or similar is encountered.

The C Standard allows, but does not require, an implementation to
do this sort of buffering. The C Standard *does* require that the
library "transmit" buffered data to the "host environment" under
various conditions, including (but not limited to):

- printing a newline to a line-buffered stream;
- various calls to fflush().

Any given stdio implementation may choose not to do any buffering
at all, which will trivially satisfy these requirements. Most real
implementations at least do *some*, because the "transmission to
the host environment" is done with something that has some extra
overhead (e.g., a "system call").

On host systems that provide some kind of protection -- so that an
"ordinary user program" cannot crash the machine, for instance --
requests that we (library and/or system implementors) tend to label
"system calls" have to do something that we (system implementors)
call "crossing a protection domain", to use the fully generalized
term. More specifically, in a Unix-like OS, a "system call" switches
from "user mode" to "kernel mode", using some sort of CPU-dependent
mechanism. Depending on the hardware (and to some extent, software),
this mode switch has a minimum cost of tens to even thousands of
nanoseconds, which is the equivalent of a few dozen to a few thousand
"ordinary instruction" times. In these specific cases, a "system
call" may well be, on average, about 20 to 200 times slower than
an "ordinary" function (or "library") call. (That is, if the code
for the operation could be moved from the kernel to a user library, a
call to that code would take about 1/20th to 1/200th as much time.
Of course, this tends to depend greatly on the code, and in most
cases there is some good reason that it cannot be moved this way.)

(Other host systems have lighter-weight mechanisms. On such systems,
a "system call" may take about the same amount of time as a "library
call". In this case, there is little reason to bother with fancy
buffering schemes. Some such systems do it anyway, either because
the code was imported from some other system in which it *was* a good
idea to buffer, or just to mimic the behavior of those systems.)

for file IO, it is less clear if/when exactly contents are written,
but we do know at least that fflush and fclose cause contents to be
written.

(Or rather, "transmitted to the host environment", whatever that
ultimately means. Note that on some systems, a block of data sent
to the host, but not terminated with a newline, is simply buffered
inside the host, i.e., not yet presented to a human who may be
typing at a terminal. Fortunately, such systems are quite rare.)

More specifically, any stdio stream that is not connected to an
"interactive device" need not be line-buffered. Note that the
implementation -- whatever C system you are using -- determines
which stdio streams, if any, are connected to an "interactive
device". If the C system decrees that *all* output is to "interactive
devices", then *all* streams will, by default, be line-buffered or
unbuffered; if it decrees that *no* output is ever interactive, it
is possible that all output streams will be fully-buffered by
default.

In any case, you can call setbuf() or setvbuf() to control buffering.
Simply make an appropriate call before doing any output, and you
can arrange for a given output stream to be line-buffered or
un-buffered (or fully-buffered) at your choosing.

I am not sure if in general fflush has lower-level effects (for
example, if it also flushes OS-level file caching or commits changes
to the underlying filesystem, or if it only ensures that changes
are commited to the OS proper). this would mostly effect cases
like, say, one calls fflush, and then very shortly following, the
kernel crashes or the power fails or something...

This depends not only on the C implementation but also on the
underlying OS. Some OSes implement "reliable" file systems that
never lose data, some implement "fast" file systems that can lose
data in a crash, some provide hybrids ("semi-fast" file systems
with optional partial or complete journaling, for instance), and
some provide everything.

 

[Rob Kendrick]

Yes you can. No one cares how long the "printf part is" - clearly the
value of importance is the total time to return.

Knowing the total time to return is clearly different from knowing what
is called.

B.

 

[Charlie Gordon]

WTF is all this stuff? Postscript?

looks like Latex to me.

 

[Charlie Gordon]

santosh wrote On 11/06/07 15:32,:

[... two printf() calls generate just one write() ...]

Why is that? Who is telling our
program to combine to calls to printf() in a single wrtie() ? Is it
the libc?

No. It's the compiler's optimiser.

What compiler does this? Libraries have been
buffering output since time immemorial, but I've
never seen a compiler merge two successive printf()
calls into one call. (I've seen a compiler replace
printf("Hello!\n") with puts("Hello!"), but never
the transformation you describe.)

Which compiler was that?
Did it expand less trivial printf formats into a series of explicit calls to
conversion functions?

 

[Ben Pfaff]

Which compiler was that?

I think that GCC will do that, in some versions and at some
optimization levels.

Did it expand less trivial printf formats into a series of explicit calls to
conversion functions?

Haven't noticed it do that.

 

[Ben Bacarisse]

I think that GCC will do that, in some versions and at some
optimization levels.

Some gcc's (the one have to hand) can't be stopped from doing it -- at
least it seems to do it even with no optimisation turn on.