writing to pipe

[Brian Darcey]

Assuming a pipe has a buffer of 4096 bytes,
what happens when using the write system call
on the pipe with more bytes? Does write simply
return the value 4096, being the maximum
number of bytes writable to the pipe?

Thanks

 

[Richard Tobin]

Assuming a pipe has a buffer of 4096 bytes,
what happens when using the write system call
on the pipe with more bytes? Does write simply
return the value 4096, being the maximum
number of bytes writable to the pipe?

You're probably interested in Posix pipes, so look at the Posix
specification. If I recall correctly, it will block (eventually
returning the full count) unless the file descriptor is in
non-blocking mode.

In any case, you should write your code on the assumption that
it may return a short count - how often can you be sure that
you're writing to a pipe rather than something else?

-- Richard

 

[Keith Thompson]

Assuming a pipe has a buffer of 4096 bytes,
what happens when using the write system call
on the pipe with more bytes? Does write simply
return the value 4096, being the maximum
number of bytes writable to the pipe?

Pipes are not a feature of standard C. Try asking in
comp.unix.programmer. (Or read your system's documentation, or try it
and see what happens.)

 

[vippstar]

Assuming a pipe has a buffer of 4096 bytes,
what happens when using the write system call
on the pipe with more bytes? Does write simply
return the value 4096, being the maximum
number of bytes writable to the pipe?

That question is off-topic here. You could try comp.unix.programmer.

<OT>
You can determine the buffer size with fpathconf and _PC_PIPE_BUF
(both in <unistd.h>) or you can examine the _POSIX_PIPE_BUF in
<limits.h>.
POSIX requires this to be at least 512. Writing() more than the pipe
buffer size is not an atomic operation.
</OT>

 

[Walter Roberson]

Assuming a pipe has a buffer of 4096 bytes,
what happens when using the write system call
on the pipe with more bytes? Does write simply
return the value 4096, being the maximum
number of bytes writable to the pipe?

Mu.

C has no idea what pipes are, or write system calls. The
result will depend upon the details of whatever system extension(s)
you are using that offer those facilities.

[OT]

The answer in POSIX.1-1990 is complicated. Below, nbytes is the
number of requested bytes to transfer:

(a) implimentation defined if nbytes > SSIZE_MAX,

(b) unless there is an error, all bytes will be written if O_NONBLOCK
is clear and nbytes will be returned
(c1) if O_NONBLOCK is set, unless there is an error,
transfer what it can and return the number of bytes transferred;
if no bytes could be written (e.g., because the pipe was full),
return -1 and set errno to EAGAIN

In cases (b) and (c1) above "Writes of greater than {PIPE_BUF}
bytes may have data interleaved, on arbitrary boundaries, with
writes by other processes, whether ot not the O_NONBLOCK flag
of the file status flags is set."


In the case where the number of bytes to write is less or equal to
PIPE_BUF (your question was strictly for > PIPE_BUF but I'm adding
this for completeness for others reading):
(a) and (b), above, apply;
(c2) if O_NONBLOCK is set, unless there is an error, if there
is sufficient space in the buffer, write all of the bytes and return
nbytes; but if there was not sufficient space in the buffer to
write -all- of the bytes, then do not write -any- of the bytes and
return -1 and set errno to EAGAIN

In these cases, when no more than PIPE_BUF bytes were requested
to transfer, the data will never be interleaved with that of
(simultaneous) writes from other processes. So if you write no
more than PIPE_BUF at a time, the entire chunk will come through
in one contiguous piece, but if you write more than PIPE_BUF
then even if you have O_NONBLOCK set so that all of the bytes
will be written before the write() call returns, what you have
to write could be mushed up arbitrarily with that from other processes,
including within chunks smaller than PIPE_BUF.

 

[Morris Dovey]

C has no idea what pipes are, or write system calls. The
result will depend upon the details of whatever system extension(s)
you are using that offer those facilities.

[OT]

Is there an exercise in K&R2 that illustrates any of the above?

Not that I could spot in a quick skimming. APUE, however, has a
good section on interprocess communication and the discussion of
pipes includes several C examples.

 

[c gordon liddy]

Mu.

A Hofstadter-esque response.

C has no idea what pipes are, or write system calls. The
result will depend upon the details of whatever system extension(s)
you are using that offer those facilities.

[OT]

The answer in POSIX.1-1990 is complicated. Below, nbytes is the
number of requested bytes to transfer:

(a) implimentation defined if nbytes > SSIZE_MAX,

(b) unless there is an error, all bytes will be written if O_NONBLOCK
is clear and nbytes will be returned
(c1) if O_NONBLOCK is set, unless there is an error,
transfer what it can and return the number of bytes transferred;
if no bytes could be written (e.g., because the pipe was full),
return -1 and set errno to EAGAIN

In cases (b) and (c1) above "Writes of greater than {PIPE_BUF}
bytes may have data interleaved, on arbitrary boundaries, with
writes by other processes, whether ot not the O_NONBLOCK flag
of the file status flags is set."


In the case where the number of bytes to write is less or equal to
PIPE_BUF (your question was strictly for > PIPE_BUF but I'm adding
this for completeness for others reading):
(a) and (b), above, apply;
(c2) if O_NONBLOCK is set, unless there is an error, if there
is sufficient space in the buffer, write all of the bytes and return
nbytes; but if there was not sufficient space in the buffer to
write -all- of the bytes, then do not write -any- of the bytes and
return -1 and set errno to EAGAIN

In these cases, when no more than PIPE_BUF bytes were requested
to transfer, the data will never be interleaved with that of
(simultaneous) writes from other processes. So if you write no
more than PIPE_BUF at a time, the entire chunk will come through
in one contiguous piece, but if you write more than PIPE_BUF
then even if you have O_NONBLOCK set so that all of the bytes
will be written before the write() call returns, what you have
to write could be mushed up arbitrarily with that from other processes,
including within chunks smaller than PIPE_BUF.

I'm going through K&R2 again, so that my knowledge of C syntax is more of an
asset than liability, but I never did the chapter on linux. Since the linux
crowd is giving a damn about windows lately, I thought I might do likewise.

Is there an exercise in K&R2 that illustrates any of the above?

 

[Morris Dovey]

Is the treatment of unix in K&R2 still relevant to modern linux?

Yes.

 

[c gordon liddy]

C has no idea what pipes are, or write system calls. The
result will depend upon the details of whatever system extension(s)
you are using that offer those facilities.

[OT]

Is there an exercise in K&R2 that illustrates any of the above?

Not that I could spot in a quick skimming. APUE, however, has a
good section on interprocess communication and the discussion of
pipes includes several C examples.

Thanks for the leafing-through. If it's linux I best stick to the shallows
of K&R.

Is the treatment of unix in K&R2 still relevant to modern linux?

 

[c gordon liddy]

snip

I'm looking at exercise 8-1 and realize that it was exactly this that
was on my mind. It says, "change the cat exercise from chp 7 while
using read, write, open, and closed ...."

Of course, I haven't done the cat exercise, at least not since George
Bush and his cabal downgraded all my knowledge to delusion. But I did
do 7-6, so I won't be far behind. A bit of a story.

I'd been asking how do funnel a file to stdin. The answer is,
apparently with children of the Bourne shell, to invert the arrows
that go to stdout. So I learn this great thing, and later that day my
buddy Seth surprises me with a social call. Half of his visit was
spent on my roof, inspecting my new woodburning-stove installation.
I'm the best in the business.

Otherwise, we were at the computer, and I wanted to show off what I
learned that day, so I showed him how to divert a file on dos to the
executable for the wordtree exercise. He saw what I did and said
write:
cat | executable or something
. Dos has no idea what he's talking about. He realizes it instantly.

How do I write a cat.exe that will read the pipe and funnel a file to
stdin?
--

 

[Morris Dovey]

snip

I'm looking at exercise 8-1 and realize that it was exactly this that
was on my mind. It says, "change the cat exercise from chp 7 while
using read, write, open, and closed ...."

Of course, I haven't done the cat exercise, at least not since George
Bush and his cabal downgraded all my knowledge to delusion. But I did
do 7-6, so I won't be far behind. A bit of a story.

I'd been asking how do funnel a file to stdin. The answer is,
apparently with children of the Bourne shell, to invert the arrows
that go to stdout. So I learn this great thing, and later that day my
buddy Seth surprises me with a social call. Half of his visit was
spent on my roof, inspecting my new woodburning-stove installation.
I'm the best in the business.

Otherwise, we were at the computer, and I wanted to show off what I
learned that day, so I showed him how to divert a file on dos to the
executable for the wordtree exercise. He saw what I did and said
write:
cat | executable or something
. Dos has no idea what he's talking about. He realizes it instantly.

How do I write a cat.exe that will read the pipe and funnel a file to
stdin?

Usage: cat <files>

"cat writes the contents of each given file, or the standard
input if none
are given or when a file named `-' is given, to the standard
output."

So check argc and if it's not more than 1, then just read the
data from stdin. Otherwise get the filenames from argv, and if
one of the filenames is "-", then read that file from stdin. For
all other files, just open the file and read it normally. All
output, of course, gets written to stdout.

Duck soup! If you get stuck, we're here - and if all goes
smoothly, you're invited to pop back here and brag.

 

[c gordon liddy]

c gordon liddy wrote:


Usage: cat <files>

"cat writes the contents of each given file, or the standard
input  if  none
are given or when a file named `-' is given, to the standard
output."

So check argc and if it's not more than 1, then just read the
data from stdin. Otherwise get the filenames from argv, and if
one of the filenames is "-", then read that file from stdin. For
all other files, just open the file and read it normally. All
output, of course, gets written to stdout.

Duck soup! If you get stuck, we're here - and if all goes
smoothly, you're invited to pop back here and brag.

Heathfield's soln uses one fewer non-standard header than the one in
my hard copy:

/*
Andrew Tesker
ucat.c
a version of cat using UNIX system access
*/

#include <stdio.h>
#include <fcntl.h>
#define BUFSIZE 1024


int main(int argc, char *argv[])
{
int fd1;
void filecopy(int f, int t);

if(argc == 1)
filecopy(0, 1);

else {
while(--argc > 0)
if(( fd1 = open(*++argv, O_RDONLY, 0)) == -1) {
printf("unix cat: can't open %s\n", *argv);
return 1;
}
else {
filecopy(fd1, 1);
close(fd1);
}
}

return 0;

}

void filecopy(int from, int to)
{
int n;
char buf[BUFSIZE];

while((n=read(from, buf, BUFSIZE)) > 0 )
write(to, buf, n);
}

// gcc -o cat ucat.c
// end source resume comment

I think that my addition, the goocher, the commented-out command line,
is important. Without a cat.exe in the appropriate path, dos will
complain.

This program behaves as far as I tested it, which was to ask it to
give me a file in the same directory.

Nothing in this program addresses a pipe in the command line.
Furthermore, I couldn't find the earlier cat program in K&R that this
was supposed to amplify.
--

 

[Morris Dovey]

I think that my addition, the goocher, the commented-out command line,
is important. Without a cat.exe in the appropriate path, dos will
complain.

That makes sense.

This program behaves as far as I tested it, which was to ask it to
give me a file in the same directory.

Yuppers. Looks like it should.

Nothing in this program addresses a pipe in the command line.
Furthermore, I couldn't find the earlier cat program in K&R that this
was supposed to amplify.

Try this command line that uses a (shell) pipe:

dir | cat

I wrote a version last night that allows use of "-" to designate
stdin so you can do

dir | cat cat.c -

to allow piped input to be included along with named files.
Source is at

http://www.iedu.com/c/cat.c

 

[c gordon liddy]

That makes sense.


Yuppers. Looks like it should.


Try this command line that uses a (shell) pipe:

   dir | cat

I wrote a version last night that allows use of "-" to designate
stdin so you can do

   dir | cat cat.c -

to allow piped input to be included along with named files.
Source is at

   http://www.iedu.com/c/cat.c

Thanks for your attention, Morris. This is what you have so far:
if (argc < 2) print_file(stdin);
else
{ while (*++argv)
{ if (strcmp(*argv,"-"))
{ if (!(fp= fopen(*argv,"r")))
{ fprintf(stderr,"Error opening %s\n",*argv);
r = EXIT_FAILURE;
continue; /* break to quit on error */
#end snippet
One thing that I'm not getting here is what the usage of cat "should"
be. If you want text42.txt to get sent to wordtree.exe and printed
out, do you write
cat wordtree | text42.txt

In chapter 8.1, it talks of < and > being similar to the pipe, but I'm
not satisfied that I really know what I'm going for. I would think
that we'd have:
if (strcmp(*argv,"|"))
then
send *argv++ to *argv--

My question at this point is what usage precisely am I trying to
imitate with dos?

 

[Morris Dovey]

One thing that I'm not getting here is what the usage of cat "should"
be. If you want text42.txt to get sent to wordtree.exe and printed
out, do you write
cat wordtree | text42.txt

No (but I have an example using cat below). If wordtree expects
input from a file specified in the command line, then:

wordtree text42.txt

Or, if wordtree expects input from stdin:

wordtree < text42.txt

causes wordtree's stdin to be associated with the (automatically
opened) text42.txt file. In other words, so that when wordtree
reads from stdin it gets data from text42.txt

Another way to accomplish the same end might be:

cat text42.txt | wordtree

which causes cat to copy the contents of text42.txt to its
stdout, and which data is, in turn, fed to wordtree's stdin.

Am I making any sense (are you following)?

In chapter 8.1, it talks of < and > being similar to the pipe, but I'm
not satisfied that I really know what I'm going for. I would think
that we'd have:
if (strcmp(*argv,"|"))

You won't see the '|' because the shell doesn't pass it to your
program. In a command like:

program1 | program 2

the pipe token tells the shell to intercept what program1 sends
to stdout for delivery to program2 at program2's stdin.

> file1

tells the shell that data written to stdout should be placed in
the file file1. Similarly

< file2

tells the shell that data read from stdin should come from file2.

then
send *argv++ to *argv--

My question at this point is what usage precisely am I trying to
imitate with dos?

I don't understand the question. Try and make what sense you can
of what I've written and then ask again/more if needed.

 

[c gordon liddy]

I have it backwards here. I talked to my buddy today about what the
pipe does, and he said hooking stdout to stdin.

Another way to accomplish the same end might be:

   cat text42.txt | wordtree

which causes cat to copy the contents of text42.txt to its
stdout, and which data is, in turn, fed to wordtree's stdin.

Am I making any sense (are you following)?

Yeah. Talking through it helps. With the program you posted saved as
cat2.c, I have the following goocher:
// gcc -o cat2 cat2.c
// cat2 text42.txt | wordtree >text43.txt

text42.txt contains:
the quick brown fox
the lazy dog

Text43.txt contains:
Total Words : 6
2 the
1 brown
1 fox
1 lazy
1 dog
1 quick

So I think I'm squared away. Before we drop this theme, and I have no
idea why it would be OT, what other things does cat do in unix that we
haven't coded behavior for?
--

 

[Morris Dovey]

So I think I'm squared away. Before we drop this theme, and I have no
idea why it would be OT, what other things does cat do in unix that we
haven't coded behavior for?

It can do anything you can dream up (and code <g>). These are
the options for the version on my system:

-b, --number-nonblank
Number all nonblank output lines, starting with 1.

-e Equivalent to -vE.

-n, --number
Number all output lines, starting with 1.

-s, --squeeze-blank
Replace multiple adjacent blank lines with a
single blank
line.

-t Equivalent to -vT.

-u Ignored; for Unix compatibility.

-v, --show-nonprinting
Display control characters except for LFD and
TAB using
`^' notation and precede characters that have the
high bit
set with `M-'.

-A, --show-all
Equivalent to -vET.

-E, --show-ends
Display a `$' after the end of each line.

-T, --show-tabs
Display TAB characters as `^I'.

--help Print a usage message and exit with a status code
indicat-
ing success.

--version
Print version information on standard output then
exit.

 

[c gordon liddy]

It can do anything you can dream up (and code  <g>). These are
the options for the version on my system:

       -b, --number-nonblank
              Number all nonblank output lines, starting with 1.

       -e     Equivalent to -vE.

       -n, --number
              Number all output lines, starting with 1.

       -s, --squeeze-blank
              Replace multiple adjacent blank lines with a
single  blank
              line.

       -t     Equivalent to -vT.

       -u     Ignored; for Unix compatibility.

       -v, --show-nonprinting
              Display  control  characters  except for LFD and
TAB using
              `^' notation and precede characters that have the
high bit
              set with `M-'.

I think I'll set about coding the behavior for the -v switch. I've
had trouble displaying link information because some control character
in the output kills my spacing.

I think we had a good result for one thread. Thanks for your
thoughtful comment. I think I'm overdue to read all of chp 8.

End OT.
--

 

[Morris Dovey]

I think I'll set about coding the behavior for the -v switch. I've
had trouble displaying link information because some control character
in the output kills my spacing.

Sounds good. I wrote a version in which I hijacked the -t option
for defining tab stops (I used -t3 to replace tabs with spaces
for the code you saw) and I suspect that you'll find more handy
features to add.

I think we had a good result for one thread. Thanks for your
thoughtful comment. I think I'm overdue to read all of chp 8.

You're welcome - it was satisfying to see you come up to speed
so quickly.

 

[Morris Dovey]

Can you elaborate? For what I looked at -t option, it didn't seem so. But
then ....

The standard -t option is supposed to be equivalent to the option
pair -vT (or -v -T), where -v causes control characters to be
displayed as in the version you're working on, plus -T which
causes tabs to be displayed as ^I.

I wanted cat to replace tabs with spaces so that, among other
things, I could format C source for posting here. I started out
by writing a utility program that processed a single file. Then I
modified the utility to process a list of files. Later I added a
feature to read from stdin if the utility was invoked without any
files specified on the command line. The final step was to
recognize "-" as a command line argument to trigger reading from
stdin when the command line included file names. The only option
designation that seemed appropriate to me was -t followed by a
numerical column width.

I've thought about adding the -n (number lines) and -s (squeeze
multiple blank lines to a single blank line), but haven't ever
needed 'em. I've also thought about adding -d and -u options to
specify DOS <CR><LF> or unix <LF> line endings, but already have
a separate filter for that and have been too lazy to incorporate
it into this program. Maybe someday when/if I have more time...