Is this fully portable and/or smart?

[Bill Reid]

This is how I handle a check that the last character of a text
file is a newline:

/* checks if newline is last character of text file */
unsigned check_text_file_newline_termination(FILE *test_file) {
int end_char;

fseek(test_file,-1L,SEEK_END);

end_char=getc(test_file);

rewind(test_file);

if(end_char=='\n') return TRUE;
else return FALSE;
}

The question is: is this actually guaranteed to work properly on
all "conforming" C "implementations"? My reading of the spec says
"no", but of course it works just fine on the several systems I've
used it on...

Also, would this be the fastest way possible to make this check,
as opposed to a perhaps more "conformant" scheme of reading
every character in the file to find the end of the file?

 

[Peter Nilsson]

This is how I handle a check that the last character of a text
file is a newline:

Why bother?

/* checks if newline is last character of text file */
unsigned check_text_file_newline_termination(FILE *test_file) {
int end_char;

fseek(test_file,-1L,SEEK_END);

"For a text stream, either offset shall be zero, or offset shall be a
value returned by an earlier successful call to the ftell function
on a stream associated with the same file and whence shall be
SEEK_SET."

And you should at least check whether the call succeeds. If it
doesn't, what will you return? I suggest you change the return
type to int and return EOF, 0 or some positive number.

end_char=getc(test_file);
rewind(test_file);

if(end_char=='\n') return TRUE;
else return FALSE;
}

The question is: is this actually guaranteed to work properly on
all "conforming" C "implementations"?

No. But even if you read the whole file, rewind can fail.

 

[Keith Thompson]

This is how I handle a check that the last character of a text
file is a newline:

/* checks if newline is last character of text file */
unsigned check_text_file_newline_termination(FILE *test_file) {
int end_char;

fseek(test_file,-1L,SEEK_END);

end_char=getc(test_file);

rewind(test_file);

if(end_char=='\n') return TRUE;
else return FALSE;
}

The usual type for boolean values (unless you have C99's _Bool) is
int, not unsigned. It doesn't matter, but anyone reading your code is
going to waste a little time wondering why you used unsigned rather than
int.

Presumably TRUE and FALSE are defined somewhere -- but you really
don't need them. I'd replace the if/else with:

return end_char == '\n';

You don't check whether fseek() succeeded. Not all files are
seekable. (Try seeking to the end of stdin, when it's reading from
your keyboard.)

rewind(test_file) goes back to the beginning of the file -- which
isn't necessarily where it was before the function was called. If you
want to restore the file's position, use ftell() and fseek().

The question is: is this actually guaranteed to work properly on
all "conforming" C "implementations"? My reading of the spec says
"no", but of course it works just fine on the several systems I've
used it on...

You're right, it's not guaranteed by the standard. For a text file,
fseek() requires either an offset of zero, or an offset returned by an
earlier call to ftell() and a whence value of SEEK_SET. (As you've
seen, it happens to work on your system.)

Also, would this be the fastest way possible to make this check,
as opposed to a perhaps more "conformant" scheme of reading
every character in the file to find the end of the file?

Probably.

 

[Bill Reid]

Bill Reid wrote:

Why bother?

Because sometimes I care...if only a little. And I've heard that
some systems don't "like" text files that aren't terminated by a
newline. So when I get a text file from an "unreliable" source,
I generally "fix" it, by appending the newline if needed...

"For a text stream, either offset shall be zero, or offset shall be a
value returned by an earlier successful call to the ftell function
on a stream associated with the same file and whence shall be
SEEK_SET."

Yeah, that's some good copy'n'pasting there...the documentation
for my "implementation" even reads pretty much the same...and yet,
confoundedly enough, the code works fine, and on other systems
too...

And you should at least check whether the call succeeds. If it
doesn't, what will you return?

Hey, I'll go you one better, my "implementation" may "silently"
fail an fseek()...What WILL you do, WHAT WILL YOU DO?!??!!

I suggest you change the return
type to int and return EOF, 0 or some positive number.

Suggestion "under advisement"...

No.

Great, now I've two contrary opinions...

But even if you read the whole file, rewind can fail.

ANOTHER problem!!! What WILL you do, WHAT WILL
YOU DO??!!?!!

In any event, maybe I shouldn't have used the word "guarantee",
although I actually was looking for a little "spec lawyering" (and
came to the right place!). Maybe as an alternate question, just
how often WILL all these things fail?

 

[Ulrich Eckhardt]

/* checks if newline is last character of text file */
unsigned check_text_file_newline_termination(FILE *test_file) {
int end_char;

fseek(test_file,-1L,SEEK_END);

end_char=getc(test_file);

rewind(test_file);

if(end_char=='\n') return TRUE;
else return FALSE;
}

A few comments:
* Both fseek() and rewind() could fail, but you don't check for errors.
* Just return end_char=='\n' in order to save some code.
* I'd also change the returntype to int, as others mentioned, and think
about how to handle errors.

The question is: is this actually guaranteed to work properly on
all "conforming" C "implementations"?

It will pretty much fail on any implementation when the file is some kind of
pipe, i.e. a non-seekable stream like stdin. The stanza about textfiles
being not relatively seekable was already quoted in this thread, and since
that makes the implementation easier you can be sure that it will be done
on some systems that distinguish textfiles.

My reading of the spec says "no", but of course it works just fine on the
several systems I've used it on...

Well, in that case there is one simple thing you should do: unit testing.
Simply create a testsuite and if you happen to port to a new system or
rather C implementation, use the test to make sure that everything works.
This is not "portable" in the sense of "guaranteed by the standard" but in
practice I find this sufficient. Also document that you are relying on
implementation-specific behaviour, why and how this could be implemented
without.

Also, would this be the fastest way possible to make this check,
as opposed to a perhaps more "conformant" scheme of reading
every character in the file to find the end of the file?

I'd say that it is fast.

Uli

 

[Keith Thompson]

Because sometimes I care...if only a little. And I've heard that
some systems don't "like" text files that aren't terminated by a
newline. So when I get a text file from an "unreliable" source,
I generally "fix" it, by appending the newline if needed...


Yeah, that's some good copy'n'pasting there...the documentation
for my "implementation" even reads pretty much the same...and yet,
confoundedly enough, the code works fine, and on other systems
too...

The quoted text is not in a constraint. In the C standard, the use of
the word "shall" outside a constraint means that, if the requirement
is violated, the behavior is undefined. "Works fine" is one possible
consequence of undefined behavior.

Hey, I'll go you one better, my "implementation" may "silently"
fail an fseek()...What WILL you do, WHAT WILL YOU DO?!??!!

If your implementation's fseek() function can fail without properly
reporting the error, then that's a bug; you should report it to the
vendor.

Suggestion "under advisement"...


Great, now I've two contrary opinions...

What two contrary opinions? I don't recall anyone saying that it's
guaranteed to work. If anyone did, they were mistaken.

ANOTHER problem!!! What WILL you do, WHAT WILL
YOU DO??!!?!!

I don't know. It's your program; what will you do? Error handling
isn't easy.

In any event, maybe I shouldn't have used the word "guarantee",
although I actually was looking for a little "spec lawyering" (and
came to the right place!). Maybe as an alternate question, just
how often WILL all these things fail?

Elsethread, Eric Sosman wrote:
| On some systems that mark line endings with something
| other than a one-byte sentinel, seeking to one byte before
| the end of the file and reading what you find there will be
| misleading at best.

For example, Windows uses a CR LF pair to mark end-of-line. If you
seek to one byte before the end of a properly terminated text file,
and then read a single character, you'll probably get a single LF
character, which will probably be translated to '\n'. Normally the CR
LF pair is what gets translated to '\n'. I'm actually not 100%
certain what will happen; it might depend on the C implementation.

Other systems use things other than character sequences to mark line
endings. VMS, for example, has a rather sophisticated
record-management system. Some mainframes use fixed-width lines
(inherited from punch card images), though I don't know whether this
is still in use. This kind of thing is exactly why the standard is so
vague about the behavior of fseek on text files.

If you don't mind the fact that your code is non-portable, that's
fine. The fseek trick is likely to be a whole heck of a lot faster
than the portable method of reading the entire file and seeing how it
ends.

For that matter, even that slow method isn't guaranteed to work. C99
7.19.2p2:

Whether the last line requires a terminating new-line character is
implementation-defined.

And the standard doesn't say what happens if the implementation
requires a terminating new-line and a particular file doesn't have
one. In my opinion, the behavior is undefined. It might report an
error, or it might quietly add a new-line on input.

 

[Flash Gordon]

Keith Thompson wrote, On 20/05/08 01:54:

rewind(test_file) goes back to the beginning of the file -- which
isn't necessarily where it was before the function was called. If you
want to restore the file's position, use ftell() and fseek().

Better to use fgetpos/fsetpos just in case the position does not fit in
a long (e.g. large files on a system with a 32 bit long, and yes they do
exist before Bill complains about yet another non-existent problem).

 

[Flash Gordon]

Bill Reid wrote, On 20/05/08 02:00:

Hey, I'll go you one better, my "implementation" may "silently"
fail an fseek()...What WILL you do, WHAT WILL YOU DO?!??!!

You asked if it would work on all possible conforming implementations,
so why are you shouting at someone for pointing out places where it
might fail without your code spotting it?

 

[Bill Reid]

The usual type for boolean values (unless you have C99's _Bool) is
int, not unsigned. It doesn't matter, but anyone reading your code is
going to waste a little time wondering why you used unsigned rather than
int.

In general, I'm only interested in boolean values as a return from
a function; this is part of my "scheme" for gracefully exiting from
failed operations which I alluded to in another thread...

Presumably TRUE and FALSE are defined somewhere -- but you really
don't need them. I'd replace the if/else with:

return end_char == '\n';

Yeah, that is so "C", so obscure, so arcane...and a friggin' boolean
value to boot, so I like it...I'll take it under advisement...

You don't check whether fseek() succeeded. Not all files are
seekable. (Try seeking to the end of stdin, when it's reading from
your keyboard.)

Give me a break, I am clearly only interested in ACTUAL text files,
not all the crap that can be called a "file stream"...

rewind(test_file) goes back to the beginning of the file -- which
isn't necessarily where it was before the function was called. If you
want to restore the file's position, use ftell() and fseek().

That's EXACTLY what I want, except in most cases I'm not even
interested in that...as I said, I mostly use this to append a newline
to the end of the file if "needed"...

You're right, it's not guaranteed by the standard. For a text file,
fseek() requires either an offset of zero, or an offset returned by an
earlier call to ftell() and a whence value of SEEK_SET. (As you've
seen, it happens to work on your system.)

Actually, several systems...

Probably.

Great! Nothing like blinding speed at the sake of reliability and
portability for no apparent actual performance benefit...

 

[Bill Reid]

Bill Reid wrote:

A few comments:
* Both fseek() and rewind() could fail, but you don't check for errors.

Yeah, heard THAT before...

* Just return end_char=='\n' in order to save some code.
IBID.

* I'd also change the returntype to int, as others mentioned, and think
about how to handle errors.

IBID redux. Strangely enough, I'm not THAT interested whether
this function actually works, because if the file doesn't have a newline
termination, the rest of my code can handle that. I might wind up
with a superfluous newline, or none at all, it all will wind up in the
same place anyway...

It will pretty much fail on any implementation when the file is some kind of
pipe, i.e. a non-seekable stream like stdin.

But will it always work on ACTUAL text files? You know, on-a-disk
text files?

The stanza about textfiles
being not relatively seekable was already quoted in this thread, and since
that makes the implementation easier you can be sure that it will be done
on some systems that distinguish textfiles.

OK, I guess my confusion is that my "implementation" apparently goes to
the extra "trouble" of making this work while at the same time documenting
it as NOT working (the documentation was apparently "cribbed" from the
"standard" and not from the actual "implementation"). It SHOULD and DOES
work on "systems" that DON'T "distinguish" between text and "binary" files,

at least in my experience.

Well, in that case there is one simple thing you should do: unit testing.
Simply create a testsuite and if you happen to port to a new system or
rather C implementation, use the test to make sure that everything works.

Yes, that would work, and also be very "worky". I do have "conforming"
code that is commented out that could be quickly "uncommented" if need
be...and I actually do have a something resembling a "unit test" for all
my libraries...

This is not "portable" in the sense of "guaranteed by the standard" but in
practice I find this sufficient. Also document that you are relying on
implementation-specific behaviour, why and how this could be implemented
without.

Done and done.

I'd say that it is fast.

OK, that makes me feel better, but since I haven't tested it who knows,
but it probably IS faster...and of course, as a practical matter, the
difference
is probably not even worth worrying about in terms of the actual
application...

 

[Bill Reid]

The quoted text is not in a constraint. In the C standard, the use of
the word "shall" outside a constraint means that, if the requirement
is violated, the behavior is undefined. "Works fine" is one possible
consequence of undefined behavior.

There's a lot of that kind of stuff in the "standard", ain't there?

If your implementation's fseek() function can fail without properly
reporting the error, then that's a bug; you should report it to the
vendor.

It's a problem with the operating system and is documented by
the "vendor". It IS a bother, a real bother, but what are you going
to do? It's the way the stupid system works, or, at least, what the
"vendor" says is the way the stupid system works, maybe just as
an excuse...

What two contrary opinions? I don't recall anyone saying that it's
guaranteed to work. If anyone did, they were mistaken.

I'm not going to narc anybody out, but somebody DID say it was
fine and completely conforming except for the "TRUE" and "FALSE"
defines...

I don't know. It's your program; what will you do? Error handling
isn't easy.

It gets easier if you just ignore it, hoping the error will "go away"...

Elsethread, Eric Sosman wrote:
| On some systems that mark line endings with something
| other than a one-byte sentinel, seeking to one byte before
| the end of the file and reading what you find there will be
| misleading at best.

For example, Windows uses a CR LF pair to mark end-of-line. If you
seek to one byte before the end of a properly terminated text file,
and then read a single character, you'll probably get a single LF
character, which will probably be translated to '\n'.

Unless the "implementation" goes to the "extra trouble" of translating
the CR LF to a single newline for "text files" even when "seeking" from
the end of the file, which apparently mine does!

Normally the CR
LF pair is what gets translated to '\n'. I'm actually not 100%
certain what will happen; it might depend on the C implementation.

Precisely! But AGAIN, it might be one of those things that "work"
just about all the time; "statistically", it "works" much more often
across the spectrum of possible implementations than it doesn't
"work"...

Other systems use things other than character sequences to mark line
endings. VMS, for example, has a rather sophisticated
record-management system.

You could still "implement" a version of fseek() for text files that
would "work"...I've actually worked with VMS systems, and getting
VMS file systems to "work" with "C" seemed to be "doable", if not
actually easy...

Some mainframes use fixed-width lines
(inherited from punch card images), though I don't know whether this
is still in use. This kind of thing is exactly why the standard is so
vague about the behavior of fseek on text files.

Exactly...the standard is kind of like "Java" in that regard...but
better, in the sense that it is just a bunch of empty prose, and not
an empty programming "language"...

If you don't mind the fact that your code is non-portable, that's
fine. The fseek trick is likely to be a whole heck of a lot faster
than the portable method of reading the entire file and seeing how it
ends.

This thrills me no end...

For that matter, even that slow method isn't guaranteed to work. C99
7.19.2p2:

Whether the last line requires a terminating new-line character is
implementation-defined.

Yes, although this really isn't a problem for me, I would prefer the
maximum "portable" solution to text files...

And the standard doesn't say what happens if the implementation
requires a terminating new-line and a particular file doesn't have
one. In my opinion, the behavior is undefined. It might report an
error, or it might quietly add a new-line on input.

That's what I've written! If the "implementation" adds the newline,
then I don't have to, even though I really didn't have to in the first
place...

 

[Keith Thompson]

A few comments:
* Both fseek() and rewind() could fail, but you don't check for errors.

[...]

If you want to check for errors, don't use rewind(); it specifically
has no way to report an error condition.

C99 7.19.8.5:

The rewind function sets the file position indicator for the
stream pointed to by stream to the beginning of the file. It is
equivalent to

(void)fseek(stream, 0L, SEEK_SET)

except that the error indicator for the stream is also cleared.

 

[Keith Thompson]

Yeah, heard THAT before...

That sounds dismissive. Was it meant to be?

You'll hear it again, every time you post code here that doesn't check
for errors.

Here's some advice. Decide exactly what your function is supposed to
do, and *document* it. Think about all the corner cases. What if
test_file was opened in binary mode? What if it's not seekable? What
if it's not a well-formed text file for whatever OS you're using?
Think about what your function should do in each of those cases.

In some cases, the answer may well be "I don't care" (equivalent to
the way the standard leaves behavior undefined in many cases).
Perhaps you're 100% certain that some conditions will never occur,
because you'll be careful when you write calls to your function. For
example, if test_file has been passed to fclose(), I don't think
there's anything you can do to detect it; you just have to avoid
making such a call.

But for each possible error condition, *think* about how your function
can and should respond. If you decide not to handle some case, then
*decide* not to handle it; don't just ignore it.

[...]

But will it always work on ACTUAL text files? You know, on-a-disk
text files?

I don't know. I'm fairly sure that it *won't* work on text files on
some systems, but I'm not familiar with every system in existence.
What I am familiar with is the set of guarantees provided by the
standard. I think you've already been told just about everything
there is to say about that.

[snip]

 

[Keith Thompson]

This is how I handle a check that the last character of a text
file is a newline:

/* checks if newline is last character of text file */
unsigned check_text_file_newline_termination(FILE *test_file) { [snip]
if(end_char=='\n') return TRUE;
else return FALSE;
}

The usual type for boolean values (unless you have C99's _Bool) is
int, not unsigned. It doesn't matter, but anyone reading your code is
going to waste a little time wondering why you used unsigned rather than
int.

In general, I'm only interested in boolean values as a return from
a function; this is part of my "scheme" for gracefully exiting from
failed operations which I alluded to in another thread...

Ok. What does that have to do with using unsigned for boolean values?
Why did you choose to use unsigned rather than int? (This is a fairly
minor style issue.)

Yeah, that is so "C", so obscure, so arcane...and a friggin' boolean
value to boot, so I like it...I'll take it under advisement...


Give me a break, I am clearly only interested in ACTUAL text files,
not all the crap that can be called a "file stream"...

What a shame that your function takes a FILE* argument, and can
therefore potentially be called with any stream.

At least document your assumptions. And if you choose to ignore the
results of functions that are designed to give error indications, you
can do that -- but don't expect us not to point it out.

[...]

 

[Keith Thompson]

Keith Thompson wrote, On 20/05/08 01:54:


Better to use fgetpos/fsetpos just in case the position does not fit
in a long (e.g. large files on a system with a 32 bit long, and yes
they do exist before Bill complains about yet another non-existent
problem).

Good point.

I suspect the function itself won't work on such a system (it depends
on fseek() to jump to a point just before the end of the file). But
yes, it could at least restore the file to its previous position and
report and error.

Or it can leave the file at some arbitrary position if that's part of
its defined behavior.

 

[Bill Reid]

Keith Thompson wrote, On 20/05/08 01:54:



Better to use fgetpos/fsetpos just in case the position does not fit in
a long (e.g. large files on a system with a 32 bit long, and yes they do
exist before Bill complains about yet another non-existent problem).

I'm well aware of systems with "short" longs, but I'm willing to run
the risk of not handling TEXT files over 2GB in exchange for 100%
POSIX-compliance, which fgetpos/fsetpos isn't...

 

[Kenneth Brody]

This is how I handle a check that the last character of a text
file is a newline:

/* checks if newline is last character of text file */
unsigned check_text_file_newline_termination(FILE *test_file) {
int end_char;

fseek(test_file,-1L,SEEK_END);

end_char=getc(test_file);

rewind(test_file);

if(end_char=='\n') return TRUE;
else return FALSE;
}

The question is: is this actually guaranteed to work properly on
all "conforming" C "implementations"? My reading of the spec says
"no", but of course it works just fine on the several systems I've
used it on...

What happens on systems where the last character of a text file
isn't necessarily '\n', even if there is a "newline" at the end
of the last line?

Consider MS-DOS, where the last character may be Ctrl-Z.

Now, you're probably saying "but I already tried it on Windows,
where the end-of-line isn't a single character, and it worked!"
Yes, it "worked", because the multi-character end-of-line
sequence happens to have '\n' as the last character. What will
happen if the system happens to use LF-CR rather than CR-LF?

Finally, consider VMS. It's been years, but as I recall, text
files under VMS cannot be randomly-seeked, as the files are
stores as variable-length records, and you can only seek to
a record boundary. (This is a good example of only being able
to pass certain values, such as those returned from ftell.)

Also, would this be the fastest way possible to make this check,
as opposed to a perhaps more "conformant" scheme of reading
every character in the file to find the end of the file?

On systems where your logic "works" (ie: probably most Unix
systems, though even there you have to take into account things
like pipes which cannot be rewound), your method is probably the
fastest. However, on those systems where your logic fails, it's
obviously not "fastest", as it won't work at all, regardless of
how "fast" it fails.


Finally, the definitive answer on portability would be from
7.19.9.2p4:

For a text stream, either offset shall be zero, or offset shall
be a value returned by an earlier successful call to the ftell
function on a stream associated with the same file and whence
shall be SEEK_SET.

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h> |
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <mailto:[email protected]>

 

[Keith Thompson]

I'm well aware of systems with "short" longs, but I'm willing to run
the risk of not handling TEXT files over 2GB in exchange for 100%
POSIX-compliance, which fgetpos/fsetpos isn't...

Where did you get the idea that fgetpos and fsetpos aren't
POSIX-compliant? They're standard C functions (both C90 and C99), and
therefore they're POSIX-compliant as well.

If you're willing to settle for POSIX compliance without necessarily
having code that's fully portable C, you should ask for advice in
comp.unix.programmer (this would let you use fseeko() and ftello(),
for example).

 

[Bill Reid]

Bill Reid wrote:
[...]
fseek(test_file,-1L,SEEK_END);
"For a text stream, either offset shall be zero, or offset shall be a
value returned by an earlier successful call to the ftell function
on a stream associated with the same file and whence shall be
SEEK_SET."

Yeah, that's some good copy'n'pasting there...the documentation
for my "implementation" even reads pretty much the same...and yet,
confoundedly enough, the code works fine, and on other systems
too...

Let's take a quick poll: How many c.l.c. readers have
(1) driven an automobile while not wearing a seat belt,
and (2) been killed in an automobile accident while doing
so? Hands, anyone?

Yeah, and while you're at it, will somebody answer my
question from a few months ago: "how many times have
you been struck by lightening 10 times in a row while
simultaneously being eaten by a shark...on land?"

"I got away with it, once" is not the same as "It works,
always, or even often."

Yes, but in the instant case I don't think it is possible for it
to have worked thousands of times on a particular "implementation"
and then to quit working, and as a matter of fact, if it works ONCE
I can't see why it would fail to work forever...other than that, your
ability to distinguish between different probability domains is
improving, if only slightly...

You will bail out. Instead of returning TRUE "This file
ends with a newline" or FALSE "This file does not end with a
newline," you report "I don't know about this file." The
existence of Boolean algebra does not imply that you can
answer TRUE or FALSE to every question. "TRUE or FALSE: The
human whose dung became the coprolite recently discovered in
Oregon was left-handed." "TRUE or FALSE: The answer to this
question is FALSE."

OK, as usual, I'll "take it under advisement"...

On some systems an invalid fseek() will not produce an
immediate failure, but the subsequent I/O operation to the
invalid location will.

Obviously not what I was doing, but actually something I
might want to think about for other code, since I do use it
for some forms of file parsing...again, though, seems to
work OK, for tens of thousands, even millions, of uses...

On some systems that mark line endings with something
other than a one-byte sentinel, seeking to one byte before
the end of the file and reading what you find there will be
misleading at best.

How often is that? What's your statistical universe, and
is your seat belt fastened?

How often do you port your applications (with a GUI interface)
to different systems anyway?

 

[Bill Reid]

[...]
Strangely enough, I'm not THAT interested whether
this function actually works, [...]

Then why did you start this thread in the first place,
and why do you pursue it?

Idle curiousity. Something came up recently about parsing
out text files, I remembered this code, was kind of wondering
why it worked...I wanted to confirm it was just a figment of
my imagination...