Bug on Python2.3.4 [FreeBSD]?

[Uwe Mayer]

<posted & mailed>

Hi,

AFAICT there seems to be a bug on FreeBSD's Python 2.3.4 open function. The
documentation states:

Modes 'r+', 'w+' and 'a+' open the file for updating (note that 'w+'
truncates the file). Append 'b' to the mode to open the file in binary
mode, on systems that differentiate between binary and text files (else it
is ignored). If the file cannot be opened, IOError is raised.

Consider:

$ cat test
lalala

$ python2.3
Python 2.3.4 (#2, Jan 4 2005, 04:42:43)
[GCC 2.95.4 20020320 [FreeBSD]] on freebsd4
Type "help", "copyright", "credits" or "license" for more information.[1]+ Stopped python2.3
$ cat test
lalala

-> write did not work; ok

$ fg
python2.3''

-> append mode does not read from file, *not ok*
$ cat test

-> w+ truncated file, ok


Can anyone confirm that? Is there any other way of opening a file for
appending instead of a+?

Thanks,
Ciao
Uwe

 

[Uwe Mayer]

Friday 12 August 2005 22:12 pm paolino wrote:
[...]

This is right IMO 'a' is appending so seek(-1)

True, thank you.
f.tell() shows the file pointer is at EOF. On my Debian Linux (unstable),
Python 2.3.4 +2.3.5, however, the file pointer is at the beginning of the
file.
Is that behaviour intended?

Ciao
Uwe

 

[David Bolen]

AFAICT there seems to be a bug on FreeBSD's Python 2.3.4 open function. The
documentation states:

Modes 'r+', 'w+' and 'a+' open the file for updating (note that 'w+'
truncates the file). Append 'b' to the mode to open the file in binary
mode, on systems that differentiate between binary and text files (else it
is ignored). If the file cannot be opened, IOError is raised.

Consider:

$ cat test
lalala

$ python2.3
Python 2.3.4 (#2, Jan 4 2005, 04:42:43)
[GCC 2.95.4 20020320 [FreeBSD]] on freebsd4
Type "help", "copyright", "credits" or "license" for more information.[1]+ Stopped python2.3
$ cat test
lalala

-> write did not work; ok

Strange, I tried this with Python 2.3.3 and 2.3.5 on two FreeBSD 4.10
systems and it seemed to append to the file properly in both cases.
Going back further, it also worked with Python 2.2.2 on a FreeBSD 4.7
system. I don't see happen to have a 2.3.4 installation, but can't
see any changes to the source for the file object between 2.3.4 and
2.3.5, for example.

~> python
Python 2.3.5 (#2, May 5 2005, 11:11:17)
[GCC 2.95.4 20020320 [FreeBSD]] on freebsd4
Type "help", "copyright", "credits" or "license" for more information.~> cat test
lalala
testing # no newline was present

Which version of FreeBSD are you running? I thought it might be a
dependency on needing to seek between reads and writes on a duplex
stream (which is ANSI), but FreeBSD doesn't require that, at least
back as far as a 4.7 system I have, and I assume much earlier than
that.

One dumb question - are you absolutely sure it wasn't appending? As
written, there's no trailing newline on the file, so your final "cat
test" would produce output where the "testing" was on the same line as
your next command prompt, and can sometimes be missed visually.

Can anyone confirm that? Is there any other way of opening a file for
appending instead of a+?

Well, if you really just want appending, I'd just use "a". It creates
the file if necessary but always appends to the end. Of course, it's
not set up for reading, but you wouldn't need that for appending.

-- David

 

[paolino]

<posted & mailed>

Hi,

AFAICT there seems to be a bug on FreeBSD's Python 2.3.4 open function. The
documentation states:

Modes 'r+', 'w+' and 'a+' open the file for updating (note that 'w+'
truncates the file). Append 'b' to the mode to open the file in binary
mode, on systems that differentiate between binary and text files (else it
is ignored). If the file cannot be opened, IOError is raised.

Consider:

$ cat test
lalala

$ python2.3
Python 2.3.4 (#2, Jan 4 2005, 04:42:43)
[GCC 2.95.4 20020320 [FreeBSD]] on freebsd4
Type "help", "copyright", "credits" or "license" for more information.

'lalala\n'

[1]+ Stopped python2.3
$ cat test
lalala

-> write did not work; ok

This worked here on freebsd 5.4 / python 2.4

$ fg
python2.3


''

-> append mode does not read from file, *not ok*

This is right IMO 'a' is appending so seek(-1)


Ciao

 

[Uwe Mayer]

Which version of FreeBSD are you running? I thought it might be a
dependency on needing to seek between reads and writes on a duplex
stream (which is ANSI), but FreeBSD doesn't require that, at least
back as far as a 4.7 system I have, and I assume much earlier than
that.

One dumb question - are you absolutely sure it wasn't appending? As
written, there's no trailing newline on the file, so your final "cat
test" would produce output where the "testing" was on the same line as
your next command prompt, and can sometimes be missed visually.


Well, if you really just want appending, I'd just use "a". It creates
the file if necessary but always appends to the end. Of course, it's
not set up for reading, but you wouldn't need that for appending.

I was debugging an application I released, on a forreign FreeBSD system:
$ uname -a
FreeBSD hephaistos 4.11-RELEASE FreeBSD 4.11-RELEASE #0: Sun Feb 27 21:09:39
CET 2005 (e-mail address removed):/storage/obj-4.11/usr/src/sys/OLYMPUS i386

The application as able to create and write to files, but not to open and
read them. It seems the file poitner is positioned at the end of a file
when using:

on FreeBSD, but at the beginning of the file on Linux (at least on my Debian
unstable, Python 2.3.4 and 2.3.5; its running on Gentoo Linux and Suse,
too, though I don't know the Python version numbers).
I was assuming the fp was set to 0 in this mode, thus the app was not
working on above FreeBSD and the owner set up a login for me to debug the
situation.

The problem is easyly fixed by appending a f.seek(0) after opening the file,
but the position of the fp is strange.

Thanks,
Ciao
Uwe

 

[Donn Cave]

Friday 12 August 2005 22:12 pm paolino wrote:
[...]

This is right IMO 'a' is appending so seek(-1)

True, thank you.
f.tell() shows the file pointer is at EOF. On my Debian Linux (unstable),
Python 2.3.4 +2.3.5, however, the file pointer is at the beginning of the
file.
Is that behaviour intended?

I don't think Python pretends to have any intentions here,
it has to take what it gets from the C library fopen(3)
function. BSD man pages generally say a+ positions the
stream at end of file (period.) They claim conformance
with the ISO C90 standard. I couldn't dig up a (free) copy
of that document, so don't know what it says on this matter.

GNU C man pages say it positions the stream at end for
write and at beginning for read.

Donn Cave, (e-mail address removed)

 

[Terry Reedy]

I don't think Python pretends to have any intentions here,
it has to take what it gets from the C library fopen(3)
function. BSD man pages generally say a+ positions the
stream at end of file (period.) They claim conformance
with the ISO C90 standard. I couldn't dig up a (free) copy
of that document, so don't know what it says on this matter.

STandard C, by Plauger & Brodie says that 'a' plus whatever else means all
writes start at the current end-of-file.

TJR

 

[Donn Cave]

Quoth "Terry Reedy" <[email protected]>:
| | > I don't think Python pretends to have any intentions here,
| > it has to take what it gets from the C library fopen(3)
| > function. BSD man pages generally say a+ positions the
| > stream at end of file (period.) They claim conformance
| > with the ISO C90 standard. I couldn't dig up a (free) copy
| > of that document, so don't know what it says on this matter.
|
| STandard C, by Plauger & Brodie says that 'a' plus whatever else means all
| writes start at the current end-of-file.

Of course, but the question was, where do reads start? I would
guess the GNU C library "innovated" on this point. But in the
end it doesn't really matter unless Python is going to try to
square that all up and make open() consistent across platforms.

Donn Cave, (e-mail address removed)

 

[Uwe Mayer]

Of course, but the question was, where do reads start? I would
guess the GNU C library "innovated" on this point. But in the
end it doesn't really matter unless Python is going to try to
square that all up and make open() consistent across platforms.

Personally, I think Python should unify this difference across plattforms
and explicitly document the behaviour in the library reference.
But I don't know how a decision on that is going to be formed.

I will forward a request for taking this point up into the library
reference's file() function, perhaps a footnote pointing out the
end-of-file position on BSD and start-of-file on Linux.

Uwe

 

[Terry Reedy]

Quoth "Terry Reedy" <[email protected]>:
| Standard C, by Plauger & Brodie says that 'a' plus whatever else means
all
| writes start at the current end-of-file.

Of course, but the question was, where do reads start?

For 'a+', that book, and perhaps the standard, does not specify where an
*initial* read starts. It only says file position is set to end-of-file
'before each write' and that reads after a write (and vice versa) 'must'
follow an intervening file-positioning call (fflush, fseek, fsetpos,
rewind).

guess the GNU C library "innovated" on this point.

If there is a hole in the standard, 'innovation' is required.

Terry J. Reedy

 

[curtis.rendon]

Using FreeBSD 4.10 and Python 2.3.4:

uname -a

FreeBSD garner_ted 4.10-RELEASE FreeBSD 4.10-RELEASE #7: Thu Apr 28
22:44:58 CDT 2005

python

Python 2.3.4 (#4, Nov 19 2004, 15:37:16)
[GCC 2.95.4 20020320 [FreeBSD]] on freebsd4
Type "help", "copyright", "credits" or "license" for more information.

using f.read('test','r+')
I get the existing data, and the
f.write('testing')
successfully appends the text in the file.

The 'a+' mode works as described, i.e. I need to f.seek(0) to read the
data.

I am now enlightened as to the usefulness of 'r+', as it starts the
read fp at the begining of the file.

Curtis W. Rendon

 

[Terry Reedy]

I am now enlightened as to the usefulness of 'r+', as it starts the

read fp at the begining of the file.

Both 'r+' and 'w+' let you write anywhere, whereas 'a+' is supposed to make
all writes EOF appends. 'r+' (and 'r') requires an existing file while
'w+' (like 'w') will open a new file or truncate an existing file.

Terry J. Reedy

 

[Donn Cave]

If there is a hole in the standard, 'innovation' is required.

I hope this perspective is a rarity. When you exploit an opportunity
to make something work differently while conforming to the existing
standards, you're creating the kind of problem standards are there
to prevent. In the end I don't care if my software works because
someone followed the standards to the letter, or because someone took
the trouble to follow existing practice whether it was codified in
a standard or not, I just don't want it to work differently on one
platform than on another. Holes in standards are at best an excuse
for accidental deviations.

In the present case, so far I see a strong Berkeley vs. everyone
else pattern, so GNU C probably wasn't the culprit after all.
Along with already documented FreeBSD, I find MacOS X, NetBSD 2
and Ultrix 4.2 position the read stream to EOF. Linux, AIX and
DEC/OSF1 (or whatever it's called these days) position it to 0.

Donn Cave, (e-mail address removed)

 

[Terry Reedy]

I hope this perspective is a rarity.

I think you over interpreted. The ''s were intentional. Let me reword.
If one if implementing a standard and it is *silent* on an initial
condition then one must *choose* something (or let it be complete accident,
and even possibly invalid).

If there is one accessible and known existing 'standard' practice that is
at least as reasonable as anything else, and the hole seems to be an
accident rather than intentional, then I too would want to be that choice
made. But if there are two existing 'standard' practices, then there is a
real choice to be made. (But either would be better than a third.)

> In the present case, so far I see a strong Berkeley vs. everyone
else pattern, so GNU C probably wasn't the culprit after all.
Along with already documented FreeBSD, I find MacOS X, NetBSD 2
and Ultrix 4.2 position the read stream to EOF. Linux, AIX and
DEC/OSF1 (or whatever it's called these days) position it to 0.

The person I responded to gave almost none of this detail. With this info,
and the knowledge that Berkeley got Unix from Bell Labs about 1975, I
speculate that the hole of not specifying the initial read position was
intentional due to divergent well-established prior arts that neither group
wanted to give up. And yes, I also wish that someone had. I even wish the
committee had at least specified 'either 0 or EOF' to preclude a Solomonic
compromise like EOF//2.

Terry J. Reedy