Negative block sizes with file-like objects

[Steven D'Aprano]

I have a proxy class that wraps an arbitrary file-like object fp and
reads blocks of data from it. Is it safe to assume that fp.read(-1) will
read until EOF? I know that's true for file.read() and StringIO.read(),
but is it a reasonable assumption to make for arbitrary file-like objects?

To put it in more concrete terms, I have a class like this:

class C(object):
# Much simplified version.
def __init__(self, fp):
self.fp = fp
def read(self, size=-1):
return self.fp.read(size)


Should I re-write the read() method like this?

def read(self, size=-1):
if size < 0:
return self.fp.read()
else:
return self.fp.read(size)

 

[Peter Otten]

I have a proxy class that wraps an arbitrary file-like object fp and
reads blocks of data from it. Is it safe to assume that fp.read(-1) will
read until EOF? I know that's true for file.read() and StringIO.read(),
but is it a reasonable assumption to make for arbitrary file-like objects?

To put it in more concrete terms, I have a class like this:

class C(object):
# Much simplified version.
def __init__(self, fp):
self.fp = fp
def read(self, size=-1):
return self.fp.read(size)


Should I re-write the read() method like this?

def read(self, size=-1):
if size < 0:
return self.fp.read()
else:
return self.fp.read(size)

Grepping through the python source shows that both -1 and None are used as
the default size.

# python2.5.2

import tarfile
open("sample", "w").write("contents-of-sample")
t = tarfile.open("sample.tar", "w")
t.add("sample")
t.close()
t = tarfile.open("sample.tar")
t.extractfile("sample").read() 'contents-of-sample'
t.extractfile("sample").read(-1)[:50]

'contents-of-sample\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

So you may fare better with the rewrite. But even that may fail:

http://docs.python.org/lib/bltin-file-objects.html#l2h-295

"""
read( [size])
[...] If the size argument is negative or omitted, read all data until EOF
is reached. [...]
Also note that when in non-blocking mode, less data than what was requested
may be returned, even if no size parameter was given.
"""

Peter