concatenate file-like objects -> file-like object

[kgk]

I would like to concatenate several file-like objects
to create a single file-like object. I've looked at fileinput,
however
this returns a fileinput object that is not very file-like.

something like
# a has 50 bytes, and b has 100 bytes
f = FileList (open('a'), open('b'))
f.read (100) # read 50 bytes from a and 50 from b

My interest is in passing several files to an incremental parser
as if they came from a single file. I would rather not load them
in memory using StringIO and the parser reads only from file-like
objects.

Any pointers appreciated
Kris

 

[Marc 'BlackJack' Rintsch]

I would like to concatenate several file-like objects
to create a single file-like object. I've looked at fileinput,
however
this returns a fileinput object that is not very file-like.

something like
# a has 50 bytes, and b has 100 bytes
f = FileList (open('a'), open('b'))
f.read (100) # read 50 bytes from a and 50 from b

My interest is in passing several files to an incremental parser
as if they came from a single file. I would rather not load them
in memory using StringIO and the parser reads only from file-like
objects.

Then program a file like object yourself. Something like this (untestet):

class FileList(object):
def __init__(self, files):
self.files = reversed(files)
self.current_file = self.files.pop()

def read(size):
result = ''
while self.files:
data = self.current_file.read(size)
result += data
if len(data) != size:
self.current_file = self.files.pop()
size = size - len(data)
return result

Ciao,
Marc 'BlackJack' Rintsch