concatenate file-like objects -> file-like object

Discussion in 'Python' started by kgk, Jul 11, 2007.

  1. kgk

    kgk Guest

    I would like to concatenate several file-like objects
    to create a single file-like object. I've looked at fileinput,
    however
    this returns a fileinput object that is not very file-like.

    something like
    # a has 50 bytes, and b has 100 bytes
    f = FileList (open('a'), open('b'))
    f.read (100) # read 50 bytes from a and 50 from b

    My interest is in passing several files to an incremental parser
    as if they came from a single file. I would rather not load them
    in memory using StringIO and the parser reads only from file-like
    objects.

    Any pointers appreciated
    Kris
     
    kgk, Jul 11, 2007
    #1
    1. Advertising

  2. On Tue, 10 Jul 2007 17:55:52 -0700, kgk wrote:

    > I would like to concatenate several file-like objects
    > to create a single file-like object. I've looked at fileinput,
    > however
    > this returns a fileinput object that is not very file-like.
    >
    > something like
    > # a has 50 bytes, and b has 100 bytes
    > f = FileList (open('a'), open('b'))
    > f.read (100) # read 50 bytes from a and 50 from b
    >
    > My interest is in passing several files to an incremental parser
    > as if they came from a single file. I would rather not load them
    > in memory using StringIO and the parser reads only from file-like
    > objects.


    Then program a file like object yourself. Something like this (untestet):

    class FileList(object):
    def __init__(self, files):
    self.files = reversed(files)
    self.current_file = self.files.pop()

    def read(size):
    result = ''
    while self.files:
    data = self.current_file.read(size)
    result += data
    if len(data) != size:
    self.current_file = self.files.pop()
    size = size - len(data)
    return result

    Ciao,
    Marc 'BlackJack' Rintsch
     
    Marc 'BlackJack' Rintsch, Jul 11, 2007
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    9
    Views:
    8,283
  2. Will Rocisky
    Replies:
    1
    Views:
    364
    Steven D'Aprano
    Aug 5, 2008
  3. Replies:
    7
    Views:
    179
    A. Sinan Unur
    Feb 20, 2006
  4. Íéêüëáïò Êïýñáò

    cannot concatenate 'str' and 'list' objects

    Íéêüëáïò Êïýñáò, Sep 15, 2012, in forum: Python
    Replies:
    42
    Views:
    1,239
    Chris Angelico
    Sep 16, 2012
  5. Carlos

    Concatenate/De-Concatenate

    Carlos, Oct 12, 2012, in forum: VHDL
    Replies:
    10
    Views:
    961
Loading...

Share This Page