Can I change one line in a file without rewriting the whole thing?

Discussion in 'Python' started by J. J. Ramsey, Jul 14, 2007.

  1. J. J. Ramsey

    J. J. Ramsey Guest

    In Perl, there is a module called "Tie::File". What it does is tie a
    list to each line of a file. Change the list, and the file is
    automatically changed, and on top of this, only the bits of the file
    that need to be changed are written to disk. At least, that's the
    general idea.

    I was wondering if something roughly similar could be done in Python,
    or at the very least, if I can avoid doing what amounts to reading the
    whole file into memory, changing the copy in memory, and writing it
    all out again.
    J. J. Ramsey, Jul 14, 2007
    #1
    1. Advertising

  2. J. J. Ramsey

    David Wahler Guest

    On 7/13/07, J. J. Ramsey <> wrote:
    > In Perl, there is a module called "Tie::File". What it does is tie a
    > list to each line of a file. Change the list, and the file is
    > automatically changed, and on top of this, only the bits of the file
    > that need to be changed are written to disk. At least, that's the
    > general idea.
    >
    > I was wondering if something roughly similar could be done in Python,
    > or at the very least, if I can avoid doing what amounts to reading the
    > whole file into memory, changing the copy in memory, and writing it
    > all out again.


    The mechanism behind Perl's ties -- an array that, when read from or
    written to, passes control to a user function -- is easy to implement
    in Python. See the documentation of the mapping protocol
    <http://docs.python.org/ref/sequence-types.html>:

    >>> class SpecialList(object):

    .... def __getitem__(self, index):
    .... return "Line %d" % index
    >>> foo = SpecialList()
    >>> foo[42]

    'Line 42'

    >From the documentation for Tie::File, it doesn't look like a trivial

    piece of code; for example, it has to maintain a table in memory
    containing the offset of each newline character it's seen for fast
    seeking, and it has to handle moving large chunks of the file if the
    length of a line changes. All this could be implemented in Python, but
    I don't know of a ready-made version off the top of my head.

    If all you want is to read the file line-by-line without having the
    whole thing in memory at once, you can do "
    David Wahler, Jul 14, 2007
    #2
    1. Advertising

  3. En Fri, 13 Jul 2007 23:46:24 -0300, J. J. Ramsey <>
    escribió:

    > In Perl, there is a module called "Tie::File". What it does is tie a
    > list to each line of a file. Change the list, and the file is
    > automatically changed, and on top of this, only the bits of the file
    > that need to be changed are written to disk. At least, that's the
    > general idea.


    That usually means, rewriting from the first modified line to the end of
    the file.

    > I was wondering if something roughly similar could be done in Python,
    > or at the very least, if I can avoid doing what amounts to reading the
    > whole file into memory, changing the copy in memory, and writing it
    > all out again.


    Simplest aproach:

    lines = list(open("myfile.txt"))
    del lines[13]
    lines[42] = "Look ma! Replacing line 42!\n"
    open("myfile.txt","w").writelines(lines)

    This of course reads the whole file in memory, but it's a compact way if
    you require random line access.
    If you can serialize the file operations, try using the fileinput module
    with inplace=1.

    (Having a true Tie::File implementation for Python would be a nice
    addition to the available tools...)

    --
    Gabriel Genellina
    Gabriel Genellina, Jul 14, 2007
    #3
  4. J. J. Ramsey

    greg Guest

    J. J. Ramsey wrote:
    > if I can avoid doing what amounts to reading the
    > whole file into memory, changing the copy in memory, and writing it
    > all out again.


    Except in very special circumstances, not really.
    If you do anything that makes a line longer or
    shorter, everything after that line in the file
    needs to be re-written, one way or another.

    If you're doing this sort of thing a lot, and
    need it to be faster than reading and rewriting the
    file, you may need to look into using a more
    sophisticated format on disk than a plain file.

    --
    Greg
    greg, Jul 14, 2007
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. drgonzo120
    Replies:
    9
    Views:
    576
    Owen Jacobson
    Oct 7, 2005
  2. \A_Michigan_User\
    Replies:
    2
    Views:
    885
    \A_Michigan_User\
    Aug 21, 2006
  3. Merciadri Luca
    Replies:
    4
    Views:
    808
  4. Zam
    Replies:
    1
    Views:
    234
    Mark Schupp
    Mar 14, 2005
  5. Roger Pack
    Replies:
    3
    Views:
    94
    Caleb Clausen
    Dec 2, 2009
Loading...

Share This Page