stripping the first byte from a binary file

Discussion in 'Python' started by rvr, Jul 10, 2007.

  1. rvr

    rvr Guest

    Would someone mind showing me how to strip the first byte from a
    binary file? For some reason I can't figure this out from the binary
    file editing examples I've read. Thanks.

    ~rvr
     
    rvr, Jul 10, 2007
    #1
    1. Advertising

  2. rvr wrote:

    > Would someone mind showing me how to strip the first byte from a
    > binary file? For some reason I can't figure this out from the binary
    > file editing examples I've read. Thanks.


    Do you mean something like this?

    f = open('test.dat', 'rb')
    f.read(1) # read 1st byte and ignore it
    rest = f.read() # read rest

    or

    data = f.read()
    data = data[1:] # skip 1st byte

    ?

    --
    Jeremy Sanders
    http://www.jeremysanders.net/
     
    Jeremy Sanders, Jul 10, 2007
    #2
    1. Advertising

  3. rvr

    rvr Guest

    On Jul 10, 6:37 pm, Jeremy Sanders <jeremy
    > wrote:
    > rvr wrote:
    > > Would someone mind showing me how to strip the first byte from a
    > > binary file? For some reason I can't figure this out from the binary
    > > file editing examples I've read. Thanks.

    >
    > Do you mean something like this?
    >
    > f = open('test.dat', 'rb')
    > f.read(1) # read 1st byte and ignore it
    > rest = f.read() # read rest
    >
    > or
    >
    > data = f.read()
    > data = data[1:] # skip 1st byte


    Is there a way to edit the file in place? The best I seem to be able
    to do is to use your second solution to read the file into the string,
    then re-open the file for writing and put the whole thing back (minus
    the first byte). Thanks.

    ~rvr
     
    rvr, Jul 11, 2007
    #3
  4. On Wed, 11 Jul 2007 01:06:04 +0000, rvr wrote:

    > Is there a way to edit the file in place? The best I seem to be able to
    > do is to use your second solution to read the file into the string, then
    > re-open the file for writing and put the whole thing back (minus the
    > first byte). Thanks.


    I don't believe that any of the popular operating systems in common use
    (Windows, Linux, Mac, *BSD) have any such functionality.

    For safety, you are best off copying the file (minus the first byte) to a
    temporary file, then renaming the copy over the original. That way if
    your process dies midway through copying the file, you don't lose data.

    Renaming the file is atomic under Linux and (probably) Mac, so it is as
    safe as possible. Even under Windows, which isn't atomic, it has a
    smaller margin for disaster than over-writing the file in place.


    --
    Steven.
     
    Steven D'Aprano, Jul 11, 2007
    #4
  5. rvr

    rvr Guest

    On Jul 11, 1:28 pm, Steven D'Aprano
    <> wrote:
    > On Wed, 11 Jul 2007 01:06:04 +0000, rvr wrote:
    > > Is there a way to edit the file in place? The best I seem to be able to
    > > do is to use your second solution to read the file into the string, then
    > > re-open the file for writing and put the whole thing back (minus the
    > > first byte). Thanks.

    >
    > I don't believe that any of the popular operating systems in common use
    > (Windows, Linux, Mac, *BSD) have any such functionality.
    >
    > For safety, you are best off copying the file (minus the first byte) to a
    > temporary file, then renaming the copy over the original. That way if
    > your process dies midway through copying the file, you don't lose data.
    >
    > Renaming the file is atomic under Linux and (probably) Mac, so it is as
    > safe as possible. Even under Windows, which isn't atomic, it has a
    > smaller margin for disaster than over-writing the file in place.


    Thanks for your response. While searching for solution, I found this:

    http://mail.python.org/pipermail/python-list/2001-December/116519.html

    Quoting from it:

    """
    Replace 2 bytes in place beginning at offset 100 (101st byte):

    f = open('text_input', 'r+b')
    f.seek(100)
    f.write(chr(123) + chr(0x80))
    f.seek(0,2)
    f.close()
    """

    Can I use the seek() and write() methods in a similar way to remove
    the first byte? For whatever reason I can't seem to make it work
    myself. Thanks again.

    ~rvr
     
    rvr, Jul 11, 2007
    #5
  6. rvr wrote:
    > On Jul 11, 1:28 pm, Steven D'Aprano
    > <> wrote:
    >> On Wed, 11 Jul 2007 01:06:04 +0000, rvr wrote:
    >>> Is there a way to edit the file in place? The best I seem to be able to
    >>> do is to use your second solution to read the file into the string, then
    >>> re-open the file for writing and put the whole thing back (minus the
    >>> first byte). Thanks.

    >> I don't believe that any of the popular operating systems in common use
    >> (Windows, Linux, Mac, *BSD) have any such functionality.
    >>
    >> For safety, you are best off copying the file (minus the first byte) to a
    >> temporary file, then renaming the copy over the original. That way if
    >> your process dies midway through copying the file, you don't lose data.
    >>
    >> Renaming the file is atomic under Linux and (probably) Mac, so it is as
    >> safe as possible. Even under Windows, which isn't atomic, it has a
    >> smaller margin for disaster than over-writing the file in place.

    >
    > Thanks for your response. While searching for solution, I found this:
    >
    > http://mail.python.org/pipermail/python-list/2001-December/116519.html
    >
    > Quoting from it:
    >
    > """
    > Replace 2 bytes in place beginning at offset 100 (101st byte):
    >
    > f = open('text_input', 'r+b')
    > f.seek(100)
    > f.write(chr(123) + chr(0x80))
    > f.seek(0,2)
    > f.close()
    > """
    >
    > Can I use the seek() and write() methods in a similar way to remove
    > the first byte? For whatever reason I can't seem to make it work
    > myself. Thanks again.


    Funny. I just happened to read ESR's "how to ask questions the smart way" and
    your posts match quite a few of the examples. :)

    No, you can't. Steven's solution is what I'd go for.

    Stefan
     
    Stefan Behnel, Jul 11, 2007
    #6
  7. rvr

    Alex Popescu Guest

    On Jul 11, 1:25 pm, Stefan Behnel <> wrote:
    > rvr wrote:
    > > On Jul 11, 1:28 pm, Steven D'Aprano
    > > <> wrote:
    > >> On Wed, 11 Jul 2007 01:06:04 +0000, rvr wrote:
    > >>> Is there a way to edit the file in place? The best I seem to be able to
    > >>> do is to use your second solution to read the file into the string, then
    > >>> re-open the file for writing and put the whole thing back (minus the
    > >>> first byte). Thanks.
    > >> I don't believe that any of the popular operating systems in common use
    > >> (Windows, Linux, Mac, *BSD) have any such functionality.

    >
    > >> For safety, you are best off copying the file (minus the first byte) to a
    > >> temporary file, then renaming the copy over the original. That way if
    > >> your process dies midway through copying the file, you don't lose data.

    >
    > >> Renaming the file is atomic under Linux and (probably) Mac, so it is as
    > >> safe as possible. Even under Windows, which isn't atomic, it has a
    > >> smaller margin for disaster than over-writing the file in place.

    >
    > > Thanks for your response. While searching for solution, I found this:

    >
    > > http://mail.python.org/pipermail/python-list/2001-December/116519.html

    >
    > > Quoting from it:

    >
    > > """
    > > Replace 2 bytes in place beginning at offset 100 (101st byte):

    >
    > > f = open('text_input', 'r+b')
    > > f.seek(100)
    > > f.write(chr(123) + chr(0x80))
    > > f.seek(0,2)
    > > f.close()
    > > """

    >
    > > Can I use the seek() and write() methods in a similar way to remove
    > > the first byte? For whatever reason I can't seem to make it work
    > > myself. Thanks again.

    >
    > Funny. I just happened to read ESR's "how to ask questions the smart way" and
    > your posts match quite a few of the examples. :)
    >
    > No, you can't. Steven's solution is what I'd go for.
    >
    > Stefan


    Forgive my newbie ignorance, but I am wondering why the other method
    would not work? I mean it may not be very safe,
    but I guess it may perform a lot better, than having to read the whole
    file just to cut out the first byte.

    TIA,

    ../alex
    --
    ..w( the_mindstorm )p.
     
    Alex Popescu, Jul 11, 2007
    #7
  8. >
    > Forgive my newbie ignorance, but I am wondering why the other method
    > would not work? I mean it may not be very safe,
    > but I guess it may perform a lot better, than having to read the whole
    > file just to cut out the first byte.


    Because seeking is not moving? Shifting data bytewise isn't something that
    is supported by the underlying OS filesystems, and thus not supported. But
    replacing bytes with others is. Which seek is for.

    Diez
     
    Diez B. Roggisch, Jul 11, 2007
    #8
  9. Alex Popescu wrote:
    > Forgive my newbie ignorance, but I am wondering why the other method
    > would not work? I mean it may not be very safe,
    > but I guess it may perform a lot better, than having to read the whole
    > file just to cut out the first byte.


    Why would you expect that? It *might* perform better if there was a system
    call for removing bytes from inside a file, as that could reduce the
    intermediate space requirements to the size of a hard disk sector rather than
    the remaining size of the file (note that the time consumption would not be
    reduced significantly, if the remaining file has to be copied around to fill
    the sectors). But since that is a rather rare use case that most people would
    prefer being handled in a safe rather than space-optimal way, I don't see the
    need for such a function.

    Stefan
     
    Stefan Behnel, Jul 11, 2007
    #9
  10. rvr

    Alex Popescu Guest

    On Jul 11, 4:15 pm, "Diez B. Roggisch" <> wrote:
    > > Forgive my newbie ignorance, but I am wondering why the other method
    > > would not work? I mean it may not be very safe,
    > > but I guess it may perform a lot better, than having to read the whole
    > > file just to cut out the first byte.

    >
    > Because seeking is not moving? Shifting data bytewise isn't something that
    > is supported by the underlying OS filesystems, and thus not supported. But
    > replacing bytes with others is. Which seek is for.
    >
    > Diez


    As far as I know seek is just about positioning and nothing else.
    So, in fact the problem boils down to os support for deleting a bytes.

    bests,

    ../alex
    --
    ..w( the_mindstorm )p.
     
    Alex Popescu, Jul 11, 2007
    #10
  11. Alex Popescu wrote:

    > On Jul 11, 4:15 pm, "Diez B. Roggisch" <> wrote:
    >> > Forgive my newbie ignorance, but I am wondering why the other method
    >> > would not work? I mean it may not be very safe,
    >> > but I guess it may perform a lot better, than having to read the whole
    >> > file just to cut out the first byte.

    >>
    >> Because seeking is not moving? Shifting data bytewise isn't something
    >> that is supported by the underlying OS filesystems, and thus not
    >> supported. But replacing bytes with others is. Which seek is for.
    >>
    >> Diez

    >
    > As far as I know seek is just about positioning and nothing else.


    It is.

    > So, in fact the problem boils down to os support for deleting a bytes.


    Which there isn't, as I and several others pointed out. And "replacing" is
    not removing.

    Diez
     
    Diez B. Roggisch, Jul 11, 2007
    #11
  12. rvr

    Alex Popescu Guest

    On Jul 11, 7:45 pm, "Diez B. Roggisch" <> wrote:
    > Alex Popescu wrote:
    > > On Jul 11, 4:15 pm, "Diez B. Roggisch" <> wrote:
    > >> > Forgive my newbie ignorance, but I am wondering why the other method
    > >> > would not work? I mean it may not be very safe,
    > >> > but I guess it may perform a lot better, than having to read the whole
    > >> > file just to cut out the first byte.

    >
    > >> Because seeking is not moving? Shifting data bytewise isn't something
    > >> that is supported by the underlying OS filesystems, and thus not
    > >> supported. But replacing bytes with others is. Which seek is for.

    >
    > >> Diez

    >
    > > As far as I know seek is just about positioning and nothing else.

    >
    > It is.
    >
    > > So, in fact the problem boils down to os support for deleting a bytes.

    >
    > Which there isn't, as I and several others pointed out. And "replacing" is
    > not removing.
    >
    > Diez


    I do agree with your last statement (and it was my mistake to consider
    replacing a possible way to remove). Just wanted to
    clarify that seek is only for positioning.

    bests,

    ../alex
    --
    ..w( the_mindstorm )p.
     
    Alex Popescu, Jul 11, 2007
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Guest
    Replies:
    2
    Views:
    10,923
    John Harrison
    Jul 26, 2004
  2. Replies:
    20
    Views:
    9,795
    licebmi
    Sep 7, 2009
  3. Replies:
    7
    Views:
    500
  4. daved170

    read text file byte by byte

    daved170, Dec 12, 2009, in forum: Python
    Replies:
    30
    Views:
    1,856
    Nobody
    Dec 16, 2009
  5. Richard Schneeman
    Replies:
    16
    Views:
    494
    Daniel Bush
    Aug 27, 2008
Loading...

Share This Page