Binary file Pt 1 - Only reading some

Discussion in 'Python' started by Mastastealth, Feb 5, 2008.

  1. Mastastealth

    Mastastealth Guest

    I'm trying to create a program to read a certain binary format. I have
    the format's spec which goes something like:

    First 6 bytes: String
    Next 4 bytes: 3 digit number and a blank byte
    ---
    Next byte: Height (Number up to 255)
    Next byte: Width (Number up to 255)
    Next byte: Number 0 - 5
    Every 2 bytes after that: Supposedly a number 0000 - 0899?

    Anyway, I'm able to do the first 2 objects fine:

    a = info.read(6)
    b = info.read(4)

    Printing both gives me what I mentioned above, a string and a 3 digit
    number with a space. However, as I continue, things get trickier.

    c = info.read(1)
    d = info.read(1)

    Printing c and d in this case gives me a block in the SPE output, or
    if I run in a DOS prompt, 2 funny symbols. How do I get an integer out
    of this? I'll probably need help once I get to the "every 2 byte"
    section, but that'll be a separate post.
     
    Mastastealth, Feb 5, 2008
    #1
    1. Advertising

  2. On 5 feb, 01:51, Mastastealth <> wrote:
    > I'm trying to create a program to read a certain binary format. I have
    > the format's spec which goes something like:
    >
    > First 6 bytes: String
    > Next 4 bytes: 3 digit number and a blank byte
    > ---
    > Next byte: Height (Number up to 255)
    > Next byte: Width (Number up to 255)
    > Next byte: Number 0 - 5
    > Every 2 bytes after that: Supposedly a number 0000 - 0899?
    >
    > Anyway, I'm able to do the first 2 objects fine:
    >
    > a = info.read(6)
    > b = info.read(4)
    >
    > Printing both gives me what I mentioned above, a string and a 3 digit
    > number with a space. However, as I continue, things get trickier.
    >
    > c = info.read(1)
    > d = info.read(1)
    >
    > Printing c and d in this case gives me a block in the SPE output, or
    > if I run in a DOS prompt, 2 funny symbols. How do I get an integer out
    > of this? I'll probably need help once I get to the "every 2 byte"
    > section, but that'll be a separate post.


    Using the struct module http://docs.python.org/lib/module-struct.html

    import struct
    data = info.read(15)
    str1, str2, blank, height, width, num2, num3 =
    struct.unpack("6s3s1cBBBh", data)

    Consider this like a "first attempt", open issues: is the data little-
    endian or big-endian? does the 0-5 mean 0x00-0x05 or "0"-"5"? the last
    numbers are 2-byte binary integers, or 0000-0899 might indicate BDC?
    But building the right format is surely faster and easier than parsing
    the data by hand.

    --
    Gabriel Genellina
     
    Gabriel Genellina, Feb 5, 2008
    #2
    1. Advertising

  3. Mastastealth

    Mastastealth Guest

    On Feb 5, 1:17 am, Gabriel Genellina <> wrote:
    > Using the struct module  http://docs.python.org/lib/module-struct.html
    >
    > import struct
    > data = info.read(15)
    > str1, str2, blank, height, width, num2, num3 =
    > struct.unpack("6s3s1cBBBh", data)
    >
    > Consider this like a "first attempt", open issues: is the data little-
    > endian or big-endian? does the 0-5 mean 0x00-0x05 or "0"-"5"? the last
    > numbers are 2-byte binary integers, or 0000-0899 might indicate BDC?
    > But building the right format is surely faster and easier than parsing
    > the data by hand.
    >
    > --
    > Gabriel Genellina


    Ah ok, thanks! That worked, though the line "str1, str2, blank,
    height, width, num2, num3 =" spit out a syntax error. However, I do
    see that it creates a tuple with all the values in a readable format
    for me. Also, I needed to change info.read(15) to 16. More questions:

    What is this value for? "6s3s1cBBBh" and why is my unpack limited to a
    length of "16"?

    Unfortunately it seems my understanding of binary is way too basic for
    what I'm dealing with. Can you point me to a simple guide to
    explaining most of it? As far as I know this is just a bunch of 1's
    and 0's right? Each byte has 8 digits of, of which somehow is
    converted to a number or letter. Don't know what most of that stuff in
    the struct page means. -_-

    As for you questions, I suppose it would be "little-endian" as the
    format is on PC (and the Python docs say: "Intel and DEC processors
    are little-endian"). 0-5 means a single digit "0" through "5". Lastly,
    I'm not building the format, it's already made (a format for tiled
    maps in a game). My program is just reading it.
     
    Mastastealth, Feb 5, 2008
    #3
  4. Mastastealth

    Mastastealth Guest

    On Feb 5, 8:50 am, Mastastealth <> wrote:
    > What is this value for? "6s3s1cBBBh" and why is my unpack limited to a
    > length of "16"?
    >
    > Unfortunately it seems my understanding of binary is way too basic for
    > what I'm dealing with. Can you point me to a simple guide to
    > explaining most of it? As far as I know this is just a bunch of 1's
    > and 0's right? Each byte has 8 digits of, of which somehow is
    > converted to a number or letter. Don't know what most of that stuff in
    > the struct page means. -_-


    Ah never mind, after closer inspection is was something on my side of
    the code that ruined the tuple unpacking. Also, I now understand (a
    little) what the string meant.

    6s = A string of 6 characters
    3s = String with 3 characters
    1c = For the blank character
    B's = An unsigned char, any number I guess?
    h = short integer. Whatever that is.

    Either way, just answering myself so you don't have to explain it. :D
     
    Mastastealth, Feb 5, 2008
    #4
  5. En Tue, 05 Feb 2008 11:50:25 -0200, Mastastealth <>
    escribi�:

    > On Feb 5, 1:17 am, Gabriel Genellina <> wrote:
    >> Using the struct module  http://docs.python.org/lib/module-struct.html
    >>
    >> import struct
    >> data = info.read(15)
    >> str1, str2, blank, height, width, num2, num3 =
    >> struct.unpack("6s3s1cBBBh", data)
    >>
    >> Consider this like a "first attempt", open issues: is the data little-
    >> endian or big-endian? does the 0-5 mean 0x00-0x05 or "0"-"5"? the last
    >> numbers are 2-byte binary integers, or 0000-0899 might indicate BDC?
    >> But building the right format is surely faster and easier than parsing
    >> the data by hand.

    >
    > Ah ok, thanks! That worked, though the line "str1, str2, blank,
    > height, width, num2, num3 =" spit out a syntax error. However, I do
    > see that it creates a tuple with all the values in a readable format
    > for me. Also, I needed to change info.read(15) to 16. More questions:
    >
    > What is this value for? "6s3s1cBBBh" and why is my unpack limited to a
    > length of "16"?


    That's the format describing how to decode your bytes: first, a string of
    six characters; then a string of 3 characters followed by a single
    character, three individual bytes, and a short integer.

    > Unfortunately it seems my understanding of binary is way too basic for
    > what I'm dealing with. Can you point me to a simple guide to
    > explaining most of it? As far as I know this is just a bunch of 1's
    > and 0's right? Each byte has 8 digits of, of which somehow is
    > converted to a number or letter. Don't know what most of that stuff in
    > the struct page means. -_-


    I'm afraid I don't know any simple guide to struct. Perhaps people think
    that if you manage binary data with struct, you must already be a Python
    guru or something...
    You could try "construct" instead: "Construct is a python library for
    parsing and building of data structures (binary or textual). It is based
    on the concept of defining data structures in a declarative manner, rather
    than procedural code. [...] It's the first library that makes parsing fun,
    instead of the usual headache it is today."
    http://pyconstruct.wikispaces.com

    There are some recipes in the Python Cookbook too:
    http://aspn.activestate.com/ASPN/Cookbook/Python

    --
    Gabriel Genellina
     
    Gabriel Genellina, Feb 5, 2008
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. ad
    Replies:
    2
    Views:
    747
  2. srikanth
    Replies:
    1
    Views:
    289
    Mike Wahler
    Jun 28, 2005
  3. Guest
    Replies:
    6
    Views:
    1,776
    Guest
    Apr 25, 2010
  4. akhilesh singhania
    Replies:
    0
    Views:
    210
    akhilesh singhania
    Dec 30, 2011
  5. akhilesh singhania
    Replies:
    0
    Views:
    184
    akhilesh singhania
    Jan 1, 2012
Loading...

Share This Page