[newbie] Saving binaries in a specific way

[Djoser]

Hi all,

I am new to this forum and also to Python, but I'm trying hard to understand it better.
I need to create a binary file, but the first 4 lines must be in signed-Integer16 and all the others in signed-Integer32. I have a program that does that with Matlab and other with Mathematica, but I'm converting all for Python.

I tried first to convert the number to binary using 'bin(number'), than I removed the '0b' and converted to 'Int16' or 'Int32', but with this approach I can't save a binary file using 'bytearray(').

How can I do that?


Thanks in advance.

 

[Chris Kaynor]

Hi all,

I am new to this forum and also to Python, but I'm trying hard to
understand it better.
I need to create a binary file, but the first 4 lines must be in
signed-Integer16 and all the others in signed-Integer32. I have a program
that does that with Matlab and other with Mathematica, but I'm converting
all for Python.

I tried first to convert the number to binary using 'bin(number'), than I
removed the '0b' and converted to 'Int16' or 'Int32', but with this
approach I can't save a binary file using 'bytearray(')

How can I do that?

What you probably want is to use the struct module:
http://docs.python.org/3/library/struct.html

You probably also want the open method and file objects:
http://docs.python.org/3/library/functions.html#open

It would help if you provided information about which Python version you
are using, and exactly what you mean by "I can't save a binary file using
'bytearray(')". Do you get a traceback, if so, copy it in your message,
along with actual code that produces your problem (preferably simplified,
but not too far).

 

[Djoser]

I'm using python 2.7.
If I understood correctly, using bytearray I will lost the information about the signed 16, 32, since it makes automatically the conversion.

Do you think that I can make the conversion as I proposed before or using struct and save with open()?

 

[Tim Chase]

I am new to this forum and also to Python, but I'm trying hard to
understand it better.

Welcome aboard!

I need to create a binary file, but the first 4 lines must be in
signed-Integer16 and all the others in signed-Integer32. I have a
program that does that with Matlab and other with Mathematica, but
I'm converting all for Python.

You seem to be conflating ideas here: a binary file doesn't really
have "lines". Do you mean "first 4 bytes"? If so, then a
signed-Integer16 really only occupies 2 bytes, so you'd have to pad
it somehow. That said, I suspect that the "struct" module will get
you what you want:

from struct import pack
header16bit = 31415
data = list(range(10))
with open('output.bin', 'wb') as f:
f.write(pack('h', header16bit))
for signed_32bit_number in data:
f.write(pack('i', signed_32bit_number))
f.write(other_stuff)

You might need to specify the byte-ordering, which you can do by
prefixing the "h" or "i" with ">", "<" or "=" s documented at [1]

-tkc

[1]
http://www.python.org/doc//current/library/struct.html





..

 

[Djoser]

Basically I have a .dat file, so I get some numbers and make a different conversion.
I'll try this struct script. I'm not used to it, but it seems to do what I want.

 

[rusi]

Basically I have a .dat file, so I get some numbers and make a different conversion.

I'll try this struct script. I'm not used to it, but it seems to do what I want.

Construct is a very powerful utility for binary parsing and building
https://pypi.python.org/pypi/construct
http://construct.readthedocs.org/en/latest/

Though for your currently stated purpose struct should be enough

 

[Oscar Benjamin]

Hi all,

Hi Djoser,

I am new to this forum and also to Python, but I'm trying hard to understand it better.
I need to create a binary file, but the first 4 lines must be in signed-Integer16 and all the others in signed-Integer32. I have a program that does that with Matlab and other with Mathematica, but I'm converting all for Python.

If you're coming from Matlab/Mathematica to Python you will likely
want to use the numpy library. This provides an array type that is
similar to Matlab arrays.

I tried first to convert the number to binary using 'bin(number'), than I removed the '0b' and converted to 'Int16' or 'Int32', but with this approach I can't save a binary file using 'bytearray(').

Using numpy you can do this as follows:

import numpy as np

# Create arrays with the appropriate numeric types
first_numbers = np.array([12, -2, 10, -1], np.int16)
other_numbers = np.array([123, 123, 432, 543, 654, 654], np.int32)

# Output direct to binary file
with open('outputfile.bin', 'wb') as fout:
first_numbers.tofile(fout)
other_numbers.tofile(fout)

# Read back in from binary file
with open('outputfile.bin', 'rb') as fin:
first_numbers_read = np.fromfile(fin, np.int16, count=4)
other_numbers_read = np.fromfile(fin, np.int32)

# Print the data that we read back to check it's right.
print(first_numbers_read)
print(other_numbers_read)


Oscar

 

[Djoser]

Thank you.
With numpy it works perfectly. I thought it would lost the information about int32 and int16 with this approach.

Now I will try to make the script with struct too, but I'll need a bit more time to really understand. For me it's a new paradigm. But that's nice.