Controlling buffer alignment in file.read()

[Haralanov, Mitko]

Hi all,

I am using Python to read from a binary device file which requires that allread sizes are in 8byte multiples and the user's buffer is 8byte aligned.

I am currently using a file object and the file.read() method. However, theissue is that the file.read() method allocates the buffer passed to C function under the covers and, therefore, the alignment is arbitrary.

Is there a way that I can get file.read() to use an 8byte aligned buffer?

Thanks,
- Mitko

 

[Gregory Ewing]

I am using Python to read from a binary device file which requires that all
read sizes are in 8byte multiples and the user's buffer is 8byte aligned.

Is there a way that I can get file.read() to use an 8byte aligned buffer?

For control at that level you'd be better off using
direct system calls, i.e. os.open() and os.read(),
then you can read exacty the number of bytes you want.

 

[Haralanov, Mitko]

For control at that level you'd be better off using

direct system calls, i.e. os.open() and os.read(),
then you can read exacty the number of bytes you want.

The problem is not controlling the number of bytes read. That part seems tobe working.
The issue is that the buffer into which the data is placed needs to be of certain alignment (8byte-aligned). Python does not seem to have a way that allows me to control that.

Thanks,
- Mitko

 

[Gregory Ewing]

The problem is not controlling the number of bytes read. That part seems to
be working. The issue is that the buffer into which the data is placed needs
to be of certain alignment (8byte-aligned). Python does not seem to have a
way that allows me to control that.

Hmmm, that could be tricky. Have you tried using os.read()?
If you're lucky, Python will be using a malloc() call or
equivalent to create a str/bytes object to read the data
into, and that will return something platform-aligned.

If you're unlucky, there's probably no pure-Python
solution, and you might need to write a small C or
Cython module to accomplish this trick.

 

[Laurent Pointal]

The problem is not controlling the number of bytes read. That part seems
to be working. The issue is that the buffer into which the data is placed
needs to be of certain alignment (8byte-aligned). Python does not seem to
have a way that allows me to control that.

Thanks,
- Mitko

Did you try to set buffering parameter to 0 (unbuffered) ?

Eventually going to os.open() which map tp low level (eventually followed by
an os.fdopen())

http://docs.python.org/2/library/os.html#os.open

A+
Laurent.