C
Chris Lambacher
Hi,
I have to do some data manipulation that needs to be fast. I had a
generator approach (that was faster than a list approch) which was
taking approximately 5 seconds to run both encode and decode.
I have done a conversion to pyrex and have gotten it down to about 2
seconds to run both encode and decode. An an excerpt from the pyrex
code is included below. My question is, is there a better way to
allocate the memory for the array.array object than:
a = array.array('B', [0] * (pixels * 2))
I already know what the lenght is going to be do I have to give it a
sequence(in this case a list of zeros) or is there a C interface that
I can use to tell array to allocate memory for pixels*2 unsigned
chars.
I would prefer to not get into using Numeric or Numarray(I know I can
do it with them).
Yes the algorithm is functionally correct. I know I am throwing away
data. Rest assured that the entropy has been changed such that the
data thrown away does not matter (too much).
Thanks
-Chris
import array
cdef extern from "Python.h":
int PyObject_AsWriteBuffer(object obj, void **buffer, int *buffer_len)
int PyObject_AsReadBuffer(object obj, void **buffer, int *buffer_len)
cdef int get_w_buffer(obj, unsigned char **data, int *length) except -1:
cdef int result
cdef void *vd
# use a void *
result = PyObject_AsWriteBuffer(obj, &vd, length)
data[0] = <unsigned char *>vd
return result
cdef int get_r_buffer(obj, unsigned char **data, int *length) except -1:
cdef int result
cdef void *vd
# use a void *
result = PyObject_AsReadBuffer(obj, &vd, length)
data[0] = <unsigned char *>vd
return result
def encode(int w, int h, in_a):
cdef unsigned int pixels, a_off, str_off
cdef int res, inlen, buffer_len
cdef unsigned char *in_str, *out_str
res = get_r_buffer(in_a, &in_str, &inlen)
if res:
raise Exception, "Could not get a readable buffer from the input"
pixels = w * h
a = array.array('B', [0] * (pixels * 2))
res = get_w_buffer(a, &out_str, &buffer_len)
if res:
raise Exception, "Could not get a writeable buffer to write to"
str_off = 0
a_off = 0
while a_off < buffer_len and str_off < inlen:
out_str[a_off] = in_str[str_off]
out_str[a_off+1] = (in_str[str_off+1] + in_str[str_off+4])/2
out_str[a_off+2] = in_str[str_off+3]
out_str[a_off+3] = (in_str[str_off+2] + in_str[str_off+5])/2
str_off = str_off + 6
a_off = a_off + 4
return a
I have to do some data manipulation that needs to be fast. I had a
generator approach (that was faster than a list approch) which was
taking approximately 5 seconds to run both encode and decode.
I have done a conversion to pyrex and have gotten it down to about 2
seconds to run both encode and decode. An an excerpt from the pyrex
code is included below. My question is, is there a better way to
allocate the memory for the array.array object than:
a = array.array('B', [0] * (pixels * 2))
I already know what the lenght is going to be do I have to give it a
sequence(in this case a list of zeros) or is there a C interface that
I can use to tell array to allocate memory for pixels*2 unsigned
chars.
I would prefer to not get into using Numeric or Numarray(I know I can
do it with them).
Yes the algorithm is functionally correct. I know I am throwing away
data. Rest assured that the entropy has been changed such that the
data thrown away does not matter (too much).
Thanks
-Chris
import array
cdef extern from "Python.h":
int PyObject_AsWriteBuffer(object obj, void **buffer, int *buffer_len)
int PyObject_AsReadBuffer(object obj, void **buffer, int *buffer_len)
cdef int get_w_buffer(obj, unsigned char **data, int *length) except -1:
cdef int result
cdef void *vd
# use a void *
result = PyObject_AsWriteBuffer(obj, &vd, length)
data[0] = <unsigned char *>vd
return result
cdef int get_r_buffer(obj, unsigned char **data, int *length) except -1:
cdef int result
cdef void *vd
# use a void *
result = PyObject_AsReadBuffer(obj, &vd, length)
data[0] = <unsigned char *>vd
return result
def encode(int w, int h, in_a):
cdef unsigned int pixels, a_off, str_off
cdef int res, inlen, buffer_len
cdef unsigned char *in_str, *out_str
res = get_r_buffer(in_a, &in_str, &inlen)
if res:
raise Exception, "Could not get a readable buffer from the input"
pixels = w * h
a = array.array('B', [0] * (pixels * 2))
res = get_w_buffer(a, &out_str, &buffer_len)
if res:
raise Exception, "Could not get a writeable buffer to write to"
str_off = 0
a_off = 0
while a_off < buffer_len and str_off < inlen:
out_str[a_off] = in_str[str_off]
out_str[a_off+1] = (in_str[str_off+1] + in_str[str_off+4])/2
out_str[a_off+2] = in_str[str_off+3]
out_str[a_off+3] = (in_str[str_off+2] + in_str[str_off+5])/2
str_off = str_off + 6
a_off = a_off + 4
return a