help with c <-> python buffer transfer

[tkirke]

How does one transfer a buffer object from python -> c and back again
(assuming the data gets modified)?
I can't seem to get this or anything else to work, but am clueless as
to what I'm doing wrong


using namespace boost:ython;

static PyObject * proc_buf(PyObject *self, PyObject *args) {
PyObject *resultobj;
char* output_samples;
int len;
if (!PyArg_ParseTuple(args,"s#|l",&output_samples, &len)) {
return NULL; /* wrong arguments provided */
}
for (int i=0;i<len;i++) {
output_samples *= 2; // for example
}
resultobj = PyString_FromStringAndSize(output_samples, len);
return resultobj;
}


This compiles ok, but when in python I do

buf = proc_buf( bufx, len(bufx)
len(buf)

I get
len() of unsized object

Thanks

 

[John Machin]

How does one transfer a buffer object from python -> c and back again
(assuming the data gets modified)?
I can't seem to get this or anything else to work, but am clueless as
to what I'm doing wrong


using namespace boost:ython;

Looks like C++, not C.

static PyObject * proc_buf(PyObject *self, PyObject *args) {

[I'm not familiar with the boost gadget, but ...] Doesn't "static" mean
that this function is *not* externally visible?

PyObject *resultobj;
char* output_samples;
int len;
if (!PyArg_ParseTuple(args,"s#|l",&output_samples, &len)) {

You have made the length an optional argument, but not initialised the
receiving variable "len". Nothing to do with your current problem, but
highly dangerous.

return NULL; /* wrong arguments provided */
}
for (int i=0;i<len;i++) {
output_samples *= 2; // for example
}

This is updating the internal representation of the input in situ. Not
a very good idea at all. Take a copy. Return the updated copy.

resultobj = PyString_FromStringAndSize(output_samples, len);
return resultobj;
}


This compiles ok, but when in python I do

Put print repr(bufx), type(bufx) here so that we're all clued in on
what you are talking about. You say "transfer a buffer object" but your
C[++] is returning a string object.

buf = proc_buf( bufx, len(bufx)

You are missing both a module name and a ")" here. It should look
something like:

buf = theextensionmodule.proc_buf( bufx, len(bufx))

Please *always* copy/paste the actual code that you executed.

len(buf)

I get
len() of unsized object

Please *always* copy/paste the actual error message & stack trace that
you get.

Try print repr(buf), type(buf) here; it might give you a clue as to
what type of object you have that is unsized. On the surface this is a
mystery, as (based on the info that you have supplied), "buf" should be
a string.

HTH .... alternatively come up a level or three and tell us what your
basic requirement is; maybe it can be solved more easily in Python or
Pyrex.

Cheers,
John

 

[tkirke]

How does one transfer a buffer object from python -> c and back again
(assuming the data gets modified)?
I can't seem to get this or anything else to work, but am clueless as
to what I'm doing wrong


using namespace boost:ython;

Looks like C++, not C.

static PyObject * proc_buf(PyObject *self, PyObject *args) {

[I'm not familiar with the boost gadget, but ...] Doesn't "static" mean
that this function is *not* externally visible

Yes. It's C++. I've built python extensions with Boost Python
successfully and copied the structure of most of this example from
other people's code.

PyObject *resultobj;
char* output_samples;
int len;
if (!PyArg_ParseTuple(args,"s#|l",&output_samples, &len)) {

You have made the length an optional argument, but not initialised the
receiving variable "len". Nothing to do with your current problem, but
highly dangerous.

return NULL; /* wrong arguments provided */
}
for (int i=0;i<len;i++) {
output_samples *= 2; // for example
}

This is updating the internal representation of the input in situ. Not
a very good idea at all. Take a copy. Return the updated copy.

Thanks for the pointers...
This is the part I need help with. I've also used PyBuffer_...
subroutines which gave similar problems. Thanks for all of your other
comments, but I was hoping someone could just tell me what was wrong
with the code without having to worry about all of the other things
that could go wrong.
For completeness here is the complete c++ module & python output

//=======================================================================
// Boost Includes
//==============================================================
#include <boost/python.hpp>
#include <boost/cstdint.hpp>
#include <boost/python/def.hpp>
#include <boost/python/args.hpp>
#include <boost/python/overloads.hpp>

// Using
=======================================================================
using namespace boost:ython;

static PyObject * proc_buf(PyObject *self, PyObject *args) {
PyObject *resultobj;
char* output_samples;
int len;
if (!PyArg_ParseTuple(args,"s#|l",&output_samples, &len)) {
return NULL; /* wrong arguments provided */
}
for (int i=0;i<len;i++) output_samples *= 2;
resultobj = PyString_FromStringAndSize(output_samples, len);
return resultobj;
}
// Module
======================================================================
BOOST_PYTHON_MODULE(spuctest)
{
def("pass_buf",&proc_buf);
}

# python code
.....
buffy = mf.read()
print type(buffy)
buf = pass_buf(buffy, len(buffy))
print type(buf)
#

#python output
<type 'buffer'>
<type 'Nonetype'>

 

[John Machin]

This is the part I need help with. I've also used PyBuffer_...
subroutines which gave similar problems. Thanks for all of your other
comments, but I was hoping someone could just tell me what was wrong
with the code without having to worry about all of the other things
that could go wrong.
For completeness here is the complete c++ module & python output

//=======================================================================
// Boost Includes
//==============================================================
#include <boost/python.hpp>
#include <boost/cstdint.hpp>
#include <boost/python/def.hpp>
#include <boost/python/args.hpp>
#include <boost/python/overloads.hpp>

// Using
=======================================================================
using namespace boost:ython;

static PyObject * proc_buf(PyObject *self, PyObject *args) {
PyObject *resultobj;
char* output_samples;
int len;
if (!PyArg_ParseTuple(args,"s#|l",&output_samples, &len)) {

Sorry, I read that too quyickly. You need THREE receptors, one for "s",
2nd for "#", 3rd for "l".
e.g. something like:
int len;
long third_arg; /* needs initialisation */
if (!PyArg_ParseTuple(args,"s#|l",&output_samples, &len,
&third_arg)) {

return NULL; /* wrong arguments provided */
}
for (int i=0;i<len;i++) output_samples *= 2;
resultobj = PyString_FromStringAndSize(output_samples, len);
return resultobj;
}
// Module
======================================================================
BOOST_PYTHON_MODULE(spuctest)
{
def("pass_buf",&proc_buf);
}

# python code
....
buffy = mf.read()
print type(buffy)
buf = pass_buf(buffy, len(buffy))
print type(buf)
#

#python output
<type 'buffer'>
<type 'Nonetype'>

I'd guess the return value of None was fortuitous -- when I tried your
code using C, the call to pass_buf just crashed. With the above fix,
the problem goes away:

|>>> import procbuf
|>>> foo = '\x01\x03\xff'
|>>> x = procbuf.passbuf(foo)
|>>> x
'\x02\x06\xfe'

HTH take2
John