Question to python C API

[Andreas Otto]

Hi,

I have the following question ...

I write a custom "*.init" method and expect a variable number or arguments

This are my questions:

1. I need something like a for loop to analyse this arguments
For now I try the "PyArg_ParseTupleAndKeywords" but i missing somehing
like an Object-Array return value as "format" ....

2. can i combine variable args with keywords ?
because it will fit into my spec to use both together

3. I want to retrieve a bool object as int value, where is a format "O"
and "O!" for a type object .... HOWTO put the type into the function from
above


mfg

Andreas Otto

 

[Stefan Behnel]

I have the following question ...

I write a custom "*.init" method and expect a variable number or arguments

What's a "*.init" method? Do you mean SomeType.__init__() ?

This are my questions:

1. I need something like a for loop to analyse this arguments
For now I try the "PyArg_ParseTupleAndKeywords" but i missing somehing
like an Object-Array return value as "format" ....

2. can i combine variable args with keywords ?
because it will fit into my spec to use both together

3. I want to retrieve a bool object as int value, where is a format "O"
and "O!" for a type object .... HOWTO put the type into the function from
above

You might want to take a look at Cython. It allows you to write C
extensions in a Python-like language, so all of the above become trivial, e.g.

cdef class MyExtensionType(object):
def __init__(self, bint some_c_bool):
print type(self) is MyExtensionType
print some_c_bool == 1

Cython will translate this to efficient C code for you.

http://cython.org/

Stefan

 

[Andreas Otto]

Hi,

I want to make a language binding for an existing C library

http://libmsgque.sourceforge.net

is this possible ?

 

[Ben Kaplan]

Hi,

I want to make a language binding for an existing C library

http://libmsgque.sourceforge.net

is this possible ?
--

Not only is itpossible, it's pretty common. All of the major GUI
toolkits do this. Look at www.swig.org if you don't want to write it
all manually.

 

[Andreas Otto]

Yes, you are right ...

I read more and found the doc about this ...

the problem I have is something more tricky ...

I allready have an extension written for java
and the easyest thing would be use this as template
and replace the java specific calls with python calls ...

but the python C API is much more complex than java JNI.

I don't understand why because the task is allways the
same

the problem is that i have a C api but don't want to
use a 1 to 1 translation of the C functions to python.

java generate the stub with java.h and you have to fill
the stub with code even if it is just a function call
of a C api function with near the same name.

for me as user I would prefer the exact same way in python
to: 1. write a python wrapper class
2. declare all external function with "native"
3. call python_h to create the necessary C header
4. and let the user bill the body
5. this require to have a clean and small language
native interface (JNI fit on one web page
that's it)

but i have helper functions which , for example, create
threads and python module have to attach to this thread.

the problem with such kind of framework is usually
that you start with the easy stuff and than (after a couple
of days/weeks) you come to the difficult stuff and you
have to figure out that this kind of problem does not
fit into the tool.

stuff what I do is:

1. create objects on the fly as connection handle
2. do callbacks from C to Python
3. create and delete threads or manage to continue
work after an fork
4. is server mode start an event-loop and never come
back


what I want:

I want to use this tool but still be able to intermix
parts of my "C" helper code with the python code

question:

it is possible to write C and python code into the
same file ?

 

[Diez B. Roggisch]

it is possible to write C and python code into the
same file ?

Not as such.

And JNI is an atrocity, btw.

But what you can do (if you have a pure C-API, no C++) is to completely
ditch the C from the equation and go for ctypes. This allows you to
easily wrap the C-functions in pure python, and then write a simple
layour for OO-goodness around it.

Callbacks from C to python work without a hitch, and generally it's
really cool to work with ctypes.

You can also try & use Cython, a Python-like language that makes
bridging between C and Python easier. I personally don't have any
experience with it (only it's predecessor Pyrex, which worked well for
me) - but to be honest: as long as you aren't in there for speed (Cython
can make things faster, if you restrict yourself to the subset the
language supports), ctypes is the easiest thing to go for. No compiler,
no hassle.

Diez

 

[Stefan Behnel]

I want to make a language binding for an existing C library

http://libmsgque.sourceforge.net

is this possible ?

Quoting the third paragraph on Cython's homepage (i.e. the link I posted):

"""
This makes Cython the ideal language for wrapping external C libraries, and
for fast C modules that speed up the execution of Python code.
"""

If you want a fast wrapper module that is easy to write and maintain, while
having all the freedom to design the Python-level API the way you want, I
don't think there is any better way than Cython.

Stefan

 

[Stefan Behnel]

the problem with such kind of framework is usually
that you start with the easy stuff and than (after a couple
of days/weeks) you come to the difficult stuff and you
have to figure out that this kind of problem does not
fit into the tool.

That is a very common problem with so-called "wrapper generators", such as
SWIG, PyBindGen, sip, and some others. They get you to a 1:1 wrapper
quickly, but as soon as you try to abstract your wrapper from the
underlying C API, you start reaching the limits of the tools almost as
quickly (or at least have a hard time pushing the tool the way you want).

stuff what I do is:

1. create objects on the fly as connection handle
2. do callbacks from C to Python
3. create and delete threads or manage to continue
work after an fork
4. is server mode start an event-loop and never come
back

Not uncommon for a C library wrapper at all. I assume that the "server
mode" event-loop calls back into Python from time to time? You will sooo
love the "with gil" function feature in Cython...

what I want:

I want to use this tool but still be able to intermix
parts of my "C" helper code with the python code

question:

it is possible to write C and python code into the
same file ?

I'm glad you asked

That's basically what the Cython language is all about. Think of it as a
programming language that is almost Python, but at the same time allows you
to work with C data types and do direct calls into C code. All of that gets
translated into very fast C code that you can easily hand-tune to your
needs. You basically have all the freedom of the Python language with all
the freedom of the C language.

Stefan

 

[Andreas Otto]

Hi,

just my first step in Cython

1. download Cython-0.11.1

2. read INSTALL.txt
(1) Run the setup.py script in this directory
as follows:

python setup.py install

This will install the Pyrex package
into your Python system.
<<<<<<<<<<<<<

Question 1: Why you wall it "Pyrex" package ?

3. this happen

python ./setup.py install

Traceback (most recent call last):
File "./setup.py", line 5, in <module>
from Cython.Compiler.Version import version
File "/home/dev1usr/src/Cython-0.11.1/Cython/__init__.py", line 2, in
<module>
from Shadow import *
ImportError: No module named Shadow

4. I use a thread enabled python V3
<<<<<<<<<<<<<

Question 2: What I have to do ?

4. than I try the second part from INSTALL.txt
(2) If you prefer not to modify your Python
installation, arrange for the directory
containing this file (INSTALL.txt) to be in
your PYTHONPATH. On unix, also put the bin
directory on your PATH.
<<<<<<<<<<<<<

5. and start to build the hello world example

I changed: print "Hello World"
to: print("Hello World") -> this is V3

6. but got the following error

cat setup.py

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

setup(
cmdclass = {'build_ext': build_ext},
ext_modules = [Extension("pymsgque", ["helloworld.pyx"])]
)

python setup.py build_ext --inplace

Traceback (most recent call last):
File "setup.py", line 3, in <module>
from Cython.Distutils import build_ext
ImportError: No module named Cython.Distutils

7. a little bit to much errors for the first step, isn't it ?


mfg

Andreas Otto

 

[alex23]

        Question 1: Why you wall it "Pyrex" package ?

From the first paragraph on the Cython site: "Cython is based on the
well-known Pyrex, but supports more cutting edge functionality and
optimizations."

Traceback (most recent call last):
  File "./setup.py", line 5, in <module>
    from Cython.Compiler.Version import version
  File "/home/dev1usr/src/Cython-0.11.1/Cython/__init__.py", line 2, in
<module>
    from Shadow import *
ImportError: No module named Shadow

Did you unpack the Cython archive correctly? Is there a Shadow.py in
your src/Cython-0.11.1/Cython/ folder?

  4. than I try the second part from INSTALL.txt
(2) If you prefer not to modify your Python
    installation, arrange for the directory
    containing this file (INSTALL.txt) to be in
    your PYTHONPATH. On unix, also put the bin
    directory on your PATH.

Traceback (most recent call last):
  File "setup.py", line 3, in <module>
    from Cython.Distutils import build_ext
ImportError: No module named Cython.Distutils

Did you follow the 2nd set of instructions & modify both your
PYTHONPATH & PATH env vars? Have you confirmed they're correct? This
is the sort of error I'd expect if those weren't set up properly.

  7. a little bit to much errors for the first step, isn't it ?

Yes, however none of these happened when I just installed Cython to
the XP, OSX & Ubuntu boxen I'm working on, so this isn't usual
behaviour.

What is your setup? What OS?

 

[Andreas Otto]

From the first paragraph on the Cython site: "Cython is based on the
well-known Pyrex, but supports more cutting edge functionality and
optimizations."


Did you unpack the Cython archive correctly? Is there a Shadow.py in
your src/Cython-0.11.1/Cython/ folder?
yes


Did you follow the 2nd set of instructions & modify both your
PYTHONPATH & PATH env vars? Have you confirmed they're correct? This
is the sort of error I'd expect if those weren't set up properly.

I put the the "Cython-0.11.1" into the PATH and don't modify the
PYTHONPATH

the INSTALL.txt say: On unix, *also* put the bin directory on your PATH

but I think I found the reason I have to set the PYTHONPATH to this
directory *and* setup the PATH. I understand *also* as something like "or"

but now the following error happen:

PYTHONPATH=~/ext/x86_64-suse-linux/thread/bin/Cython-0.11.1/

PATH=$PATH:~/ext/x86_64-suse-linux/thread/bin/Cython-0.11.1/
python ./setup.py build_ext --inplace
Traceback (most recent call last):
File "./setup.py", line 3, in <module>
from Cython.Distutils import build_ext

File "/home/dev1usr/ext/x86_64-suse-linux/thread/bin/Cython-0.11.1/Cython/__init__.py",
line 2, in <module>
from Shadow import *
ImportError: No module named Shadow

Yes, however none of these happened when I just installed Cython to
the XP, OSX & Ubuntu boxen I'm working on, so this isn't usual
behaviour.

What is your setup? What OS?

SuSE 10.3, I have compiled python on my own and install it in:
~/ext/x86_64-suse-linux/thread

 

[Andreas Otto]

yes

dev1usr@linux02:~/ext/x86_64-suse-linux/thread/bin/Cython-0.11.1> ls -al
Cython/Shadow.py
-rw-r--r-- 1 dev1usr users 4130 3. Apr 10:52 Cython/Shadow.py


mfg

Andreas Otto

 

[Stefan Behnel]

just my first step in Cython

1. download Cython-0.11.1

2. read INSTALL.txt

(1) Run the setup.py script in this directory
as follows:

python setup.py install

This will install the Pyrex package
into your Python system.
<<<<<<<<<<<<<

Question 1: Why you wall it "Pyrex" package ?

That's a left-over. We fixed the name almost everywhere by now, but that
slipped through.

That shows that the installation instructions haven't been touched in years
(although they should work in general).

3. this happen

Traceback (most recent call last):
File "./setup.py", line 5, in <module>
from Cython.Compiler.Version import version
File "/home/dev1usr/src/Cython-0.11.1/Cython/__init__.py", line 2, in
<module>
from Shadow import *
ImportError: No module named Shadow

Relative imports have changed in Py3, so as you write:

4. I use a thread enabled python V3

That's the core problem. Like many other Python packages, Cython doesn't
currently run in Py3. You can run it in any Py2 version starting from 2.3,
and the code it generates will run on 2.3 through 3.1a1. But the Cython
compiler itself requires 2.x. This is being fixed up as we speak, and the
2to3 conversion tool works pretty well on the source by now, so there's a
fair chance that Cython 0.12 will run on Py3.

Stefan

 

[Wolfgang Strobl]

5. and start to build the hello world example

I changed: print "Hello World"
to: print("Hello World") -> this is V3

AFAIK Cython doesn't support Python 3, yet. See
http://trac.cython.org/cython_trac/ticket/234

I suggest staying with Python 2.x. What C compiler are you going to
use, btw.?