Question to python C API

A

Andreas Otto

Hi,

I have the following question ...

I write a custom "*.init" method and expect a variable number or arguments

This are my questions:

1. I need something like a for loop to analyse this arguments
For now I try the "PyArg_ParseTupleAndKeywords" but i missing somehing
like an Object-Array return value as "format" ....

2. can i combine variable args with keywords ?
because it will fit into my spec to use both together

3. I want to retrieve a bool object as int value, where is a format "O"
and "O!" for a type object .... HOWTO put the type into the function from
above


mfg

Andreas Otto
 
S

Stefan Behnel

Andreas said:
I have the following question ...

I write a custom "*.init" method and expect a variable number or arguments

What's a "*.init" method? Do you mean SomeType.__init__() ?

This are my questions:

1. I need something like a for loop to analyse this arguments
For now I try the "PyArg_ParseTupleAndKeywords" but i missing somehing
like an Object-Array return value as "format" ....

2. can i combine variable args with keywords ?
because it will fit into my spec to use both together

3. I want to retrieve a bool object as int value, where is a format "O"
and "O!" for a type object .... HOWTO put the type into the function from
above

You might want to take a look at Cython. It allows you to write C
extensions in a Python-like language, so all of the above become trivial, e.g.

cdef class MyExtensionType(object):
def __init__(self, bint some_c_bool):
print type(self) is MyExtensionType
print some_c_bool == 1

Cython will translate this to efficient C code for you.

http://cython.org/

Stefan
 
A

Andreas Otto

Yes, you are right ...

I read more and found the doc about this ...

the problem I have is something more tricky ...

I allready have an extension written for java
and the easyest thing would be use this as template
and replace the java specific calls with python calls ...

but the python C API is much more complex than java JNI.

I don't understand why because the task is allways the
same

the problem is that i have a C api but don't want to
use a 1 to 1 translation of the C functions to python.

java generate the stub with java.h and you have to fill
the stub with code even if it is just a function call
of a C api function with near the same name.

for me as user I would prefer the exact same way in python
to: 1. write a python wrapper class
2. declare all external function with "native"
3. call python_h to create the necessary C header
4. and let the user bill the body
5. this require to have a clean and small language
native interface (JNI fit on one web page
that's it)

but i have helper functions which , for example, create
threads and python module have to attach to this thread.

the problem with such kind of framework is usually
that you start with the easy stuff and than (after a couple
of days/weeks) you come to the difficult stuff and you
have to figure out that this kind of problem does not
fit into the tool.

stuff what I do is:

1. create objects on the fly as connection handle
2. do callbacks from C to Python
3. create and delete threads or manage to continue
work after an fork
4. is server mode start an event-loop and never come
back


what I want:

I want to use this tool but still be able to intermix
parts of my "C" helper code with the python code

question:

it is possible to write C and python code into the
same file ?
 
D

Diez B. Roggisch

it is possible to write C and python code into the
same file ?

Not as such.

And JNI is an atrocity, btw.

But what you can do (if you have a pure C-API, no C++) is to completely
ditch the C from the equation and go for ctypes. This allows you to
easily wrap the C-functions in pure python, and then write a simple
layour for OO-goodness around it.

Callbacks from C to python work without a hitch, and generally it's
really cool to work with ctypes.

You can also try & use Cython, a Python-like language that makes
bridging between C and Python easier. I personally don't have any
experience with it (only it's predecessor Pyrex, which worked well for
me) - but to be honest: as long as you aren't in there for speed (Cython
can make things faster, if you restrict yourself to the subset the
language supports), ctypes is the easiest thing to go for. No compiler,
no hassle.

Diez
 
S

Stefan Behnel

Andreas said:
I want to make a language binding for an existing C library

http://libmsgque.sourceforge.net

is this possible ?

Quoting the third paragraph on Cython's homepage (i.e. the link I posted):

"""
This makes Cython the ideal language for wrapping external C libraries, and
for fast C modules that speed up the execution of Python code.
"""

If you want a fast wrapper module that is easy to write and maintain, while
having all the freedom to design the Python-level API the way you want, I
don't think there is any better way than Cython.

Stefan
 
S

Stefan Behnel

Andreas said:
the problem with such kind of framework is usually
that you start with the easy stuff and than (after a couple
of days/weeks) you come to the difficult stuff and you
have to figure out that this kind of problem does not
fit into the tool.

That is a very common problem with so-called "wrapper generators", such as
SWIG, PyBindGen, sip, and some others. They get you to a 1:1 wrapper
quickly, but as soon as you try to abstract your wrapper from the
underlying C API, you start reaching the limits of the tools almost as
quickly (or at least have a hard time pushing the tool the way you want).

stuff what I do is:

1. create objects on the fly as connection handle
2. do callbacks from C to Python
3. create and delete threads or manage to continue
work after an fork
4. is server mode start an event-loop and never come
back

Not uncommon for a C library wrapper at all. I assume that the "server
mode" event-loop calls back into Python from time to time? You will sooo
love the "with gil" function feature in Cython...

what I want:

I want to use this tool but still be able to intermix
parts of my "C" helper code with the python code

question:

it is possible to write C and python code into the
same file ?

I'm glad you asked :)

That's basically what the Cython language is all about. Think of it as a
programming language that is almost Python, but at the same time allows you
to work with C data types and do direct calls into C code. All of that gets
translated into very fast C code that you can easily hand-tune to your
needs. You basically have all the freedom of the Python language with all
the freedom of the C language.

Stefan
 
A

Andreas Otto

Hi,

just my first step in Cython

1. download Cython-0.11.1

2. read INSTALL.txt
(1) Run the setup.py script in this directory
as follows:

python setup.py install

This will install the Pyrex package
into your Python system.
<<<<<<<<<<<<<

Question 1: Why you wall it "Pyrex" package ?

3. this happen
python ./setup.py install
Traceback (most recent call last):
File "./setup.py", line 5, in <module>
from Cython.Compiler.Version import version
File "/home/dev1usr/src/Cython-0.11.1/Cython/__init__.py", line 2, in
<module>
from Shadow import *
ImportError: No module named Shadow

4. I use a thread enabled python V3
<<<<<<<<<<<<<

Question 2: What I have to do ?

4. than I try the second part from INSTALL.txt
(2) If you prefer not to modify your Python
installation, arrange for the directory
containing this file (INSTALL.txt) to be in
your PYTHONPATH. On unix, also put the bin
directory on your PATH.
<<<<<<<<<<<<<

5. and start to build the hello world example

I changed: print "Hello World"
to: print("Hello World") -> this is V3

6. but got the following error
cat setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

setup(
cmdclass = {'build_ext': build_ext},
ext_modules = [Extension("pymsgque", ["helloworld.pyx"])]
)
python setup.py build_ext --inplace
Traceback (most recent call last):
File "setup.py", line 3, in <module>
from Cython.Distutils import build_ext
ImportError: No module named Cython.Distutils

7. a little bit to much errors for the first step, isn't it ?


mfg

Andreas Otto
 
A

alex23

        Question 1: Why you wall it "Pyrex" package ?

From the first paragraph on the Cython site: "Cython is based on the
well-known Pyrex, but supports more cutting edge functionality and
optimizations."
Traceback (most recent call last):
  File "./setup.py", line 5, in <module>
    from Cython.Compiler.Version import version
  File "/home/dev1usr/src/Cython-0.11.1/Cython/__init__.py", line 2, in
<module>
    from Shadow import *
ImportError: No module named Shadow

Did you unpack the Cython archive correctly? Is there a Shadow.py in
your src/Cython-0.11.1/Cython/ folder?
  4. than I try the second part from INSTALL.txt
(2) If you prefer not to modify your Python
    installation, arrange for the directory
    containing this file (INSTALL.txt) to be in
    your PYTHONPATH. On unix, also put the bin
    directory on your PATH.

Traceback (most recent call last):
  File "setup.py", line 3, in <module>
    from Cython.Distutils import build_ext
ImportError: No module named Cython.Distutils

Did you follow the 2nd set of instructions & modify both your
PYTHONPATH & PATH env vars? Have you confirmed they're correct? This
is the sort of error I'd expect if those weren't set up properly.
  7. a little bit to much errors for the first step, isn't it ?

Yes, however none of these happened when I just installed Cython to
the XP, OSX & Ubuntu boxen I'm working on, so this isn't usual
behaviour.

What is your setup? What OS?
 
A

Andreas Otto

alex23 said:
From the first paragraph on the Cython site: "Cython is based on the
well-known Pyrex, but supports more cutting edge functionality and
optimizations."


Did you unpack the Cython archive correctly? Is there a Shadow.py in
your src/Cython-0.11.1/Cython/ folder?
yes


Did you follow the 2nd set of instructions & modify both your
PYTHONPATH & PATH env vars? Have you confirmed they're correct? This
is the sort of error I'd expect if those weren't set up properly.

I put the the "Cython-0.11.1" into the PATH and don't modify the
PYTHONPATH

the INSTALL.txt say: On unix, *also* put the bin directory on your PATH

but I think I found the reason I have to set the PYTHONPATH to this
directory *and* setup the PATH. I understand *also* as something like "or"

but now the following error happen:

PYTHONPATH=~/ext/x86_64-suse-linux/thread/bin/Cython-0.11.1/
PATH=$PATH:~/ext/x86_64-suse-linux/thread/bin/Cython-0.11.1/
python ./setup.py build_ext --inplace
Traceback (most recent call last):
File "./setup.py", line 3, in <module>
from Cython.Distutils import build_ext

File "/home/dev1usr/ext/x86_64-suse-linux/thread/bin/Cython-0.11.1/Cython/__init__.py",
line 2, in <module>
from Shadow import *
ImportError: No module named Shadow

Yes, however none of these happened when I just installed Cython to
the XP, OSX & Ubuntu boxen I'm working on, so this isn't usual
behaviour.

What is your setup? What OS?

SuSE 10.3, I have compiled python on my own and install it in:
~/ext/x86_64-suse-linux/thread
 
A

Andreas Otto

Andreas said:

dev1usr@linux02:~/ext/x86_64-suse-linux/thread/bin/Cython-0.11.1> ls -al
Cython/Shadow.py
-rw-r--r-- 1 dev1usr users 4130 3. Apr 10:52 Cython/Shadow.py


mfg

Andreas Otto
 
S

Stefan Behnel

Andreas said:
just my first step in Cython

1. download Cython-0.11.1

2. read INSTALL.txt

(1) Run the setup.py script in this directory
as follows:

python setup.py install

This will install the Pyrex package
into your Python system.
<<<<<<<<<<<<<

Question 1: Why you wall it "Pyrex" package ?

That's a left-over. We fixed the name almost everywhere by now, but that
slipped through.

That shows that the installation instructions haven't been touched in years
(although they should work in general).

3. this happen

Traceback (most recent call last):
File "./setup.py", line 5, in <module>
from Cython.Compiler.Version import version
File "/home/dev1usr/src/Cython-0.11.1/Cython/__init__.py", line 2, in
<module>
from Shadow import *
ImportError: No module named Shadow

Relative imports have changed in Py3, so as you write:
4. I use a thread enabled python V3

That's the core problem. :) Like many other Python packages, Cython doesn't
currently run in Py3. You can run it in any Py2 version starting from 2.3,
and the code it generates will run on 2.3 through 3.1a1. But the Cython
compiler itself requires 2.x. This is being fixed up as we speak, and the
2to3 conversion tool works pretty well on the source by now, so there's a
fair chance that Cython 0.12 will run on Py3.

Stefan
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,829
Messages
2,569,737
Members
45,521
Latest member
maryb

Latest Threads

Top