Porting Python2 C-API/Swig based modules to Python 3

[Adam Pridgen]

Hello,

I am trying to get a compiled module to work with Python3. The code I
am compiling was originally intended to be used in a Python 2.*
environment, but I updated all the Python 2.* elements and the swig
commands used by the setup.py script. I got the library to
successfully compile, but I am getting the following errors on import
(below). I am not sure how to trouble shoot this problem, and the
fact that only one symbol (_PyCObject_FromVoidPtr) is missing is
disconcerting. I Googled some, but the symbol mentioned only showed
up in a few posts, where linking was an issue. I have also gone
through the setup.py script and explicitly defined all the library
paths.

My questions:

- Has anyone ever ported Python 2* modules/libs to Python 3 that rely
on swig, and are there some changes in the C-API/swig I need to be
looking for to make this port successful?

- Does anyone have any advice/insght about how I can troubleshoot,
diagnose, and resolve this issue?

Thanks in advance,

-- Adam


<error message>

0ylibpcap-0.6.2$ python3
Python 3.2 (r32:88452, Feb 20 2011, 11:12:31)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pcap.py",
line 25, in <module>
_pcap = swig_import_helper()
File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pcap.py",
line 21, in swig_import_helper
_mod = imp.load_module('_pcap', fp, pathname, description)
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/_pcapmodule.so,
2): Symbol not found: _PyCObject_FromVoidPtr
Referenced from:
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/_pcapmodule.so
Expected in: flat namespace
in /Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/_pcapmodule.so

 

[casevh]

Hello,

I am trying to get a compiled module to work with Python3.  The code I
am compiling was originally intended to be used in a Python 2.*
environment, but I updated all the Python 2.* elements and the swig
commands used by the setup.py script.  I got the library to
successfully compile, but I am getting the following errors on import
(below).  I am not sure how to trouble shoot this problem, and the
fact that only one symbol (_PyCObject_FromVoidPtr) is missing is
disconcerting.  I Googled some, but the symbol mentioned only showed
up in a few posts, where linking was an issue.  I have also gone
through the setup.py script and explicitly defined all the library
paths.

My questions:

- Has anyone ever ported Python 2* modules/libs to Python 3 that rely
on swig, and are there some changes in the C-API/swig I need to be
looking for to make this port successful?

- Does anyone have any advice/insght about how I can troubleshoot,
diagnose, and resolve this issue?

Thanks in advance,

-- Adam

<error message>

0ylibpcap-0.6.2$ python3
Python 3.2 (r32:88452, Feb 20 2011, 11:12:31)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.>>>import pcap,py

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packa­ges/pcap.py",
line 25, in <module>
    _pcap = swig_import_helper()
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packa­ges/pcap.py",
line 21, in swig_import_helper
    _mod = imp.load_module('_pcap', fp, pathname, description)
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site­-packages/_pcapmodule.so,
2): Symbol not found: _PyCObject_FromVoidPtr
  Referenced from:
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packag­es/_pcapmodule.so
  Expected in: flat namespace
 in /Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packag­es/_pcapmodule.so



- Show quoted text -

This is a change in the C-API in 3.2. See http://bugs.python.org/issue5630

casevh

 

[Adam Pridgen]

Thanks for the help. That information got me started in the right
direction. I put my notes up on paste bin for others to use in the
future.

For the future reference of others, I have included the pertinent
details below. My process for identifying the various issues was
compiling and running the project to see what was still broken after
each fix. I know this is not the most efficient way, but it worked
for this pylibcap since the code base was pretty small. I don't think
everything is completely fixed, but its a step in the right direction.

The initial hurdle I encountered was swig. I had to modify swig's
code output, because it was not using the PyCapsule_* API yet. I
found someones solution here: http://paste.pocoo.org/show/253494/.
After applying that fix, I found that the pcap_interface.c and pcap.i
files in the project were using some deprecated API calls. So I
converted the following APIs:

PyLong_AS_INT ==> PyLong_AsLong
PyInt_* ==> PyLong_*
PyString_* ==> PyBytes_* (initialily changed to PyUnicode_* but that
was wrong, because I needed the Raw Bytes)

I also had to change s# to y#. Before, s# stood for string and #, but
it now means unicode and number.



Hopefully these details will be helpful to others in the future.
Thanks again for the nudge,

-- Adam.

Hello,

I am trying to get a compiled module to work with Python3.  The code I
am compiling was originally intended to be used in a Python 2.*
environment, but I updated all the Python 2.* elements and the swig
commands used by the setup.py script.  I got the library to
successfully compile, but I am getting the following errors on import
(below).  I am not sure how to trouble shoot this problem, and the
fact that only one symbol (_PyCObject_FromVoidPtr) is missing is
disconcerting.  I Googled some, but the symbol mentioned only showed
up in a few posts, where linking was an issue.  I have also gone
through the setup.py script and explicitly defined all the library
paths.

My questions:

- Has anyone ever ported Python 2* modules/libs to Python 3 that rely
on swig, and are there some changes in the C-API/swig I need to be
looking for to make this port successful?

- Does anyone have any advice/insght about how I can troubleshoot,
diagnose, and resolve this issue?

Thanks in advance,

-- Adam

<error message>

0ylibpcap-0.6.2$ python3
Python 3.2 (r32:88452, Feb 20 2011, 11:12:31)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.>>> import pcap,py

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packa­ges/pcap.py",
line 25, in <module>
    _pcap = swig_import_helper()
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packa­ges/pcap.py",
line 21, in swig_import_helper
    _mod = imp.load_module('_pcap', fp, pathname, description)
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site­-packages/_pcapmodule.so,
2): Symbol not found: _PyCObject_FromVoidPtr
  Referenced from:
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packag­es/_pcapmodule.so
  Expected in: flat namespace
 in /Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packag­es/_pcapmodule.so

^D- Hide quoted text -

- Show quoted text -

This is a change in the C-API in 3.2. See http://bugs.python.org/issue5630

casevh

 

[Adam Pridgen]

Link to pastebin: http://pastebin.com/102fhkgp

Thanks for the help.  That information got me started in the right
direction.  I put my notes up on paste bin for others to use in the
future.

For the future reference of others, I have included the pertinent
details below.  My process for identifying the various issues was
compiling and running the project to see what was still broken after
each fix.  I know this is not the most efficient way, but it worked
for this pylibcap since the code base was pretty small.  I don't think
everything is completely fixed, but its a step in the right direction.

The initial hurdle I encountered was swig.  I had to modify swig's
code output, because it was not using the PyCapsule_* API yet.  I
found someones solution here: http://paste.pocoo.org/show/253494/.
After applying that fix, I found that the pcap_interface.c and pcap.i
files in the project were using some deprecated API calls.  So I
converted the following APIs:

PyLong_AS_INT ==> PyLong_AsLong
PyInt_* ==> PyLong_*
PyString_* ==>  PyBytes_* (initialily changed to PyUnicode_* but that
was wrong, because I needed the Raw Bytes)

I also had to change s# to y#.  Before, s# stood for string and #, but
it now means unicode and number.



Hopefully these details will be helpful to others in the future.
Thanks again for the nudge,

-- Adam.

Hello,

I am trying to get a compiled module to work with Python3.  The code I
am compiling was originally intended to be used in a Python 2.*
environment, but I updated all the Python 2.* elements and the swig
commands used by the setup.py script.  I got the library to
successfully compile, but I am getting the following errors on import
(below).  I am not sure how to trouble shoot this problem, and the
fact that only one symbol (_PyCObject_FromVoidPtr) is missing is
disconcerting.  I Googled some, but the symbol mentioned only showed
up in a few posts, where linking was an issue.  I have also gone
through the setup.py script and explicitly defined all the library
paths.

My questions:

- Has anyone ever ported Python 2* modules/libs to Python 3 that rely
on swig, and are there some changes in the C-API/swig I need to be
looking for to make this port successful?

- Does anyone have any advice/insght about how I can troubleshoot,
diagnose, and resolve this issue?

Thanks in advance,

-- Adam

<error message>

0ylibpcap-0.6.2$ python3
Python 3.2 (r32:88452, Feb 20 2011, 11:12:31)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.>>> import pcap,py

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3..2/site-packa­ges/pcap.py",
line 25, in <module>
    _pcap = swig_import_helper()
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3..2/site-packa­ges/pcap.py",
line 21, in swig_import_helper
    _mod = imp.load_module('_pcap', fp, pathname, description)
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site­-packages/_pcapmodule.so,
2): Symbol not found: _PyCObject_FromVoidPtr
  Referenced from:
/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packag­es/_pcapmodule.so
  Expected in: flat namespace
 in /Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packag­es/_pcapmodule.so



^D- Hide quoted text -

- Show quoted text -

This is a change in the C-API in 3.2. See http://bugs.python.org/issue5630

casevh