multiprocessing on freebsd


T

Tim Arnold

Hi,
I'm checking to see if multiprocessing works on freebsd for any
version of python. My server is about to get upgraded from 6.3 to 8.0
and I'd sure like to be able to use multiprocessing.

I think the minimal test would be:
---------------------
import multiprocessing
q = multiprocessing.Queue()
---------------------

with 6.3, I get

File "/usr/local/lib/python2.6/multiprocessing/__init__.py", line
212, in Queue
from multiprocessing.queues import Queue
File "/usr/local/lib/python2.6/multiprocessing/queues.py", line 22,
in <module>
from multiprocessing.synchronize import Lock, BoundedSemaphore,
Semaphore, Condition
File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line
33, in <module>
" function, see issue 3770.")
ImportError: This platform lacks a functioning sem_open
implementation, therefore, the required synchronization primitives
needed will not function, see issue 3770.


thanks for any info,
--Tim Arnold
 
Ad

Advertisements

P

Philip Semanchuk

Hi,
I'm checking to see if multiprocessing works on freebsd for any
version of python. My server is about to get upgraded from 6.3 to 8.0
and I'd sure like to be able to use multiprocessing.

I think the minimal test would be:
---------------------
import multiprocessing
q = multiprocessing.Queue()
---------------------

with 6.3, I get

File "/usr/local/lib/python2.6/multiprocessing/__init__.py", line
212, in Queue
from multiprocessing.queues import Queue
File "/usr/local/lib/python2.6/multiprocessing/queues.py", line 22,
in <module>
from multiprocessing.synchronize import Lock, BoundedSemaphore,
Semaphore, Condition
File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line
33, in <module>
" function, see issue 3770.")
ImportError: This platform lacks a functioning sem_open
implementation, therefore, the required synchronization primitives
needed will not function, see issue 3770.

Hi Tim,
Under FreeBSD 8/Python 2.6.2 I get the same result, unfortunately.
That's a pity because sem_open works under FreeBSD >= 7.2 as we
discussed before.


Issue 3770 is closed with the note, "we've removed hard-coded platform
variables for a better autoconf approach." I'm using the Python built
from FreeBSD's ports, and the note makes me think that it's possible
that if I built my own Python from the Python.org tarball rather than
ports the problem would go away due to autoconf magic. I don't have
the time to offer to do this for you, unfortunately. But why not
install FreeBSD 8 under VirtualBox or somesuch and give it a go
yourself?

A couple of quirks I noted related to FreeBSD & POSIX IPC that you
might find useful --
- The sem and mqueuefs kernel modules must be loaded, otherwise you'll
get a message like this when you try to create a semaphore or message
queue:
Bad system call: 12 (core dumped)

Under 8.0 they're loaded by default, I think.

- C apps that want to use message queues must link to the realtime
libs (pass -lrt to the linker). This tripped me up for a while.
Linking to the realtime libs is required for all POSIX IPC calls under
Linux; FreeBSD does not require it for semaphores or shared mem, only
message queues.


Hope this helps
Philip
 
T

Tim Arnold

Hi Tim,
Under FreeBSD 8/Python 2.6.2 I get the same result, unfortunately.  
That's a pity because sem_open works under FreeBSD >= 7.2 as we  
discussed before.

Issue 3770 is closed with the note, "we've removed hard-coded platform  
variables for a better autoconf approach." I'm using the Python built  
from FreeBSD's ports, and the note makes me think that it's possible  
that if I built my own Python from the Python.org tarball rather than  
ports the problem would go away due to autoconf magic. I don't have  
the time to offer to do this for you, unfortunately. But why not  
install FreeBSD 8 under VirtualBox or somesuch and give it a go  
yourself?

A couple of quirks I noted related to FreeBSD & POSIX IPC that you  
might find useful --
- The sem and mqueuefs kernel modules must be loaded, otherwise you'll  
get a message like this when you try to create a semaphore or message  
queue:
Bad system call: 12 (core dumped)

Under 8.0 they're loaded by default, I think.

- C apps that want to use message queues must link to the realtime  
libs (pass -lrt to the linker). This tripped me up for a while.  
Linking to the realtime libs is required for all POSIX IPC calls under  
Linux; FreeBSD does not require it for semaphores or shared mem, only  
message queues.

Hope this helps
Philip

Hi Philip,
Thanks for that information (esp the linker info). Once the machine is
upgraded, I'll try building python from the tarball. I'll post back
here with the results.

here's hoping!
thanks,
--Tim
 
M

Martin P. Hellwig

Hi,
I'm checking to see if multiprocessing works on freebsd for any
version of python. My server is about to get upgraded from 6.3 to 8.0
and I'd sure like to be able to use multiprocessing.

I think the minimal test would be:
---------------------
import multiprocessing
q = multiprocessing.Queue()
---------------------

with 6.3, I get

File "/usr/local/lib/python2.6/multiprocessing/__init__.py", line
212, in Queue
from multiprocessing.queues import Queue
File "/usr/local/lib/python2.6/multiprocessing/queues.py", line 22,
in<module>
from multiprocessing.synchronize import Lock, BoundedSemaphore,
Semaphore, Condition
File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line
33, in<module>
" function, see issue 3770.")
ImportError: This platform lacks a functioning sem_open
implementation, therefore, the required synchronization primitives
needed will not function, see issue 3770.

Build mine from ports, with the following options (notice SEM & PTH):
[[email protected] /usr/home/martin]$ cat /var/db/ports/python26/options
# This file is auto-generated by 'make config'.
# No user-servicable parts inside!
# Options for python26-2.6.4
_OPTIONS_READ=python26-2.6.4
WITH_THREADS=true
WITHOUT_HUGE_STACK_SIZE=true
WITH_SEM=true
WITH_PTH=true
WITH_UCS4=true
WITH_PYMALLOC=true
WITH_IPV6=true
WITHOUT_FPECTL=true

[[email protected] /usr/home/martin]$ uname -a
FreeBSD aspire8930 8.0-STABLE FreeBSD 8.0-STABLE #3: Wed Feb 3 17:01:18
GMT 2010 [email protected]:/usr/obj/usr/src/sys/ASPIRE8930 amd64
[[email protected] /usr/home/martin]$ python
Python 2.6.4 (r264:75706, Mar 17 2010, 18:44:24)
[GCC 4.2.1 20070719 [FreeBSD]] on freebsd8
Type "help", "copyright", "credits" or "license" for more information.
hth
 
Ad

Advertisements

T

Tim Arnold

Martin P. Hellwig said:
Hi,
I'm checking to see if multiprocessing works on freebsd for any
version of python. My server is about to get upgraded from 6.3 to 8.0
and I'd sure like to be able to use multiprocessing.

I think the minimal test would be:
---------------------
import multiprocessing
q = multiprocessing.Queue()
---------------------

with 6.3, I get

File "/usr/local/lib/python2.6/multiprocessing/__init__.py", line
212, in Queue
from multiprocessing.queues import Queue
File "/usr/local/lib/python2.6/multiprocessing/queues.py", line 22,
in<module>
from multiprocessing.synchronize import Lock, BoundedSemaphore,
Semaphore, Condition
File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line
33, in<module>
" function, see issue 3770.")
ImportError: This platform lacks a functioning sem_open
implementation, therefore, the required synchronization primitives
needed will not function, see issue 3770.

Build mine from ports, with the following options (notice SEM & PTH):
[[email protected] /usr/home/martin]$ cat /var/db/ports/python26/options
# This file is auto-generated by 'make config'.
# No user-servicable parts inside!
# Options for python26-2.6.4
_OPTIONS_READ=python26-2.6.4
WITH_THREADS=true
WITHOUT_HUGE_STACK_SIZE=true
WITH_SEM=true
WITH_PTH=true
WITH_UCS4=true
WITH_PYMALLOC=true
WITH_IPV6=true
WITHOUT_FPECTL=true

[[email protected] /usr/home/martin]$ uname -a
FreeBSD aspire8930 8.0-STABLE FreeBSD 8.0-STABLE #3: Wed Feb 3 17:01:18
GMT 2010 [email protected]:/usr/obj/usr/src/sys/ASPIRE8930 amd64
[[email protected] /usr/home/martin]$ python
Python 2.6.4 (r264:75706, Mar 17 2010, 18:44:24)
[GCC 4.2.1 20070719 [FreeBSD]] on freebsd8
Type "help", "copyright", "credits" or "license" for more information.
hth

Hi Martin, thanks very much for posting that. All I can say is YAY! I'm
really looking forward to my machine's upgrade now!

thanks,
--Tim
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top