multiprocessing managers and socket connection.

[Chris]

I've been using multiprocessing managers and I really like the
functionality.

I have a question about reconnecting to a manager. I have a situation
where I start on one machine (A) a manager that is listening and then
on another machine (B) connects to that manager and uses its proxy
object to call functions on the manager's computer; this all works as
expected. But, if the manager from A shuts down, B's application won't
notice because in the MP code it ignores socket error
errno.ECONNREFUSED. If A becomes available again or is restarted, B
doesn't automatically reconnect either and continue its operation.
It's function is basically stopped.

Here is the code from connection.py:
while 1:
try:
s.connect(address)
except socket.error, e:
if e.args[0] != errno.ECONNREFUSED: # connection refused
debug('failed to connect to address %s', address)
raise
time.sleep(0.01)
else:
break

How can I have B automatically reconnect to A and continue its work
once A is available again?

 

[Terry]

I've been using multiprocessing managers and I really like the
functionality.

I have a question about reconnecting to a manager. I have a situation
where I start on one machine (A) a manager that is listening and then
on another machine (B) connects to that manager and uses its proxy
object to call functions on the manager's computer; this all works as
expected. But, if the manager from A shuts down, B's application won't
notice because in the MP code it ignores socket error
errno.ECONNREFUSED. If A becomes available again or is restarted, B
doesn't automatically reconnect either and continue its operation.
It's function is basically stopped.

Here is the code from connection.py:
while 1:
        try:
            s.connect(address)
        except socket.error, e:
            if e.args[0] != errno.ECONNREFUSED: # connection refused
                debug('failed to connect to address %s', address)
                raise
            time.sleep(0.01)
        else:
            break

How can I have B automatically reconnect to A and continue its work
once A is available again?

I think you need to retry repeatedly until successfully connected.

br, Terry

 

[Chris]

I've been using multiprocessing managers and I really like the
functionality.

I have a question about reconnecting to a manager. I have a situation
where I start on one machine (A) a manager that is listening and then
on another machine (B) connects to that manager and uses its proxy
object to call functions on the manager's computer; this all works as
expected. But, if the manager from A shuts down, B's application won't
notice because in the MP code it ignores socket error
errno.ECONNREFUSED. If A becomes available again or is restarted, B
doesn't automatically reconnect either and continue its operation.
It's function is basically stopped.

Here is the code from connection.py:
while 1:
        try:
            s.connect(address)
        except socket.error, e:
            if e.args[0] != errno.ECONNREFUSED: # connection refused
                debug('failed to connect to address %s', address)
                raise
            time.sleep(0.01)
        else:
            break

How can I have B automatically reconnect to A and continue its work
once A is available again?

I think you need to retry repeatedly until successfully connected.

br, Terry

I'm having issue after once connected. If the server goes down during
a long-running connection. I would want to be notified so I could try
to reconnect. I'm doing more experimenting now and will try to post an
example.

 

[Terry]

I've been using multiprocessing managers and I really like the
functionality.
I have a question about reconnecting to a manager. I have a situation
where I start on one machine (A) a manager that is listening and then
on another machine (B) connects to that manager and uses its proxy
object to call functions on the manager's computer; this all works as
expected. But, if the manager from A shuts down, B's application won't
notice because in the MP code it ignores socket error
errno.ECONNREFUSED. If A becomes available again or is restarted, B
doesn't automatically reconnect either and continue its operation.
It's function is basically stopped.
Here is the code from connection.py:
while 1:
        try:
            s.connect(address)
        except socket.error, e:
            if e.args[0] != errno.ECONNREFUSED: # connection refused
                debug('failed to connect to address %s', address)
                raise
            time.sleep(0.01)
        else:
            break
How can I have B automatically reconnect to A and continue its work
once A is available again?

I think you need to retry repeatedly until successfully connected.

br, Terry

I'm having issue after once connected. If the server goes down during
a long-running connection. I would want to be notified so I could try
to reconnect. I'm doing more experimenting now and will try to post an
example.

Hi Chris,

Are you sure that the proxy object keeps a permanent connection to the
server?

br, Terry

 

[Chris]

I've been using multiprocessing managers and I really like the
functionality.
I have a question about reconnecting to a manager. I have a situation
where I start on one machine (A) a manager that is listening and then
on another machine (B) connects to that manager and uses its proxy
object to call functions on the manager's computer; this all works as
expected. But, if the manager from A shuts down, B's application won't
notice because in the MP code it ignores socket error
errno.ECONNREFUSED. If A becomes available again or is restarted, B
doesn't automatically reconnect either and continue its operation.
It's function is basically stopped.
Here is the code from connection.py:
while 1:
        try:
            s.connect(address)
        except socket.error, e:
            if e.args[0] != errno.ECONNREFUSED: # connection refused
                debug('failed to connect to address %s', address)
                raise
            time.sleep(0.01)
        else:
            break
How can I have B automatically reconnect to A and continue its work
once A is available again?
I think you need to retry repeatedly until successfully connected.
br, Terry

I'm having issue after once connected. If the server goes down during
a long-running connection. I would want to be notified so I could try
to reconnect. I'm doing more experimenting now and will try to post an
example.

Hi Chris,

Are you sure that the proxy object keeps a permanent connection to the
server?

br, Terry

Sorry for the delay in response. I was able to find a solution by NOT
using sockets, but using PIPES instead. The socket connections to the
managers don't disconnect properly.

Here's how to reproduce the situation (yes it's a stupid example, but
that's the point):

mp_manager_test.py --> http://python.pastebin.com/m3d10e343
mp_manager_test_client.py --> http://python.pastebin.com/m7a8fda4c

run the following sequence which requires 3 shells.
shell1> python mp_manager_test.py start_server
shell2> python mp_manager_test_client.py
shell3> python mp_manager_test.py stop_server

Run the 3rd line while the client is accessing the server.
When you use the socket for a connection, it hangs.
When you use a PIPE, it throws the exception and actually exits; so I
can catch the exception and handle it properly instead of having a
hanging program. I am not sure how well a remote pipe will work though
as I have yet to try it over a network between machines.

To switch from socket to pipe, just switch the comments at the top in
the source files:
ADDY = ("127.0.0.1",9000)
ADDY2 = ("127.0.0.1",9001)
#ADDY = r'\\.\pipe\PipeName'
#ADDY2 = r'\\.\pipe\PipeName2'

Chris