subprocess.Popen calling httpd reload never finishes

[Nan]

Hi folks --

I have a Python script running under Apache/mod_wsgi that needs to
reload Apache configs as part of its operation. The script continues
to execute after the subprocess.Popen call. The communicate() method
returns the correct text ("Reloading httpd: [ OK ]"), and I get a
returncode of 0. But the python script (Django) that calls Popen
never seems to complete (by returning an HTTP response.

Any other Popen call I've tried exits properly. Here's some sample
code:

args = ['sudo /etc/init.d/httpd reload']
proc = subprocess.Popen(args, stdin=subprocess.PIPE,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True,
close_fds=True)
(stdout_txt, stderr_txt) = proc.communicate("")
proc.wait()
logging.debug('%d %s<hr />%s' % (proc.returncode, stdout_txt,
stderr_txt))
logging.debug('still executing')
return HttpResponse('done')

The logging statements are output, but the script doesn't exit. If
you substitute "sudo ls -l" or "sudo /etc/init.d/httpd configtest" for
"sudo /etc/init.d/httpd reload", the exits properly.

Any idea what I might be doing wrong?

Thanks!

 

[Albert Hopkins]

Hi folks --

I have a Python script running under Apache/mod_wsgi that needs to
reload Apache configs as part of its operation. The script continues
to execute after the subprocess.Popen call. The communicate() method
returns the correct text ("Reloading httpd: [ OK ]"), and I get a
returncode of 0. But the python script (Django) that calls Popen
never seems to complete (by returning an HTTP response.

Any other Popen call I've tried exits properly. Here's some sample
code:

args = ['sudo /etc/init.d/httpd reload']
proc = subprocess.Popen(args, stdin=subprocess.PIPE,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True,
close_fds=True)
(stdout_txt, stderr_txt) = proc.communicate("")
proc.wait()
logging.debug('%d %s<hr />%s' % (proc.returncode, stdout_txt,
stderr_txt))
logging.debug('still executing')
return HttpResponse('done')

The logging statements are output, but the script doesn't exit. If
you substitute "sudo ls -l" or "sudo /etc/init.d/httpd configtest" for
"sudo /etc/init.d/httpd reload", the exits properly.

Any idea what I might be doing wrong?

Thanks!

Django runs inside apache. It's kinda weird to have an apache process
restart itself and expect it to return to the caller.

If the init script does like mine, "reload" executes "apachectl -k
graceful" What that instructs apache to do is to restart, but only
kill the process(es) when there are no more connections. So apache is
waiting for your connection to close, but you are inside an HTTP request
waiting for apache to restart. So you have a race condition here.

It's not advisable to have apache kill itself and expect it to send a
status back to you telling you it's dead.

See the apache docs[1] for a better explanation.


http://httpd.apache.org/docs/2.0/stopping.html#graceful

 

[Nan]

Hi folks --

I have a Python script running under Apache/mod_wsgi that needs to
reload Apache configs as part of its operation.  The script continues
to execute after the subprocess.Popen call.  The communicate() method
returns the correct text ("Reloading httpd: [  OK  ]"), and I get a
returncode of 0.  But the python script (Django) that calls Popen
never seems to complete (by returning an HTTP response.

Any other Popen call I've tried exits properly.  Here's some sample
code:

           args = ['sudo /etc/init.d/httpd reload']
           proc = subprocess.Popen(args, stdin=subprocess.PIPE,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True,
close_fds=True)
           (stdout_txt, stderr_txt) = proc.communicate("")
           proc.wait()
           logging.debug('%d %s<hr />%s' % (proc.returncode, stdout_txt,
stderr_txt))
           logging.debug('still executing')
           return HttpResponse('done')

The logging statements are output, but the script doesn't exit.  If
you substitute "sudo ls -l" or "sudo /etc/init.d/httpd configtest" for
"sudo /etc/init.d/httpd reload", the exits properly.

 Any idea what I might be doing wrong?

Thanks!

Django runs inside apache.  It's kinda weird to have an apache process
restart itself and expect it to return to the caller.

If the init script does like mine, "reload" executes "apachectl -k
graceful"   What that instructs apache to do is to restart, but only
kill the process(es) when there are no more connections.  So apache is
waiting for your connection to close, but you are inside an HTTP request
waiting for apache to restart.  So you have a race condition here.

It's not advisable to have apache kill itself and expect it to send a
status back to you telling you it's dead.

See the apache docs[1] for a better explanation.

http://httpd.apache.org/docs/2.0/stopping.html#graceful

Ah, I'd been told that there would be no conflict, and that this was
just reloading the configuration, not restarting Apache.

I do need the web app to instruct Apache to reload because just before
this it's creating new VirtualHosts that need to be recognized. Is
there a better way to do this (e.g. to say "start doing this once I'm
finished")?

I'm getting a status code and output from the call before the Django
script stops executing... Is there a way to stop waiting for the
process to complete once I have those?

 

[Albert Hopkins]

Ah, I'd been told that there would be no conflict, and that this was
just reloading the configuration, not restarting Apache.

I do need the web app to instruct Apache to reload because just before
this it's creating new VirtualHosts that need to be recognized. Is
there a better way to do this (e.g. to say "start doing this once I'm
finished")?

I'm getting a status code and output from the call before the Django
script stops executing... Is there a way to stop waiting for the
process to complete once I have those?

I have a wireless router with a built in web server. Sometimes it needs
to "reload" it's config. Basically what it's doing is rebooting the
entire router (I can see this if I'm actuall watching the router). All
it is doing, I'm pretty sure, is calling some program that forks another
process and then exits the main program. The forked process then
reboots the router. Meanwhile before that happens the web server sends
a response. Basically in the response it sends an HTTP Refresh with x
number of seconds. Presumably x is longer than the time it requires for
the router to reboot. The router reboots, the browser refreshes and
viola.

You probably need to so something similar in that your request calls a
program that forks off and restarts apaches. It should probably not do
so immediately so that your request has time to send a response with a
refresh header (but that shouldn't take long). After a second or so,
apache will have restarted and the browser will have refreshed.

so (untested):

def reload(request):
subprocess.call(['my_apache_reloader']) # this should fork and exit
response = HttpResponse()
response['Refresh']='3; url=%s' % reverse(home) # chk back in 3 secs
return response

BTW There is a Django mailing list where this might be more appropriate
to discuss.

-a

 

[Nan]

Ah, I'd been told that there would be no conflict, and that this was
just reloading the configuration, not restarting Apache.

I do need the web app to instruct Apache to reload because just before
this it's creating new VirtualHosts that need to be recognized.  Is
there a better way to do this (e.g. to say "start doing this once I'm
finished")?

I'm getting a status code and output from the call before the Django
script stops executing... Is there a way to stop waiting for the
process to complete once I have those?

I have a wireless router with a built in web server.  Sometimes it needs
to "reload" it's config.  Basically what it's doing is rebooting the
entire router (I can see this if I'm actuall watching the router).  All
it is doing, I'm pretty sure, is calling some program that forks another
process and then exits the main program.  The forked process then
reboots the router.  Meanwhile before that happens the web server sends
a response. Basically in the response it sends an HTTP Refresh with x
number of seconds.  Presumably x is longer than the time it requires for
the router to reboot.  The router reboots, the browser refreshes and
viola.

You probably need to so something similar in that your request calls a
program that forks off and restarts apaches.  It should probably not do
so immediately so that your request has time to send a response with a
refresh header (but that shouldn't take long).  After a second or so,
apache will have restarted and the browser will have refreshed.

so (untested):

def reload(request):
    subprocess.call(['my_apache_reloader']) # this should fork and exit
    response = HttpResponse()
    response['Refresh']='3; url=%s' % reverse(home) # chk back in 3 secs
    return response

BTW There is a Django mailing list where this might be more appropriate
to discuss.

-a

Sadly, questions about subprocess spawning on the Django lists seem to
get referred back to the Python lists when they get an answer at all.
You've been very helpful, though, so thank you.

I think I'll have to mark in the database when the server needs
restarting, and run a cron job to check that and reload Apache from a
non-Apache process. It will mean delayed creation of the Virtual
Hosts and no immediate feedback to indicate success or failure, but it
doesn't look like there's much alternative.