FCGI app reloading on every request

J

John Nagle

I'm converting a web app from CGI to FCGI. The application works fine
under FCGI, but it's being reloaded for every request, which makes FCGI
kind of pointless. I wrote a little FCGI app which prints when the program is
loaded and when it gets a request. And indeed, the program gets reloaded for
each HTTP request. Something is probably misconfigured. But what?

I'm using Apache with "mod_fcgid", configured via Plesk. Plesk
sets up "/etc/httpd/conf.d/fcgid.conf" with:

# added by psa-mod-fcgid-configurator
<IfModule mod_fcgid.c>
IdleTimeout 3600
ProcessLifeTime 7200
MaxProcessCount 64
DefaultMaxClassProcessCount 8
IPCConnectTimeout 30
IPCCommTimeout 45
DefaultInitEnv RAILS_ENV production
</IfModule>

which seems reasonable enough, even though I'm not using
Rails or Ruby.

On the Python side, I'm using Python 2.5 on Linux, with Alan
Saddi's "fcgi.py" module.

Everything seems to work correctly, and there are no errors in the Apache
error log. But performance is terrible because the Python code
is reloaded for each request. I can only get about five transactions
a second through, on an dedicated server, with each transaction
doing a simple MySQL request. All the time is going into program loading;
the database is mostly idle.

John Nagle
 
?

=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=

John said:
I'm converting a web app from CGI to FCGI. The application works fine
under FCGI, but it's being reloaded for every request, which makes FCGI
kind of pointless. I wrote a little FCGI app which prints when the
program is loaded and when it gets a request. And indeed, the program
gets reloaded for
each HTTP request. Something is probably misconfigured. But what?

The most likely reason is that your FastCGI server voluntarily choses
to exit after one request. Can you share the mainloop of your
application?

Regards,
Martin
 
J

John Nagle

Martin said:
The most likely reason is that your FastCGI server voluntarily choses
to exit after one request. Can you share the mainloop of your
application?

Regards,
Martin

It turns out that's not the problem.

What's actually happening is that FCGI isn't running at all.
My .fcgi file is being executed by Apache's CGI handler, and
"fcgi.py" recognizes this, then reads the parameters as if
a CGI program. So it works just like a CGI program: one
load per request. Not sure why Apache is doing that yet.
I'm looking at Apache configuration files now.

The main loop is in "fcgi.py", which calls the
user application as a function. The user app doesn't have the
option of exiting after one request, short of aborting without
returning a result.

John Nagle
 
?

=?ISO-8859-1?Q?Michael_Str=F6der?=

John said:
What's actually happening is that FCGI isn't running at all.
My .fcgi file is being executed by Apache's CGI handler, and
"fcgi.py" recognizes this, then reads the parameters as if
a CGI program. So it works just like a CGI program: one
load per request. Not sure why Apache is doing that yet.
I'm looking at Apache configuration files now.

Are you running mod_fastcgi oder mod_fcgid?

Disclaimer: I don't claim to be an expert in that. My web2ldap source
distribution contains two examples configuration files for each of the
modules above for FastCGI over Unix Domain Socket( see directory
<web2ldap-dir>/etc/httpd/, comments of sample-mod_fcgid.conf
misleading). web2ldap can also be deployed as (multi-threaded)
FastCGI-server.

Ciao, Michael.
 
J

John Nagle

Michael said:
Are you running mod_fastcgi oder mod_fcgid?

I'm using mod_fcgid.

This is running on a dedicated server at APlus.net,
running Red Hat Fedora Core 6, Python 2.5, and managed with Plesk 8.2.
I just turned on fcgid from the Plesk control panel ("Physical hosting
setup page for domain", checked "FastCGI"), and enabled the standard
FCGI configuration, which is actually mod_fcgid.

The Python app is launched from "cgi-bin" with a file ending in
".fcgi". It uses Alan Saddi's "fcgi.py" module to interface with
an FCGI server. Python programs that use "fcgi.py" will also run
as standard CGI programs, for debug purposes. And that's what's
happening. I modified my test Python program to patch
"fcgi.CGIrequest._end" with a version that logged its calls,
and that's being called. This indicates that "fcgi.py" is running
in CGI mode, not FCGI mode. My test program is reloaded for every
HTTP request, as indicated by debug output, so it really is running
in CGI mode, with the expected low performance.

What's wierd is that the ".fcgi" suffix files get executed at all.
I'd expect them to be either ignored or run properly with FCGI.

John Nagle
 
T

Thorsten Kampe

* John Nagle (Mon, 03 Sep 2007 21:26:01 -0700)
I'm converting a web app from CGI to FCGI. The application works fine
under FCGI, but it's being reloaded for every request, which makes FCGI
kind of pointless. I wrote a little FCGI app which prints when the program is
loaded and when it gets a request. And indeed, the program gets reloaded for
each HTTP request. Something is probably misconfigured. But what?
[...]
On the Python side, I'm using Python 2.5 on Linux, with Alan
Saddi's "fcgi.py" module.

Do you use fcgi.py or flup? Go for flup


"What's the difference between flup's fcgi module and your previous
fcgi.py?

Feature-wise, very little, as I do try to keep both implementations in
sync. The most significant difference is that flup's fcgi module uses
thread pooling, fcgi.py simply starts a new thread for each
connection/request. flup's code is refactored and modularized, to
minimize duplicated code between all 6 servers."

http://trac.saddi.com/flup/wiki/FlupFaq#Whatsthedifferencebetweenflups
fcgimoduleandyourpreviousfcgi.py
 
J

John Nagle

Thorsten said:
* John Nagle (Mon, 03 Sep 2007 21:26:01 -0700)
I'm converting a web app from CGI to FCGI. The application works fine
under FCGI, but it's being reloaded for every request, which makes FCGI
kind of pointless. I wrote a little FCGI app which prints when the program is
loaded and when it gets a request. And indeed, the program gets reloaded for
each HTTP request. Something is probably misconfigured. But what?
[...]
On the Python side, I'm using Python 2.5 on Linux, with Alan
Saddi's "fcgi.py" module.


Do you use fcgi.py or flup? Go for flup

I think that's irrelevant. The Python app is being launched in
CGI mode, indicating trouble on the Apache side.

Anything executable in the
cgi-bin directory is being launched as a CGI program. A file
named "example.foo", if executable, will launch as a CGI program.
Nothing launches with FCGI.

Tried putting this in the .htaccess file:

<Files *.fcgi>
SetHandler fcgid-script
Options ExecCGI
allow from all
</Files>

<Files *.foo>
ErrorDocument 403 "File type not supported."
</Files>

Even with that, a ".foo" file gets executed as a CGI script,
and so does a ".fcgi" file. It's an Apache configuration problem.

John Nagle
 
G

Guest

Anything executable in the
cgi-bin directory is being launched as a CGI program. A file
named "example.foo", if executable, will launch as a CGI program.
Nothing launches with FCGI.

Perhaps you have a SetHandler declaration somewhere that makes all
files CGI by default? I would advise against Sethandler, and
recommend AddHandler instead.

Regards,
Martin
 
?

=?ISO-8859-1?Q?Michael_Str=F6der?=

John said:
This is running on a dedicated server at APlus.net,
running Red Hat Fedora Core 6, Python 2.5, and managed with Plesk 8.2.
I just turned on fcgid from the Plesk control panel ("Physical hosting
setup page for domain", checked "FastCGI"), and enabled the standard
FCGI configuration, which is actually mod_fcgid.
[..]
What's wierd is that the ".fcgi" suffix files get executed at all.
I'd expect them to be either ignored or run properly with FCGI.

Then I would ask APlus.net what's really going on there.

Ciao, Michael.
 
G

Guest

John said:
Tried putting this in the .htaccess file:

<Files *.fcgi>
SetHandler fcgid-script
Options ExecCGI
allow from all
</Files>

<Files *.foo>
ErrorDocument 403 "File type not supported."
</Files>

Even with that, a ".foo" file gets executed as a CGI script,
and so does a ".fcgi" file. It's an Apache configuration problem.

Are you sure that the server configuration allows overriding this in the
..htaccess file?

Ciao, Michael.
 
S

Sion Arrowsmith

John Nagle said:
Tried putting this in the .htaccess file:

<Files *.fcgi>
SetHandler fcgid-script
Options ExecCGI
allow from all
</Files>

<Files *.foo>
ErrorDocument 403 "File type not supported."
</Files>

Even with that, a ".foo" file gets executed as a CGI script,
and so does a ".fcgi" file. It's an Apache configuration problem.

I'd look to see if you've got a AllowOverride None set somewhere
unhelpful (probably on the cgi-bin directory, although I note the
default Apache2 config on my machine here does it for the document
root too). Mind you, if you're managing this with a web tool rather
than having access to the Apache config files, it might not be so
straightforward to do.
 
J

John Nagle

Sion said:
I'd look to see if you've got a AllowOverride None set somewhere
unhelpful (probably on the cgi-bin directory, although I note the
default Apache2 config on my machine here does it for the document
root too). Mind you, if you're managing this with a web tool rather
than having access to the Apache config files, it might not be so
straightforward to do.

I now know what's wrong; it's a combination of several problems.

First, configuring Apache via Plesk has very limited options,
and no documentation telling you what it's doing. The actual
problem is that Plesk generates, in "/etc/conf/httpd.conf".

ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"

This makes anything in cgi-bin executable with mod-cgi,
regardless of anything set up for any other suffix. For
historical reasons, CGI gets better treatment in Apache than
other "mod-" components, and apparently ScriptAlias outranks
Files and AddHandler. So that's why .fcgi programs were being
run as .cgi programs.

With the Plesk-created configuration, an executable ".fcgi" file
in any web-server-visible directory OTHER than "cgi-bin" will execute
with mod_fcgi. So that's how to get around this. Note that this is
backwards from the usual CGI configuration, where scripts will execute
only from "cgi-bin". This has security implications.

If you're using Plesk, you can't change the defaults, because Plesk
regenerates the config files when you use the Plesk control panel app.
Nor will overrides in .htaccess work; those are disallowed by an
"AllowOverride None". So you're stuck with what Plesk gives you.
Even on a dedicated server using Plesk.

Then there's mod_fcgid, which Plesk configures but has almost no
documentation either. If mod_fcgid can't spawn an application
process, for any reason, including a simple Python error,
the error message is "[warn] mod_fcgid: can't apply process slot for ...".
That's all you get in the Apache log. The web user gets "Service
Temporarily Unavailable", because mod_fcgid doesn't distinguish
internally between "don't have enough resources to spawn a new process"
and "new process didn't launch correctly". I had to look at the
source code for mod_fcgid to figure out what it meant. (Ruby on Rails
people get this message now and then, and they're puzzled by it.)

So that's what's going on. This sort of thing is why mod_fcgid,
which is actually a good concept, isn't used much.

John Nagle
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,902
Latest member
Elena68X5

Latest Threads

Top