speedier cgi and perf monitoring

[Eric S. Johansson]

I've seen a couple solutions for making CGI faster without significant
rewrite. The only one I've been able to locate is fastCGI. I seem to
recall something like speed CGI or speedy CGI but I can't find I thought
I saw (not a puddytat). References to this other CGI speedup tool would
be appreciated.

But I've also been playing with performance monitoring to try and
improve performance with a couple of CGI that are very very hard on disk
storage. Even so, performance isn't what I think it should be. On the
simple, non disk intensive CGI, running the profiler on the CGI shows
only 0.2 seconds execution time but in reality is takng 10 seconds from
button click to page redisplay. The HTML is only three or four
kilobytes of relatively simple stuff as well so transmission or
rendering time is not a big factor.

I have some disk intensive CGI which I have measured which showed only a
few tenth of seconds execution time but also takes sometimes 15 or 20
seconds elapsed time.

what's curious is that I'm not seem anyway a detecting which methods are
taking large amounts of elapsed time. I've tried a different (albeit
simplistic) time bases to try and measure elapsed time rather than CPU
time but with no luck.

suggestions would be most appreciated

---eric (cartoon junkie)

 

[Thomas Guettler]

Am Sun, 24 Oct 2004 18:28:23 -0400 schrieb Eric S. Johansson:

I've seen a couple solutions for making CGI faster without significant
rewrite. The only one I've been able to locate is fastCGI. I seem to
recall something like speed CGI or speedy CGI but I can't find I thought
I saw (not a puddytat). References to this other CGI speedup tool would
be appreciated.

Have you looked at SCGI? It uses pre-forked worker processes.

But I've also been playing with performance monitoring to try and
improve performance with a couple of CGI that are very very hard on disk
storage.

This performance doesn't get better with SCGI. It only helps you to speed
up the start up time (loading of python, connect to database, ...)

I have some disk intensive CGI which I have measured which showed only a
few tenth of seconds execution time but also takes sometimes 15 or 20
seconds elapsed time.

I would create a standalone script for testing the performance (no
cgi, not network traffic, ...) of the parts, which are independant of
the cgi frontend.

HTH,
Thomas

 

[Eric S. Johansson]

Am Sun, 24 Oct 2004 18:28:23 -0400 schrieb Eric S. Johansson:




Have you looked at SCGI? It uses pre-forked worker processes.

thank you, thank you, thank you. That was exactly what I was looking for

This performance doesn't get better with SCGI. It only helps you to speed
up the start up time (loading of python, connect to database, ...)

yes, I know. I was not clear enough in my initial query. Read further
(and between the lines)

I would create a standalone script for testing the performance (no
cgi, not network traffic, ...) of the parts, which are independant of
the cgi frontend.

good suggestion but not the help I was looking for. What I was
observing is that the profiling tool measures CPU time not clock time.
No surprises after all that is what the documentation says. The
performance problems I'm experiencing require observing clock time (I
believe). So far, I haven't figured out how to specify clock time for
the profiling tools. any suggestions?

thinking about your suggestion about a stand-alone script, it would be
rather difficult as I would need to simulate a complete CGI form plus
cookies plus hidden variables etc. any frameworks for capturing and
replaying that kind of information.

--- eric

 

[Josiah Carlson]

good suggestion but not the help I was looking for. What I was

observing is that the profiling tool measures CPU time not clock time.
No surprises after all that is what the documentation says. The
performance problems I'm experiencing require observing clock time (I
believe). So far, I haven't figured out how to specify clock time for
the profiling tools. any suggestions?

Don't use profiling tools. Do manual profiling via time.time() calls
(which provides wall-clock time). Start coarse, and narrow down your
search each iteration.

- Josiah

 

[Eric S. Johansson]

> Don't use profiling tools. Do manual profiling via time.time() calls
(which provides wall-clock time). Start coarse, and narrow down your
search each iteration.

I appreciate the kindness of your suggestion and I will hope you will
understand when I say it's a sucky option but one I will have to live with.

---eric