Webware vs. PHP - performance comparision!!!

[JZ]

I created an application based on Webware and Python. I was surprised
why it works so slowly. So I decided to check how fast are different
Python solutions against PHP.

My hardware and software settings are:

WindowsXP Pro, Athlon XP2800+, 512DDRdual,

Python 3.3.2 stable
Webware & WebKit & mod_webkit.dll CVS 2003-09-19
Cheetah CVS 2003-09-19
Spyce 3.11
vs.
PHP 4.3.3 stable
MMCache 2.4.1

For page request I used wget.

My testing code is
=============
import os, time
url = [
r'http://localhost/WK/test/test1/testpython.py',
r'http://localhost/WK/test/test1/testcheetah.tmpl',
r'http://localhost/WK/test/test1/testcheetah.py',
r'http://localhost/WK/test/test1/testpsp.psp',
r'http://localhost/testphp.php',
r'http://localhost/testspyce.spy',
]
for u in url:
t1=time.time()
print '>>>%s' % u,
os.system('wget -m %s -o stress.log' % u)
t = time.time()-t1
print ' => time: %f s' % t

testpython.py:
===========
from WebKit.Page import Page
class testpython(Page):
def writeHTML(self):
cpunter = 100
i=int(self.request().field('c', 0))
self.write('<p>Try: %d</p>' % i)
if 0 < i <100:
i += 1
self.write("<a href='http://%s?c=%d'>next</a>" %\
(self.request().serverURL(), i))
else:
self.write("<a href='http://%s?c=1'>next</a>" %\
self.request().serverURL())

testcheetah.tmpl:
=============
#set $counter = 100
#set $i=$int($request.field('c', 0))
<p>Try: $i</p>
#if 0 < $i <$counter:
<a href='http://${request.serverURL()}?c=${i+1}'>next</a>
#else
<a href='http://${request.serverURL()}?c=1'>next</a>
#end if


testpsp.psp:
==========
<%
counter = 100
i=int(req.field('c', 0))
res.write("<p>Try: %d</p>" % i)
if 0 < i <counter:
res.write("<a href='http://%s?c=%d'>next</a>" % (req.serverURL(),
i+1))
else:
res.write("<a href='http://%s?c=1'>next</a>" % req.serverURL())
%>

testspyce.spy
==========
<%\
counter = 100
i =request.get1('c', 0)
%>
<p>Try: <%=i%></p>
<%if 0 < i < counter:{%>
<%i += 1%>
<a
href='http://localhost<%=request.uri("path")%>?c=<%=i%>'>next</a>
<%} else: {%>
<a href='http://localhost<%=request.uri("path")%>?c=1'>next</a>
<%}%>

testphp.php:
==========
<?php
$counter = 100;
$i =$_GET['c'] ? $_GET['c'] : 0;
echo "<p>Try: $i</p>";
if ($i > 0 and $i < $counter) {
$i += 1;
echo "<a
href='http://{$_SERVER['SERVER_NAME']}{$_SERVER['PHP_SELF']}?c=$i'>next</a>";
} else {
echo "<a
href='http://{$_SERVER['SERVER_NAME']}{$_SERVER['PHP_SELF']}?c=1'>next</a>";
}
?>


RESULTS:
=========

Webware servlet without templates:
Webware servlet with its PSP templates
Webware with Cheetah template/servlet:
Webware with Cheetah servlet (compiled with cheetah-compile script):
PHP with mod_php4 and MMCache free accelerator

http://localhost/testphp.php => time: 3.891000 s [mcache & mod_php4]

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PHP as CGI
Spyce as CGII coud not test Spyce and mod_python because mod_python 3.0.3 does
not work with Python 2.3.2)

What you think about it? PHP with MMCache accelerator is about 6x
faster than teh fastest Python! I like Python but I am very
disappointed its performance!!! :-((((( The difference is so huge,
that I think about rewriting my Python code back to PHP....

Links:

Python http://www.python.org/
Webware http://webware.sourceforge.net/
Cheetah http://www.cheetahtemplate.org/
Spyce http://spyce.sourceforge.net/

PHP http://www.php.net/
MMCache http://turck-mmcache.sourceforge.net/

 

[kosh]

My hardware and software settings are:

WindowsXP Pro, Athlon XP2800+, 512DDRdual,

Python 3.3.2 stable
Webware & WebKit & mod_webkit.dll CVS 2003-09-19
Cheetah CVS 2003-09-19
Spyce 3.11
vs.
PHP 4.3.3 stable
MMCache 2.4.1

For page request I used wget.

My testing code is
=============
import os, time
url = [
r'http://localhost/WK/test/test1/testpython.py',
r'http://localhost/WK/test/test1/testcheetah.tmpl',
r'http://localhost/WK/test/test1/testcheetah.py',
r'http://localhost/WK/test/test1/testpsp.psp',
r'http://localhost/testphp.php',
r'http://localhost/testspyce.spy',
]
for u in url:
t1=time.time()
print '>>>%s' % u,
os.system('wget -m %s -o stress.log' % u)
t = time.time()-t1
print ' => time: %f s' % t

testpython.py:
===========
from WebKit.Page import Page
class testpython(Page):
def writeHTML(self):
cpunter = 100
i=int(self.request().field('c', 0))
self.write('<p>Try: %d</p>' % i)
if 0 < i <100:
i += 1
self.write("<a href='http://%s?c=%d'>next</a>" %\
(self.request().serverURL(), i))
else:
self.write("<a href='http://%s?c=1'>next</a>" %\
self.request().serverURL())

Okay I modified this code to this:

cpunter = 100
i=int(context.REQUEST.form.get('c', 0))
print '<p>Try: %d</p>' % i
if 0 < i <100:
i += 1
print "<a href='%s?c=%d'>next</a>" % (context.REQUEST.URL, i)
else:
print "<a href='%s?c=1'>next</a>" % context.REQUEST.URL
return printed

Which I used in a zope python script and modified your test script to just use
only the one url I have. I tested it on an athlon xp 2000 with 1G of ram
under linux with zope 2.6.2 running python 2.2.3 and I got

time: 0.679494 s

That places it at about 5.8x faster then your php mcache version on a slower
cpu and with no caching.

I have also profiled zope before doing very simple pages like this and it
seems to spend about 95% of it time in security checks which make almost no
impact on the speed of a real page. I find it very hard to believe that I
have slower ram, a slower cpu and am running zope and manage to beat all of
your tests by such an extreme margin.

Overall considering all the security checks that zope makes just about
anything should be faster that is not performing security checks. I suspect
your setup is flawed somewhere or that windows xp as got to be the most
aweful os to do web stuff on ever to have seen the light of day which I admit
is certainly possible and even likely.

You should look at your test results again and how you ran the test since
there is probably a major flaw somewhere. Also the zope solution I have shown
is certainly not optimized and I could make it faster. Python is not the slow
part in this test.

 

[Lothar Scholz]

I guess the main difference is that webware has this typical "i'm to
stupid to write in two languages" problem. Everything is done in
python.

Because your code is very short the overhead, request/response parsing
etc., is completely done in 'C' on the PHP side but written in python
on the 'webware' site. Also webware has a lot of unnecessary
historical code that is written for compatibility and flexibility and
not really optimized for a fastest possible servlet response model.

The other main thing is that your PHP code is handled by the same
code/thread/process in the apache server. With an Ideal IP stack
implementation there isn't any need for a task switch. But in Webware
you must

1) apache reads from socket
1) apache write into the socket stream for the webware application
server
2) switch to to application server process context
3) read from socket
4) start thread (if not already in thread pool)
5) process servlet
6) write response to socket
7) switch back to apache
8) apache must read the socket
9) apache writes into into the HTTP socket connection.


Again, PHP only needs step 1+5+9.

The other steps has a lot of overhead, i wished that someone would
start to write an apache embedded application server in python. Don't
now if Apache 2 is able to use only exact 1 process for itself.

The theory is that webware is useful for larger highly interactive
applications.
If you can cache data in memory you must not repeat SQL queries all
the time and win a lot of time. But i doubt that someone has checked
this on a larger application.

I don't make any estimation about speed in this area anymore. I lost a
few Euros because i told a friend that i can't believe that caching
data in the database is faster then writing it to a writing/reading a
temporary file.

 

[Ian Bicking]

My testing code is
=============
import os, time
url = [
r'http://localhost/WK/test/test1/testpython.py',
r'http://localhost/WK/test/test1/testcheetah.tmpl',
r'http://localhost/WK/test/test1/testcheetah.py',
r'http://localhost/WK/test/test1/testpsp.psp',
r'http://localhost/testphp.php',
r'http://localhost/testspyce.spy',
]
for u in url:
t1=time.time()
print '>>>%s' % u,
os.system('wget -m %s -o stress.log' % u)
t = time.time()-t1
print ' => time: %f s' % t

I don't think this testing code is reliable. When I use ab
(ApacheBench, comes with the Apache distribution) I get something like
130 requests/sec for the Python servlet, and 250 requests/sec for the
PHP script (though without MMCache). My system doesn't match yours --
I'm working off a slightly different version of Webware, on Linux
instead of Windows, and maybe other differences as well. But it's
pretty far off from your results.

To do the tests I run "ab -n 1000 http://localhost/whatever"

To test *Python's* speed vs. PHP, you'd have to use mod_python -- it's
the only Python *environment* that is equivalent to PHP (well, PHP as
CGI is like Python as CGI, but that's not a realistic test -- if you
care about raw performance neither option is good). That's if you are
wondering about the raw speed of the language. But I'm confident that
Python would do quite well in that test for any non-trivial application.

If you want to test the framework's speed, well, just be sure to use the
right tools, and take results from trivial applications with some salt.
Persistent Python applications can be optimized in a lot of ways -- many
of those optimizations are language-neutral (caching database output,
avoiding network connections, calculating dependencies, etc), but are
easier to implement in Python than PHP.

Ian

 

[Wilk]

I created an application based on Webware and Python. I was surprised
why it works so slowly. So I decided to check how fast are different
Python solutions against PHP.

You compare litles script where speed doesn't matter finaly.

Python web frameworks will win when you'll use them for big application
where speed can be important. You'll cache in memory for example and it
will become a lot faster than a php solution...

I've rewrited a php-cgi application with twisted, the result was a lot
faster, with less resources (mono-thread) for a code a lot cleaner... No
doubt !

I'll try to post an example...

 

[Richard Shea]

I created an application based on Webware and Python. I was surprised
why it works so slowly. So I decided to check how fast are different
Python solutions against PHP.

I was interested to see your results. They do suggest that Python
isn't all that good doing this sort of thing. However a couple of
things puzzled me about them and I wondered if perhaps the test could
stand some improvements.

First the 3 seconds for the PHP seemed remarkably slow to me. I use
PHP on a *much* less powerful machine than yours (albeit a different
OS) and that figure does seem quite slow to me for what the script is
doing.

Second if I understand your test correctly you switch from webware to
webware/templates to cheetah ... etc and take your timing immediately.
I'm just wondering whether a better (or at least more representative)
test would be to allow a different (avoid any form of caching) script
to run with each technique before allowing your timed script to run
.... in this way any "startup costs" are taken by the dummy script
before your timed script (I'm making the assumption here that in the
real world you'd use webware or cheetah not expect your server to
switch between them from request to request)

Last thing is just an instinctive thing can't really say why but those
first three Python based techniques differ in time by less than 1% ...
just seems a bit strange to me that they would all be quite so similar
in speed.

Anyway I was interested to see the numbers and maybe people a lot more
knowledgeable then me can throw some light on them.

regards

richard shea.

My hardware and software settings are:

WindowsXP Pro, Athlon XP2800+, 512DDRdual,

Python 3.3.2 stable
Webware & WebKit & mod_webkit.dll CVS 2003-09-19
Cheetah CVS 2003-09-19
Spyce 3.11
vs.
PHP 4.3.3 stable
MMCache 2.4.1

For page request I used wget.

My testing code is
=============
import os, time
url = [
r'http://localhost/WK/test/test1/testpython.py',
r'http://localhost/WK/test/test1/testcheetah.tmpl',
r'http://localhost/WK/test/test1/testcheetah.py',
r'http://localhost/WK/test/test1/testpsp.psp',
r'http://localhost/testphp.php',
r'http://localhost/testspyce.spy',
]
for u in url:
t1=time.time()
print '>>>%s' % u,
os.system('wget -m %s -o stress.log' % u)
t = time.time()-t1
print ' => time: %f s' % t

testpython.py:
===========
from WebKit.Page import Page
class testpython(Page):
def writeHTML(self):
cpunter = 100
i=int(self.request().field('c', 0))
self.write('<p>Try: %d</p>' % i)
if 0 < i <100:
i += 1
self.write("<a href='http://%s?c=%d'>next</a>" %\
(self.request().serverURL(), i))
else:
self.write("<a href='http://%s?c=1'>next</a>" %\
self.request().serverURL())

testcheetah.tmpl:
=============
#set $counter = 100
#set $i=$int($request.field('c', 0))
<p>Try: $i</p>
#if 0 < $i <$counter:
<a href='http://${request.serverURL()}?c=${i+1}'>next</a>
#else
<a href='http://${request.serverURL()}?c=1'>next</a>
#end if


testpsp.psp:
==========
<%
counter = 100
i=int(req.field('c', 0))
res.write("<p>Try: %d</p>" % i)
if 0 < i <counter:
res.write("<a href='http://%s?c=%d'>next</a>" % (req.serverURL(),
i+1))
else:
res.write("<a href='http://%s?c=1'>next</a>" % req.serverURL())
%>

testspyce.spy
==========
<%\
counter = 100
i =request.get1('c', 0)
%>
<p>Try: <%=i%></p>
<%if 0 < i < counter:{%>
<%i += 1%>
<a
href='http://localhost<%=request.uri("path")%>?c=<%=i%>'>next</a>
<%} else: {%>
<a href='http://localhost<%=request.uri("path")%>?c=1'>next</a>
<%}%>

testphp.php:
==========
<?php
$counter = 100;
$i =$_GET['c'] ? $_GET['c'] : 0;
echo "<p>Try: $i</p>";
if ($i > 0 and $i < $counter) {
$i += 1;
echo "<a
href='http://{$_SERVER['SERVER_NAME']}{$_SERVER['PHP_SELF']}?c=$i'>next</a>";
} else {
echo "<a
href='http://{$_SERVER['SERVER_NAME']}{$_SERVER['PHP_SELF']}?c=1'>next</a>";
}
?>


RESULTS:
=========

Webware servlet without templates:
Webware servlet with its PSP templates
Webware with Cheetah template/servlet:
Webware with Cheetah servlet (compiled with cheetah-compile script):
PHP with mod_php4 and MMCache free accelerator

http://localhost/testphp.php => time: 3.891000 s [mcache & mod_php4]

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PHP as CGI
Spyce as CGII coud not test Spyce and mod_python because mod_python 3.0.3 does
not work with Python 2.3.2)

What you think about it? PHP with MMCache accelerator is about 6x
faster than teh fastest Python! I like Python but I am very
disappointed its performance!!! :-((((( The difference is so huge,
that I think about rewriting my Python code back to PHP....

Links:

Python http://www.python.org/
Webware http://webware.sourceforge.net/
Cheetah http://www.cheetahtemplate.org/
Spyce http://spyce.sourceforge.net/

PHP http://www.php.net/
MMCache http://turck-mmcache.sourceforge.net/

 

[JZ]

To do the tests I run "ab -n 1000 http://localhost/whatever"

Where can I find binary version?

To test *Python's* speed vs. PHP, you'd have to use mod_python -- it's
the only Python *environment* that is equivalent to PHP

I am using mod_webkit.dll which is even faster than mod_python
http://webware.sourceforge.net/Webware-0.8.1/WebKit/Docs/InstallGuide.html#mod-python

 

[JZ]

Hello Ian,

I don't think this testing code is reliable. When I use ab
(ApacheBench, comes with the Apache distribution)

Hmm, I am trying to solve problem with my Webware application. It's
behavior is strange. It works quite good.... but sometime it hangs for
several seconds and I do not why. I changed max. nr of concurent
servlets from 20 to 100 but it did not helped? I am finding
solution so I check the problem from different points of view.

I get something like 130 requests/sec for the Python servlet, and
250 requests/sec for the PHP script (though without MMCache).

But PHP is still faster, isn't it? MMCache can speed up PHP to 10x.
But the real problem is as I wrote abowe. My Webware&Cheetah
application hangs for several seconds. It appears from time to time.
It is confusing.

To do the tests I run "ab -n 1000 http://localhost/whatever"

Where can I find binary version?

To test *Python's* speed vs. PHP, you'd have to use mod_python -- it's
the only Python *environment* that is equivalent to PHP (well, PHP as
CGI is like Python as CGI, but that's not a realistic test

I am using mod_webkit.dll which is even faster than mod_python
acordint to manual:
http://webware.sourceforge.net/Webware-0.8.1/WebKit/Docs/InstallGuide.html#mod-python

But I'm confident that Python would do quite well in that test for
any non-trivial application.

I wish to say it.

 

[Ian Bicking]

Hello Ian,



Hmm, I am trying to solve problem with my Webware application. It's
behavior is strange. It works quite good.... but sometime it hangs for
several seconds and I do not why. I changed max. nr of concurent
servlets from 20 to 100 but it did not helped? I am finding
solution so I check the problem from different points of view.

There was also a problem in your code,

if 0 < i < 100: ...

This isn't proper Python, it's interpreted as (0 < i) < 100. Since you
are using wget with the mirror option, the stalling is probably in how
wget interprets the pages -- does it mirror 100 pages (the loop size in
PHP), or does it mirror until it decides there's an infinite depth of
pages?

But PHP is still faster, isn't it? MMCache can speed up PHP to 10x.

Yes, PHP is faster in this instance. As some other people have noted,
Webware has more intermediary steps for each request. I don't know much
about MMCache, but I doubt it can do anything close to 10x when
presented with a trivial script, and I expect the benchmarking of
significant applications give much different results than this
benchmarking of a trivial application. A trivial application only
measures the basic connection overhead.

But the real problem is as I wrote abowe. My Webware&Cheetah
application hangs for several seconds. It appears from time to time.
It is confusing.


Where can I find binary version?

It comes with Apache. Since you are on Windows, I don't know where in
the Apache distribution it would be. Debian puts it in /usr/sbin/ab

I am using mod_webkit.dll which is even faster than mod_python
acordint to manual:
http://webware.sourceforge.net/Webware-0.8.1/WebKit/Docs/InstallGuide.html#mod-python

That's in the context of Webware. Webware's ModPythonAdapter doesn't
actually run servlets in the Apache process, it just uses a mod_python
handler to connect to the separate Webware process. mod_webkit does the
same thing, only written in C.

Ian

 

[kosh]

There was also a problem in your code,

if 0 < i < 100: ...

This isn't proper Python, it's interpreted as (0 < i) < 100. Since you
are using wget with the mirror option, the stalling is probably in how
wget interprets the pages -- does it mirror 100 pages (the loop size in
PHP), or does it mirror until it decides there's an infinite depth of
pages?

I thought it might be a problem when I tested a zope implementation of it
which I posted to this thread earlier but it actually worked just fine and it
worked far far faster then any of JZ's results. The output also looks correct
to me but the problem is so trivial that on more complex frameworks it just
measures the small overhead the framework has and blows it out of proportion
to a normal usage case.

It seems if this is really the kind of usage case you care about that you
should not be using php or python. You should be using tux(kernel webserver)
under linux with a hand coded asm solution to really show how fast this is
but then you would end up with wget being the cause of the slow downs. Now
admittedly I don't see why this would be very useful but somebody seems to
think that these kinds of trivial benchmarks apply to real world situations
since there sure are a lot of them around.

 

[=?iso-8859-1?Q?Fran=E7ois?= Pinard]

[Ian Bicking]

There was also a problem in your code,

if 0 < i < 100: ...

This isn't proper Python, it's interpreted as (0 < i) < 100.

Python is not PL/I. What is improper in PL/I might be proper in
Python. "if 0 < i < 100:" is interpreted as "if (0 < i) and (i < 100):".
I do not know, however, if `i' is evaluated once or twice.

 

[Lothar Scholz]

Where can I find binary version?


I am using mod_webkit.dll which is even faster than mod_python
http://webware.sourceforge.net/Webware-0.8.1/WebKit/Docs/InstallGuide.html#mod-python

Then you don't understand the differnce between an application server
like webware and mod_python/mod_php as a CGI.

Ian was talking about handling requests as a CGI.

 

[Mitch Chapman]

There was also a problem in your code,

if 0 < i < 100: ...

This isn't proper Python, it's interpreted as (0 < i) < 100.

Eh? I thought it was proper Python:

From http://www.python.org/doc/current/ref/comparisons.html

"Comparisons can be chained arbitrarily, e.g., x < y <= z is equivalent
to x < y and y <= z , except that y is evaluated only once (but in
both cases z is not evaluated at all when x < y is found to be false)."

 

[=?ISO-8859-2?B?SmFyb3OzYXcgWmFiaWWzs28=?=]

This isn't proper Python, it's interpreted as (0 < i) < 100.

No, you are wrong. It is interpreted as (0 < i ) and (i < 100)
.... print 'ok'
.... else:
.... print 'bad'
....
bad.... print 'ok'
.... else:
.... print 'bad'
....
bad
.... print 'ok'
.... else:
.... print 'bad'
....
ok

does it mirror 100 pages

Yes. It does.

It comes with Apache.

Yes! I found it in bin directory. Thanx. It is much better tool than
wget.

That's in the context of Webware.

Yes. I do not execute python scripts outside context of Webware
(I even could not make mod_python to work with Python 2.3.2)

Webware's ModPythonAdapter doesn't actually run servlets in the
Apache process, it just uses a mod_python handler to connect to the
separate Webware process. mod_webkit does the same thing, only
written in C.

Yes, and acording to webware manual mod_webkit works faster for
Webware than mod_python.

I found out, the real problem for my application is no pure
performance of Python language. It really works quite fast, but... it
oftem (for the same pages!) I have to wait even 10 or more seconds for
refresh the page. I do not know, what is wrong. It appears from time
to time. So I try to to solve this problem and I test different
approach.

 

[Peter Hansen]

[Ian Bicking]

There was also a problem in your code,

if 0 < i < 100: ...

This isn't proper Python, it's interpreted as (0 < i) < 100.

Python is not PL/I. What is improper in PL/I might be proper in
Python. "if 0 < i < 100:" is interpreted as "if (0 < i) and (i < 100):".
I do not know, however, if `i' is evaluated once or twice.

As was just mentioned in another thread yesterday, it's evaluate only once.

-Peter

 

[JZ]

Then you don't understand the differnce between an application server
like webware and mod_python/mod_php as a CGI.

Ian was talking about handling requests as a CGI.

I understand the difference. I am not interested in CGI because of its
poor performance. You do not understand the difference. CGI is the
word used for external executed scripts. mod_python, mod_php or
mord_perl are modules of Apache. They are loaded and stay in the
memory all the time. They are not CGI nor FCGI.

I compared mod_php with mod_webware although the second one is
executed through socket 8086. But mod_webkit adapter is the fastest
way for Webware. It is faster than mod_python.

 

[JZ]

Almost all of my problems gone when I created independent module with
all Cheetah definitions (also for some sql queries). This module is
compiled only once and is imported by all my servlets. My application
started to work very fast and stable. It works *much faster* than
every our other PHP applications. I am under impression.

Thanx to all for suggestions. Python realy works much faster than PHP
for non trivial code. I had poor experience with preparing
efficient code for servlets before.