Using Python for my web site

[northband]

Hi, I am interested in re-writing my website in Python vs PHP but have
a few questions. Here are my specs, please advise as to which
configuration would be best:

1.Dell Poweredge Server, w/IIS, currently Windows but considering
FreeBSD
2. Site consists of result pages for auctions and items for sale (100
per page)
3. MySQL (Dell Poweredge w/AMD) database server connected to my web
server
4. Traffic, 30 million page loads/month

I am trying to have the fastest page loads, averaging 100 items per
result page. I have read about using Apache's mod_python so I could
use PSP. Any help or tips are appreciated.

-Adam

 

[Norman Khine]

Hi, I am interested in re-writing my website in Python vs PHP but have
a few questions. Here are my specs, please advise as to which
configuration would be best:

1.Dell Poweredge Server, w/IIS, currently Windows but considering
FreeBSD
2. Site consists of result pages for auctions and items for sale (100
per page)
3. MySQL (Dell Poweredge w/AMD) database server connected to my web
server
4. Traffic, 30 million page loads/month

I am trying to have the fastest page loads, averaging 100 items per
result page. I have read about using Apache's mod_python so I could
use PSP. Any help or tips are appreciated.

-Adam

you should try http://www.ikaaro.org/itools it is a python library and
has all you need, although it is still in alpha state, it is running
some heavy applications such as http://destinationsflybe.com as an example.

for the mysql you can use the csv module.

for now it has not been ported on window$, try gentoo or freebsd

 

[northband]

Thanks I will look into it.

-Adam

you should try http://www.ikaaro.org/itools it is a python library and
has all you need, although it is still in alpha state, it is running
some heavy applications such as http://destinationsflybe.com as an example.

for the mysql you can use the csv module.

for now it has not been ported on window$, try gentoo or freebsd

 

[Bruno Desthuilliers]

Hi, I am interested in re-writing my website in Python vs PHP but have
a few questions. Here are my specs, please advise as to which
configuration would be best:

1.Dell Poweredge Server, w/IIS, currently Windows but considering
FreeBSD

I may be a bit biased, but I would not run a web server under Windows...

2. Site consists of result pages for auctions and items for sale (100
per page)
3. MySQL (Dell Poweredge w/AMD) database server connected to my web
server

Have you considered Postgresql instead ?

4. Traffic, 30 million page loads/month

I am trying to have the fastest page loads, averaging 100 items per
result page. I have read about using Apache's mod_python so I could
use PSP.

That's a possible solution.

Any help or tips are appreciated.

You may eventually be interested in Django (full stack MVC web
framework, runs on mod_apache, supports both MySQL and Postgresql)

http://www.djangoproject.com/

or Myghty (Python's extended port of Perl::Mason, runs on mod_apache,
not tied to any RDBMS) + eventually SQLAlchemy (orm)
http://www.myghty.org/
http://www.sqlalchemy.org/

Now I can't tell if either one or the other beats PSP when it comes to
raw perfs...

My 2 cents

 

[northband]

Thanks for the info. Reason why we are interested in PSP is because we
think our developers would be able to quickly adapt to the migration,
they are familiar with php and asp.

I agree on the windows webserver, we are looking into using FreeBSD
instead.

What would postgre's advantage over MySQL be?

I will look into Django.

Thanks for your .2

-Adam

 

[Bruno Desthuilliers]

Thanks for the info. Reason why we are interested in PSP is because we
think our developers would be able to quickly adapt to the migration,
they are familiar with php and asp.

Yes, I guessed it had do to with something like this. OTOH, the
ServerPage model can quickly lead to a disgusting and barely
maintainable mess of application logic and presentation, while the MVC
approach helps designing and writing much more cleaner code (IMHE). As a
matter of fact, lots of PHP apps turns into some more or less well
backed reimplementation of the MVC model.

I agree on the windows webserver, we are looking into using FreeBSD
instead.

What would postgre's advantage over MySQL be?

PostgreSQL is a mature, rock-solid, full-blown RDBMS. While not as
laughable as it used to be until very recently, MySQL is still barely a
DBMS at all - and our experiences with it is that it breaks down way too
easily and is a king-size PITA to maintain and upgrade. In fact, the
real question IMHO is: what would MySQL advantage over PostgreSQL be ?-)

I will look into Django.

I can't tell if it'll meets your needs, but I really thinks it's worth
giving it a try.

 

[Gerhard Fiedler]

In fact, the real question IMHO is: what would MySQL advantage over
PostgreSQL be ?-)

A few years ago I did some research, and the result was that while
PostgreSQL was claimed to have more features and a better design, the
reports of database corruption seemed to have been more frequent than with
MySQL. The usual reason given was that MySQL was more mature.

I assume you don't agree...

Gerhard

 

[Cliff Wells]

A few years ago I did some research, and the result was that while
PostgreSQL was claimed to have more features and a better design, the
reports of database corruption seemed to have been more frequent than with
MySQL. The usual reason given was that MySQL was more mature.

I assume you don't agree...

I certainly don't. MySQL provides two different ways to corrupt your
data: actual database corruption *and* data integrity corruption. You
can escape the second by using InnoDB rather than MyISAM tables, but
that increases your chances of the first (InnoDB not being as "mature"
as either MyISAM or PostgreSQL).

Also, saying "a few years ago I did some research" in software terms is
pretty much equivalent to saying "I don't know".

Regards,
Cliff

--

 

[Gerhard Fiedler]

I assume you don't agree...

I certainly don't. [...]

Also, saying "a few years ago I did some research" in software terms is
pretty much equivalent to saying "I don't know".

Exactly. So what's your point with this comment?

I stated what was my impression at the time, with the hope that others
might want to comment. Thanks for the comment.

OTOH, anybody who says "I know" regarding a comparison in reliability
between databases must have pretty good data to back that up. Few have.
Most are in the "I don't know, but my impression is that ..." group.

Gerhard

 

[northband]

So seems the best approach would be a MVC model rather than server
scripting? Currently our site is built with a closed source hypertext
preprocessor much like PHP. So it would be easier for us to script
another site, but if we would gain performance via a MVC model, then
that's what we need.

I am not very familiar with developing via MVC, any good tutorials out
there or good places to start?

-Adam

 

[Cliff Wells]

I assume you don't agree...

I certainly don't. [...]

Also, saying "a few years ago I did some research" in software terms is
pretty much equivalent to saying "I don't know".

Exactly. So what's your point with this comment?

My point is to stop FUD right at that comment. I don't doubt your
research from "a few years ago", but ancient research is entirely
irrelevant for making a decision *today*.

However, had I let it pass, then someone else might not make that
distinction and come away with the impression that your research was
somehow still relevant and that PostgreSQL is less reliable than MySQL.

I stated what was my impression at the time, with the hope that others
might want to comment. Thanks for the comment.

You're welcome.

OTOH, anybody who says "I know" regarding a comparison in reliability
between databases must have pretty good data to back that up. Few have.
Most are in the "I don't know, but my impression is that ..." group.

Absolutely. I can only give you anecdotal evidence myself. Further, if
someone were to present a whitepaper of some sort demonstrating that one
is superior with regard to reliability or performance, I'd probably be
highly suspect of their motives.

Regardless, since my job as a hoster requires that I assist customers
with database issues, I have hands-on experience with dozens of
instances of each and my *very recent* experience tells me that MySQL is
far more prone to database corruption than PostgreSQL. In the past 6
months, I've repaired or restored at least 4 MySQL databases (and for no
apparent reason, to boot), but I've had to do the same for exactly zero
PostgreSQL installs since I started hosting over 3 years ago. And just
to be clear, the number of PostgreSQL installs far exceeds the number of
MySQL installs. Were there equal numbers of each I'd expect even more
MySQL problems.

Regards,
Cliff

--

 

[northband]

Just spoke with my department and looks like we still want to go with a
server scripting method. Although MVC may be better fit, for the sake
of the learning curve, we want to use a PSP style method.

So as of now we are looking at using FreeBSD, MySQL, and some form of
Python that will allow us to achieve great performance serving
30million page loads / month.

-Adam

 

[Sybren Stuvel]

Gerhard Fiedler enlightened us with:

A few years ago I did some research, and the result was that while
PostgreSQL was claimed to have more features and a better design,
the reports of database corruption seemed to have been more frequent
than with MySQL. The usual reason given was that MySQL was more
mature.

I'd like to counter that. We got rid of *all* our database problems
when we moved from MySQL to PostgreSQL. PostgreSQL seems to be the
more mature one.

Sybren

 

[Cliff Wells]

Just spoke with my department and looks like we still want to go with a
server scripting method. Although MVC may be better fit, for the sake
of the learning curve, we want to use a PSP style method.

I'm with the others who suggest using an MVC framework. The learning
curve for Django, TurboGears, Pylons, et al, is ridiculously short, and
the maintainability of the resulting code is infinitely superior.

Why don't you take a look at the 20 minute wiki screencast that
TurboGears has and make a decision then. Although the screencast is
specifically about TurboGears, a similar screencast could be made for
almost any of the other MVC-style frameworks:

http://files.turbogears.org/video/20MinuteWiki2nd.mov
http://www.turbogears.org/preview/docs/tutorials/wiki20/index.html

Developing in a PHP/ASP embedded style is an anachronism these days and
for good reason. Spend a couple days learning a modern framework. The
time will be well-spent and quickly made up in shortened development
time and code maintainablility.

So as of now we are looking at using FreeBSD, MySQL, and some form of
Python that will allow us to achieve great performance serving
30million page loads / month.

If I were you, I'd cease worrying about the performance of the framework
itself and research caching proxies and load balancing solutions
instead. The payoff in performance will be much higher and you won't
have to make architectural compromises.

Regards,
Cliff

--

 

[northband]

Makes sense, I will follow your advice. Sounds like more time invest
upfront will equal time saved over the long run. I am defitely
interested in proxy caching and load balancing. Which do you
recommend? I have used #Pound while working for a university.

-Adam

 

[Diez B. Roggisch]

So seems the best approach would be a MVC model rather than server
scripting? Currently our site is built with a closed source hypertext
preprocessor much like PHP. So it would be easier for us to script
another site, but if we would gain performance via a MVC model, then
that's what we need.

MVC and server side scripting are two totally orthogonal things - you
don't do the one and ditch the other as a consequence of that.

MVC is a pattern of software design. It won't affect the performance
itself, at least not the system performance. It most probably will boost
your developing performance, as following the clear separation of data
(model), application logic (controller) and view-logic (view) will help
maintaining the codebase and make changes easier.

And MVC doesn't depend on the language used - you can even do that in
PHP, albeit it makes things much more dependent on discipline.

Diez

 

[Bruno Desthuilliers]

Gerhard Fiedler a écrit :

A few years ago I did some research, and the result was that while
PostgreSQL was claimed to have more features and a better design, the
reports of database corruption seemed to have been more frequent than with
MySQL. The usual reason given was that MySQL was more mature.
>
I assume you don't agree...

I obviously don't, based on experience with both.

 

[Cliff Wells]

Makes sense, I will follow your advice. Sounds like more time invest
upfront will equal time saved over the long run. I am defitely
interested in proxy caching and load balancing. Which do you
recommend? I have used #Pound while working for a university.

I currently use Pound, mostly for proxying to virtual hosts on my shared
hosting systems, but for heavy page hits and dynamic content, Squid is
probably of more interest since Pound doesn't do caching.

Regards,
Cliff

 

[Bruno Desthuilliers]

northband a écrit :

So seems the best approach would be a MVC model rather than server
scripting?

This is still "server-scripting" - at least since on the server-side and
is done with something frequently labelled as a "script language" !-)

Currently our site is built with a closed source hypertext
preprocessor much like PHP. So it would be easier for us to script
another site,

Depends on how your app is written - one can do MVC with PHP too.

but if we would gain performance via a MVC model,

This won't change anything to performances. What you can gain is a
better design, well-decoupled code, hence far less maintenance problems.

I am not very familiar with developing via MVC, any good tutorials out
there or good places to start?

Do the Django tutorial. They actually name things differently (ie they
call controllers views and views templates), which can be a bit
misleading, but that's really MVC.

 

[Bruno Desthuilliers]

northband a écrit :

Just spoke with my department and looks like we still want to go with a
server scripting method. Although MVC may be better fit, for the sake
of the learning curve, we want to use a PSP style method.

I really don't think the learning curve will be a problem. We learned
Django while using it for an app, and the app was completed and
delivered within a week (total 8 man/day). Granted, it was a dead simple
app, but it would have take as much time doing it in PHP. The last one
we did was not trivial, and was ready to deliver in 2 weeks - one week
before schedule. This very first release is still running, and we didn't
have a single bug report. Not to say that Django is a silver-bullet, but
it's actually the best tool we found for sql-based web apps so far.

So as of now we are looking at using FreeBSD, MySQL, and some form of
Python that will allow us to achieve great performance serving
30million page loads / month.

FWIW, when it comes to web apps, and given a reasonnably well
designe/implemented app, "performance" is more a matter of
tuning/hardware/etc than anything else.