Python web frameworks

[BartlebyScrivener]

Django is best among the frameworks so I
downloaded it and I found it very difficult to configure. I referred
the djangobook.

It's not a turnkey type thing like WordPress or Joomla. It's a
webframework. Also watch versions. The book examples work only in .
96. It's easy in the tutorial to stray into docs for .95 or SVN. I
think that's part of the confusion.

rd

 

[Steven Bethard]

The only difficulties I have had have been with serving static media.
Specifically, in the differences in setup between the development
environment and production, and setting it up so that I don't have to
make any changes to the code in order to roll out upgrades to a
product.

I ran into this same problem. I wish it was simply a matter of starting
the django project on another server, but a bunch of paths and URLs
always seem to need to be changed. (I end up having to modify my
urls.py files because I go from being at the URL root on the development
server to being in a subdirectory on the actual server.)

Other that those issues though, I've really enjoyed working in Django.

STeVe

 

[Istvan Albert]

I would say that that is now debatable. Overall mod_wsgi is probably a
better package in terms of what it has to offer. Only thing against
mod_wsgi at this point is peoples willingness to accept something that
is new in conjunction with Linux distributions and web hosting
companies being slow to adopt new packages.

Yes that is to be expected, many people want someone else to pay the
early adopter's costs. Nonetheless mod_wsgi seems like the right
direction to move the python world.

One confounding factor that may slow its adoption could be the need of
running plain old CGI in an efficient way. I'm not sure how that fits
into the WSGI picture.

i.

 

[Ian Bicking]

Pylons too, it's good for development but using the bundled web server
is not recommended for production.

It's actually fine to use it for production. There's two servers that
are used commonly with Pylons, the one in Paste and the CherryPy
server (it's a one-line configuration change to switch). The
generally recommended production deployment is using one of these,
with another HTTP server in front proxying requests back. There's
lots of side-benefits to that particular setup, like hosting the
Pylons app alongside static files, PHP, etc. But there's quite a few
people who serve directly from Python (probably more often using the
CherryPy server, though I don't really know the differences in a
production situation).

Anyway, mostly an aside for choosing a framework.

Ian

 

[Graham Dumpleton]

Yes that is to be expected, many people want someone else to pay the
early adopter's costs. Nonetheless mod_wsgi seems like the right
direction to move the python world.

One confounding factor that may slow its adoption could be the need of
running plain old CGI in an efficient way. I'm not sure how that fits
into the WSGI picture.

Do note that using mod_wsgi doesn't preclude you still running plain
old CGI scripts using mod_cgi or mod_cgid. As to making it more
efficient, one can go two paths on this.

The preferred path would be to put in the effort to convert the Python
CGI application to WSGI. If one still needs to run it as CGI with
other hosting solutions, then use a CGI-WSGI adapter.

Second, albeit a bit of a kludge, just like mod_python.cgihandler is a
kludge, is to emulate the CGI environment on top of WSGI. This would
work if using single threaded Apache prefork MPM, or using mod_wsgi
daemon mode with multiple processes but where each is single threaded.
It wouldn't be practical though to do it for multithread Apache worker
MPM, or multithreaded daemon processes with mod_wsgi daemon mode.
Because of how Python leaks environment variables between sub
interpreters, you would also only want to be running one sub
interpreter within a process. This would be fine if using mod_wsgi
daemon mode as different CGI scripts could be delegated to run in
different daemon processes as necessary to keep them separate, but may
not be practical if using embedded mode if hosting a range of other
WSGI applications at the same time in embedded mode.

So, it is doable, but effort would be better off expended at least
converting the Python CGI script to WSGI instead. It would save a lot
of trouble in the long run, especially with CGI scripts which weren't
designed to be run multiple times in same memory context, ie., where
they don't clean up after themselves.

If someone really wanted to host an existing CGI script under WSGI
where the same process was used over and over, and had good reasons
for doing so, it wouldn't be hard for me to come up with a workable
adapter that allowed it.

Graham

 

[TYR]

Perhaps we need a pythonic FRONTEND.

If you're meant to be able to run java code in a browser vm; and
flash; and javascript...why not a reduced version of python?

I'm thinking a sandboxed interpreter, perhaps based on EmbeddedPython,
and a restricted set of classes; core logic, string and maths, some
good graphics stuff (perhaps similar to nodebox), SSL support,
filesystem access to a cache, network, xml, soap support. A standard
model for development defining graphic entities on one hand, and
controls on the other that the graphics are bound to. Perhaps a single
class for "RemoteControls" covering web service calls intended to be
bound to a graphical entity at one end and a mod_python function at
the other?

Force pre-compiling into .pyc; include an app to do interactive
interpreter in a browser and some basic IDE functions with each plugin
download.

 

[SamFeltus]

"""Perhaps we need a pythonic FRONTEND. """

Should have happened years ago.

 

[Ian Bicking]

Yes that is to be expected, many people want someone else to pay the
early adopter's costs. Nonetheless mod_wsgi seems like the right
direction to move the python world.

One confounding factor that may slow its adoption could be the need of
running plain old CGI in an efficient way. I'm not sure how that fits
into the WSGI picture.

Practically running CGI quickly is hard. All of the modern batch of
frameworks contain too much code to do this; the startup cost of
loading all that code for each request is just too much.

For commodity servers that only support CGI scripts, and periodically
kill long-running requests, I had an idea for making WSGI applications
look like CGI scripts:
http://blog.ianbicking.org/2007/08/03/fast-cgi-that-isnt-fastcgi/ --
basically using the CGI script as a pipe to the application, with some
process management built in.

Of course, it still would need to be implemented. I suspect it would
be a fairly simple task for someone familiar with C.

This isn't really a big deal for most professional web developers, who
can control their server environment, but lack of this style of
deployment does make it a lot harder for web application users to use
Python applications.

Ian

 

[TYR]

"""Perhaps we need a pythonic FRONTEND. """

Should have happened years ago.

Python Internet Environment: PIE.

 

[joe jacob]

I ran into this same problem. I wish it was simply a matter of starting
the django project on another server, but a bunch of paths and URLs
always seem to need to be changed. (I end up having to modify my
urls.py files because I go from being at the URL root on the development
server to being in a subdirectory on the actual server.)

Other that those issues though, I've really enjoyed working in Django.

STeVe- Hide quoted text -

- Show quoted text -

Thanks everyone for the response. I installed Django and configured
apache to execute the python scripts. I am learning it now.

 

[BartlebyScrivener]

I'm just learning Django and feeling my way through all of this server
terminology. Where does Django's memcached feature fit into all of
this? When you all speak of start up costs and memory intensive
loading for each requests, doesn't the caching feature eliminate most
of that overhead?

http://www.djangoproject.com/documentation/cache/

I'm probably mixing appliances and oragutans but just curious about
how you experts feel about Django's caching.

rd

 

[Graham Dumpleton]

Do note that using mod_wsgi doesn't preclude you still running plain
old CGI scripts using mod_cgi or mod_cgid. As to making it more
efficient, one can go two paths on this.

The preferred path would be to put in the effort to convert the Python
CGI application to WSGI. If one still needs to run it as CGI with
other hosting solutions, then use a CGI-WSGI adapter.

Second, albeit a bit of a kludge, just like mod_python.cgihandler is a
kludge, is to emulate the CGI environment on top of WSGI. This would
work if using single threaded Apache prefork MPM, or using mod_wsgi
daemon mode with multiple processes but where each is single threaded.
It wouldn't be practical though to do it for multithread Apache worker
MPM, or multithreaded daemon processes with mod_wsgi daemon mode.
Because of how Python leaks environment variables between sub
interpreters, you would also only want to be running one sub
interpreter within a process. This would be fine if using mod_wsgi
daemon mode as different CGI scripts could be delegated to run in
different daemon processes as necessary to keep them separate, but may
not be practical if using embedded mode if hosting a range of other
WSGI applications at the same time in embedded mode.

So, it is doable, but effort would be better off expended at least
converting the Python CGI script to WSGI instead. It would save a lot
of trouble in the long run, especially with CGI scripts which weren't
designed to be run multiple times in same memory context, ie., where
they don't clean up after themselves.

If someone really wanted to host an existing CGI script under WSGI
where the same process was used over and over, and had good reasons
for doing so, it wouldn't be hard for me to come up with a workable
adapter that allowed it.

I had a play with this and got an initial solution working. It allows
mod_wsgi to be pointed at a directory of existing unmodified Python
CGI scripts and it will run them, maintaining the code for the CGI
script in memory between requests. One doesn't even have to rename
scripts to have a .py extension or follow Python module naming
conventions like mod_python required with its CGI handler.

Now, where as CGI script would run say at 10 requests per second for a
simple hello world program, using mod_wsgi to manage the CGI scripts
one could achieve 450+ requests per second with embedded mode of
mod_wsgi, and 250+ requests per second with daemon mode. Not as quick
as what can be achieved for an equivalent WSGI specific version of the
script, which runs at 900+ requests per second for embedded mode and
500+ requests for daemon mode, but still a lot better than a plain old
CGI script which requires a new process for each request.

One would still need to see how it works for more complicated Python
CGI scripts. Also may need to be tweaked to allow one to configure its
behaviour to suit the CGI script. For example, should additional
modules imported by the script be deleted out of sys.modules when done
or preserved for next request. Similarly, should the globals and/or
locals of the script be preserved between requests. Whether one would
need to adjust such behaviour would depend on the nature of the
specific CGI script.

The other question is whether there is even a demand for this. Do
people want to be able to take unmodified Python CGI scripts and try
to run them persistently in this way, or would they be better off
converting them to proper WSGI applications.

Graham

 

[Jan Claeys]

Op Fri, 23 Nov 2007 09:29:38 -0800, schreef BartlebyScrivener:

I'm just learning Django and feeling my way through all of this server
terminology. Where does Django's memcached feature fit into all of this?
When you all speak of start up costs and memory intensive loading for
each requests, doesn't the caching feature eliminate most of that
overhead?

It helps a lot for data or content that is dynamically generated from
database queries, but doesn't change too frequently.

http://www.djangoproject.com/documentation/cache/

I'm probably mixing appliances and oragutans but just curious about how
you experts feel about Django's caching.

I'm not an "expert", but I think Django's caching framework and the
caching middleware are great.

 

[Jeffrey Froman]

The other question is whether there is even a demand for this. Do
people want to be able to take unmodified Python CGI scripts and try
to run them persistently in this way, or would they be better off
converting them to proper WSGI applications.

I would personally be interested in such an adapter. While recently
considering whether to re-write a standalone mod_python application as CGI
or WSGI, I was scared off by this paragraph from PEP333:

-------------------------------------
Note: although we refer to it as an "application" object, this should not be
construed to mean that application developers will use WSGI as a web
programming API! It is assumed that application developers will continue to
use existing, high-level framework services to develop their applications.
WSGI is a tool for framework and server developers, and is not intended to
directly support application developers.
-------------------------------------

Do you have any thoughts about this warning? I'd much rather have the
performance of mod_wsgi for my application, so if you can provide any
mollifying views about WSGI's suitability for standalone applications my
first choice would be to use WSGI directly.

Regardless of its appropriateness for new applications, a CGI/WSGI adapter
would still be useful for third-party CGI scripts that I have no interest
in rewriting myself, as well as for the interim period during which my own
CGI applications are being rewritten. It would be advantageous for the
hosting company I work for, for example, to be able to boast that "your
python CGI scripts run faster on our mod_wsgi-enabled servers."


Thank you,
Jeffrey

 

[Istvan Albert]

The other question is whether there is even a demand for this. Do
people want to be able to take unmodified Python CGI scripts and try
to run them persistently in this way, or would they be better off
converting them to proper WSGI applications.

I think CGI will be with us for many decades.

It will be awesome if mod_wsgi can run CGI without invoking python on
each access.

i.

 

[Thomas Guettler]

It will be awesome if mod_wsgi can run CGI without invoking python on
each access.

For SCGI there is something like this: cgi2scgi: it is small executable written in C,
which connects to a running SCGI server.

Executing this small binary on every request is no big overhead.

Nevertheless I never used it.

 

[Graham Dumpleton]

For SCGI there is something like this: cgi2scgi: it is small executable written in C,
which connects to a running SCGI server.

Executing this small binary on every request is no big overhead.

Nevertheless I never used it.

That isn't the same as what is being talked about here with unmodified
CGI scripts running under mod_wsgi. That program you are talking about
is more in line with what Ian was talking about, but would still
require the CGI script to be converted to work with SCGI directly or
WSGI and then use a SCGI/WSGI adapter.

Graham

 

[Aaron Watters]

"""Perhaps we need a pythonic FRONTEND. """

Should have happened years ago.

It did. Mark Hammond embedded Python under MSIE about
the same time javascript and java applets came along (94, maybe?)
It didn't fly because of political and marketing problems,
I think. One technical killer was that C-Python can
be crashed relatively easily by malicious code, and
browser providers don't like that too much. Also, the
stuff you need to do to make it crash proof would make
the interpreter a lot slower (which may be why javascript
is pretty slow).

-- Aaron Watters

===
http://www.xfeedme.com/nucular/pydistro.py/go?FREETEXT=msie+de-facto+recognized+dim

 

[TYR]

It did. Mark Hammond embedded Python under MSIE about
the same time javascript and java applets came along (94, maybe?)
It didn't fly because of political and marketing problems,
I think. One technical killer was that C-Python can
be crashed relatively easily by malicious code, and
browser providers don't like that too much. Also, the
stuff you need to do to make it crash proof would make
the interpreter a lot slower (which may be why javascript
is pretty slow).

-- Aaron Watters

===http://www.xfeedme.com/nucular/pydistro.py/go?FREETEXT=msie+de-facto+...

Time for another go, perhaps, seeing as the web has become the world's
default user interface?

 

[Jeffrey Froman]

While recently
considering whether to re-write a standalone mod_python application as CGI
or WSGI, I was scared off by this paragraph from PEP333:

<snip scary warning about using WSGI as an application API>

As a followup, I did go ahead and convert my CGI handler to WSGI, and doing
so was not difficult at all. The steps were basically:

1. accept environ and start_response parameters in the handler's __init__
2. return output from __iter__ instead of writing it to stdout.
3. pass environ and wsgi.input to the cgi.FieldStorage() parser.

I'd still be interested in a mod_wsgi wrapper for 3rd-party CGI scripts.


Thank you,
Jeffrey