revisiting web development in Perl...where to start?

[TheGist]

It has been a very long time since I have done
web programming. The last time I did this
was circa 2000 and at the time, like, most
of the world I wrote CGI pages in Perl.
In the years since the projects I have worked
on have (thankfully!) been in other directions.
Recently I have decided to get back into
some Web programming.
I really don't want to go down some other
road such as a Ruby or Java based solution.
Surely CGIs are
no longer widely used, right? What is the modern
way to do dynamic web pages in Perl?
I have noticed that O'Reilly hasn't updated
the Perl CGI book since 2000! People still use
Perl for web programming, right?

 

[Xho Jingleheimerschmidt]

It has been a very long time since I have done
web programming. The last time I did this
was circa 2000 and at the time, like, most
of the world I wrote CGI pages in Perl.
In the years since the projects I have worked
on have (thankfully!) been in other directions.
Recently I have decided to get back into
some Web programming.
I really don't want to go down some other
road such as a Ruby or Java based solution.
Surely CGIs are
no longer widely used, right?

I use CGI's all the time.

What is the modern
way to do dynamic web pages in Perl?

The modern way is probably to write Perl which generates JavaScript
which generates Flash which redirects through 18 layers of web services,
AOP, SLAs, IoC, XML, SOBs, and web 2.0.0.0.0.0, finally producing a pale
blue square with a blinking orange message that says "unknown error,
please try again". Modernity isn't all its cracked up to be.

Xho

 

[Jens Thoms Toerring]

It has been a very long time since I have done
web programming. The last time I did this
was circa 2000 and at the time, like, most
of the world I wrote CGI pages in Perl.
In the years since the projects I have worked
on have (thankfully!) been in other directions.
Recently I have decided to get back into
some Web programming.
I really don't want to go down some other
road such as a Ruby or Java based solution.
Surely CGIs are
no longer widely used, right? What is the modern
way to do dynamic web pages in Perl?
I have noticed that O'Reilly hasn't updated
the Perl CGI book since 2000! People still use
Perl for web programming, right?

"Web programming" is a wide field, so any useful answer
probably will start with "it depends";-) I wouldn't see
what's wrong with CGI for simple problems. But then, if
you want to do something more complex, you definitely
should take a look at

http://www.catalystframework.org/

which makes a lot of difficult things rather simple (and
does most of the boring things all by itself;-). Mix that
with running Apache with mod_perl;-)

Regards, Jens

 

[Randal L. Schwartz]

TheGist> Surely CGIs are no longer widely used, right?

Please don't confuse the CGI technology with the use of Perl.

CGI technology (which can be used with many different languages) is still
perfectly fine for websites needing under a dozen dynamic pages a minute, which
is 95% of the market.

Perl is most often these days used with mod_perl (Perl inside Apache), for
both large legacy apps and even some "green shoots" that I'm aware of.

It's far from dead, or outdated. It's the mainstream web. In fact, dynamic
languages are seeing a resurgence these days, as the promises made by the Java
Server folks are coming up a bit hollow, and people want to get back to rapid
(and sane development models.

print "Just another Perl hacker,"; # the original

 

[Nathan Keel]

It has been a very long time since I have done
web programming. The last time I did this
was circa 2000 and at the time, like, most
of the world I wrote CGI pages in Perl.
In the years since the projects I have worked
on have (thankfully!) been in other directions.
Recently I have decided to get back into
some Web programming.
I really don't want to go down some other
road such as a Ruby or Java based solution.
Surely CGIs are
no longer widely used, right? What is the modern
way to do dynamic web pages in Perl?
I have noticed that O'Reilly hasn't updated
the Perl CGI book since 2000! People still use
Perl for web programming, right?

Nothing wrong with CGI, it hasn't gone anywhere, it's still widely used
(and not just Perl using CGI). You have options if you don't want to
use CGI though. Mod_perl has been around for a very long time, you can
check into that, or even write your own Apache modules (if you use
Apache) and those modules themselves can be written in Perl if you
prefer not to use C, for example. Perl is used by a lot of large sites
for web stuff, it doesn't have to have anything to do with CGI.
Anyway, most people I know use Perl over PHP, Python or other
languages, it's just a choice.

 

[Bart Lateur]

The modern way is probably to write Perl which generates JavaScript
which generates Flash which redirects through 18 layers of web services,
AOP, SLAs, IoC, XML, SOBs, and web 2.0.0.0.0.0, finally producing a pale
blue square with a blinking orange message that says "unknown error,
please try again". Modernity isn't all its cracked up to be.

Thanks for the chuckle...

 

[sln]

TheGist> Surely CGIs are no longer widely used, right?

Please don't confuse the CGI technology with the use of Perl.

CGI technology (which can be used with many different languages) is still
perfectly fine for websites needing under a dozen dynamic pages a minute, which
is 95% of the market.

You lost me. Its got to be the CGI interface to a perl script that apparently
slows the pace to what 12 dynamic pages/minute? Because I could easily generate
100's per second dynamic pages in Perl.

The cgi modules to perl are pretty standard, no? Its not perl that would slow
page generation, must be the cgi.

Glad to give any demonstrations on page generation/sec in a perl program.

-sln

 

[Nathan Keel]

You lost me.

That's not difficult.

Its got to be the CGI interface to a perl script that
apparently slows the pace to what 12 dynamic pages/minute? Because I
could easily generate 100's per second dynamic pages in Perl.

The cgi modules to perl are pretty standard, no? Its not perl that
would slow page generation, must be the cgi.

Glad to give any demonstrations on page generation/sec in a perl
program.

-sln

Yes, CGI spawns overhead, not Perl, and CGI is slow (compared to dso or
core modules), but it's not so slow that it's really that big of a
deal, unless you have a massive amount of traffic. Good code in any
language for CGI is still better than using a web server module like
mod_perl if the code is bad or has memory leaks. But, in mod_perl, it's
going to handle more traffic if the code is good. Yes, you should be
able to have a more than 12 dynamic page requests per minute, maybe he
meant per second.

 

[TheGist]

TheGist> Surely CGIs are no longer widely used, right?

Please don't confuse the CGI technology with the use of Perl.

CGI technology (which can be used with many different languages) is still
perfectly fine for websites needing under a dozen dynamic pages a minute, which
is 95% of the market.

Perl is most often these days used with mod_perl (Perl inside Apache), for
both large legacy apps and even some "green shoots" that I'm aware of.

It's far from dead, or outdated. It's the mainstream web. In fact, dynamic
languages are seeing a resurgence these days, as the promises made by the Java
Server folks are coming up a bit hollow, and people want to get back to rapid
(and sane development models.

I think it is pretty awesome that one of the people
responding to my CGI thread is the author of one of the
books I
used to write my first CGI scripts!
Chapter 19 of the Second Edition of Learning Perl
helped me immensely when I was trying this stuff out
ten years ago.
Anyway, I think now is a good time to look into Catalyst
and Mason.
Based on this thread and other readings I do now
see how plain CGI scripts are still perfectly fine.

 

[ccc31807]

Surely CGIs are
no longer widely used, right? What is the modern
way to do dynamic web pages in Perl?
I have noticed that O'Reilly hasn't updated
the Perl CGI book since 2000! People still use
Perl for web programming, right?

I have found that this is a controversial topic. Ask a question and
you'll get twenty different answers. People will also tell you are
insane to use any solution other than their pet solution.

I write web based front ends for databases for a living. I can tell
you how I do it, and can tell you that it works for me. I can also
tell you that this is specific for reading and writing to databases,
and that 'web programming' probably implies a lot of other
functionality than I will mention here.

I use mod_perl with Apache, writing scripts as standard CGI scripts. I
don't mess with generating HTML using CGI functions, although I do use
CGI.

I write an SQL module for my SQL queries and connection to the
database. I write an HTML module that contains templates for my
standard HTML, like menus, buttons, and so on. I write scripts that
output HTML pages, CSS, and JavaScript, calling HTML.pm and SQL.pm as
appropriate.

I don't do fancy Flex or Flash or anything like that. I produce basic
HTML. Most of my development effort goes into the database side. This
probably doesn't answer your question, and may not work for your
conception of 'web programming', but I can assure you that Perl is
alive and well for spitting out HTML, CSS, and JavaScript in August,
2009.

CC

 

[Peter J. Holzer]

I use mod_perl with Apache, writing scripts as standard CGI scripts.

If you use mod_perl to execute your scripts, they are *not* standard CGI
scripts. The CGI specification mandates that each invocation of a CGI
script creates a new process which uses the environment, stdin and
stdout to communicate with the server. mod_perl doesn't do this.

hp

 

[ccc31807]

Please don't confuse the CGI technology with the use of Perl.

There's no trick to this -- you use a programming language to spit out
HTML. What programming languages are popular now?
- Perl
- Python
- Ruby
- C#
- ASP (old)
- PHP
- JSP
- ColdFusion
- the .NET languages

New ones are coming online, such as Lisp and Erlang. Wrox will publish
Erlang Web Applications this year, and a couple of years ago Peter
Seibel came out with a wildly popular book that touches on generating
HTML with Common Lisp.

The issue of frameworks is separate. Look at
http://en.wikipedia.org/wiki/Comparison_of_web_application_frameworks
The Perl frameworks listed include Catalyst, Interchange, Mason, and
Maypole.

The issue of CGI is separate still. AFAIK, when people say CGI, they
usually mean some kind of embedded technology, such as mod_perl. I had
a JSP period and an ASP period, and found these wanting compared to
Perl in terms of speed -- although this was years ago and I'm sure
vast improvements have been made.

My advice is that the language you use and the framework you may
prefer are incidental. I use Perl because I know it well and it has
the tools to do the job. This doesn't mean that you can't use Scala,
Lua, Clojure, or even server-side JavaScript to do the same thing.

CC

 

[Jamie]

I use CGI's all the time.


The modern way is probably to write Perl which generates JavaScript
which generates Flash which redirects through 18 layers of web services,
AOP, SLAs, IoC, XML, SOBs, and web 2.0.0.0.0.0, finally producing a pale
blue square with a blinking orange message that says "unknown error,
please try again". Modernity isn't all its cracked up to be.

That is sooooo true!

 

[Jamie]

It has been a very long time since I have done
web programming. The last time I did this
was circa 2000 and at the time, like, most
of the world I wrote CGI pages in Perl.
In the years since the projects I have worked
on have (thankfully!) been in other directions.
Recently I have decided to get back into
some Web programming.
I really don't want to go down some other
road such as a Ruby or Java based solution.
Surely CGIs are
no longer widely used, right? What is the modern
way to do dynamic web pages in Perl?
I have noticed that O'Reilly hasn't updated
the Perl CGI book since 2000! People still use
Perl for web programming, right?

Most people I see these days are using PHP.

That doesn't mean it's better! (I use PHP, primarily because everyone else
does.. as someone who knows their way around both, I assure you, PHP sucks)

One thing PHP does pretty good at is templating, it's a web designers
templating system, not a serious programming language.

Believe it or not, CGI is still pretty common, aside from the bloated
nature of CGI.pm, it's still a pretty good fit. There is a CGI::Lite(?)
module I used once or twice, it's good.

One thing I'm surprised no one has mentioned is FastCGI(?)

I've spoken to people who use mod_perl and they're now in the process
of moving toward "fastcgi". I like fastcgi because, with a little careful
design, you can make an application that can switch from CGI to FastCGI
and back again. (have to use clean code for this)

The advantage with plain CGI is, you don't consume resources for scripts
that are run maybe once a day and they're also MUCH easier to debug.

FastCGI is really good for cases that have a lot of hits, it beats the pants
off of other technologies (cough, PHP) because the process sits in memory. It's
great for loading in things like configuration data and then sharing them with
any new requests.

FastCGI is not a good fit for administrative control panels or utilities that
are seldom run.

FastCGI (indeed, any serious perl work) is generally only found with people who
know what they're doing, so if you're looking for opportunities, perl probably
isn't it. (this is very unfortunate)

You asked what people are using today, in terms of popularity, PHP wins.
(be prepared for massive irritation if you use it..)

Jamie

 

[ccc31807]

If you use mod_perl to execute your scripts, they are *not* standard CGI
scripts. The CGI specification mandates that each invocation of a CGI
script creates a new process which uses the environment, stdin and
stdout to communicate with the server. mod_perl doesn't do this.

It's interesting to note that a number of books advocating different
technologies often have a section near the front as to why their
technology is superior to the others. Most of these sections feature
Perl as one of the 'big' rivals, and most slam Perl by accusing it of
using CGI and therefore being slow and creating a lot of unnecessary
overhead.

Several years ago, I had a conversation with a rather large employer
about web technologies. They were using mostly JSP, migrating to .NET.
Now, I did some JSP years ago, and build a couple of sites with .NET
(C# and VB, not ASP), and my complaint was that these were slow,
particularly on the first invocation. I asked the person I talked to
about using Perl, and he said (and this is almost an exact quote),
"Perl is too slow, JSP is much faster." When I asked for the reason
for Perl's slowness, he said 'CGI.' I asked him if he used mod_perl,
and he said, and I quote, "What's mod_perl?"

You are absolutely right about the original meaning of CGI, but these
days many people use CGI to refer to a style of coding, using Perl,
Python, Ruby, and maybe some other languages. I'm guilty of this
myself, since I describe what I do as Perl/CGI, even though the last
time I actually wrote a real CGI script was about 2001.

As I pointed out in a previous post, the web development field is very
crowded with no one technology standing out from the crowd. I continue
to believe that Perl is an exceptionally strong competitor, and over
the years I've developed a strong distaste for embedded technologies
(PHP, CF, JSP, ASP, and the like).

I've played some with Erlang and found it much to my liking. Wrox has
a book coming out this year entitled 'Erlang Web Application
Development' by Nick Gerakines, and I suspect that some Erlang CGI
scripts will be in my future. Erlang probably is the closest thing to
real CGI that we have had in a long time, but because the processes
live in the language layer rather than in the OS layer, it's extremely
quick. Maybe we can say in the future that we should use CGI because
it is much faster and has much less overhead ... who knows?

CC

 

[Jamie]

It's interesting to note that a number of books advocating different
technologies often have a section near the front as to why their
technology is superior to the others. Most of these sections feature
Perl as one of the 'big' rivals, and most slam Perl by accusing it of
using CGI and therefore being slow and creating a lot of unnecessary
overhead.

I usually see it slammed because it's older and therefore, inferior.

If you're using mod_perl, I highly recommend checking into FastCGI, for
90% of the things a web app is going to do, FastCGI handles it better
than mod_perl (IMO), because it lives outside the web server as a separate
daemon process, you can keep apache lean & trim plus, it's much easier
to work with. (I found it to be more elegant anyway)

mod_perl is still better for writing apache-centric stuff though.

Several years ago, I had a conversation with a rather large employer
about web technologies. They were using mostly JSP, migrating to .NET.
Now, I did some JSP years ago, and build a couple of sites with .NET
(C# and VB, not ASP), and my complaint was that these were slow,
particularly on the first invocation. I asked the person I talked to

JSP! I actually really liked jsp (actually, servlets), it's just too..
convoluted to be practical, it's like they created something great and then,
rather than stop there, kept going until it's a mess. (But then, thats java
for you)

Java is well worth messing around with, if you can find an excuse to use
it for something, because of the ideas it contains.

You are absolutely right about the original meaning of CGI, but these
days many people use CGI to refer to a style of coding, using Perl,
Python, Ruby, and maybe some other languages. I'm guilty of this
myself, since I describe what I do as Perl/CGI, even though the last
time I actually wrote a real CGI script was about 2001.

I always thought of CGI as a kind of protocol, didn't have to be a new
process, just the way the web server communicated with another process
using stdio and environment variables. (never realized it /had/ to be a
new process)

As I pointed out in a previous post, the web development field is very
crowded with no one technology standing out from the crowd. I continue
to believe that Perl is an exceptionally strong competitor, and over
the years I've developed a strong distaste for embedded technologies
(PHP, CF, JSP, ASP, and the like).

I deal with PHP quite a bit, it's an awesome template system, but a
horrible programming language.

The criteria for selecting a technology is strictly popularity,
the underlying technology is irrelevant.

Popularity and group mentality are strong motivators, if you look at a
mob for example, the individual people would NEVER toss a brick through
someones window, BUT.. get people in a group and suddenly they start
doing foolish things, it actually seems rational to smash windows.

It's a kind of snowball effect. A casual google for one of these PHP scripts (a
popular content manager) yields 284,000,000 hits, while the keyword "perl"
yields only 65,000,000

I suspect most of the php coders out there have PHP as their first "language" and
aren't programmers so much as web designers, they figure out how to do a couple
things with it and walla, they jump into writing full-fledged php applications
that aren't planned out.

Because these people are web designers first, programmers second, the scripts
they churn out look attractive to people.

I'd sure like to find a way to turn the tide, but 284 million hits can't
be wrong, can they?

Jamie

 

[Nathan Keel]

I suspect most of the php coders out there have PHP as their first
"language" and aren't programmers so much as web designers, they
figure out how to do a couple things with it and walla, they jump into
writing full-fledged php applications that aren't planned out.

That's exactly what I've seen. Of course, none of us are claiming there
aren't skilled coders in PHP, but far more unskilled people code in PHP
than in Perl.

 

[Peter J. Holzer]

I usually see it slammed because it's older and therefore, inferior.

If you're using mod_perl, I highly recommend checking into FastCGI, for
90% of the things a web app is going to do, FastCGI handles it better
than mod_perl (IMO), because it lives outside the web server as a separate
daemon process, you can keep apache lean & trim plus, it's much easier
to work with. (I found it to be more elegant anyway)

I second that recommendation.

Can you describe this "style of coding"?

I always thought of CGI as a kind of protocol, didn't have to be a new
process, just the way the web server communicated with another process
using stdio and environment variables. (never realized it /had/ to be a
new process)

The CGI protocol (from the viewpoint of the CGI script) is basically:

* At startup, information about the header of the request plus some
other information is in the environment.
* The body of the request (if any) can be read from stdin.
* The response (including headers) must be written to stdout.
The end of the response is signalled by closing stdout.

At least on Unix-like systems (and this where CGI was invented) this
means that a process running a CGI script cannot serve more than one
request: There is no way it could get a new environment, and it cannot
reopen stdout. So the server must start a new process for the next
request.

FastCGI got around this by

* transmitting all the information from the web server to the fastcgi
script through the socket.
* Designing the protocol in such a way that both sides can signal "I am
finished with this request" without closing the communication
channel.

So the FastCGI script can continue running, and the web server can just
send one request after the other to it.

Java application servers use similar techniques.

hp

 

[ccc31807]

Can you describe this "style of coding"?

Yes. I can't say this is a global categorization, so please don't take
this too literally, but maybe as an impression and not as mutually
exclusive methods or styles.

HTML by itself is simply a markup language, it's not a programming
language. There are two ways to use HTML with programming: you can
either write the HTML and embed the programming elements in the HTML,
like PHP or JSP, or you can write the application and have it emit
HTML as output.

It's been my impression over the years that when people say 'CGI' they
don't mean using the CGI protocol in a literal sense, but writing
scripts that output HTML.

In my case, I write a lot of scripts that are heavy on the
computational side and light on the output side. For example, I just
finished a script that input two data files, one of about 37,000
records, chopped them to pieces and rearranged the data, and output
about 85 PDF files, 1 CSV file, and several HTML files. I don't know
what you would call this style of programming, but with respect to the
generation of HTML files I would call it CGI using Perl, not because
it uses the CGI protocol in any way, but because it's the way I once
wrote CGI scripts (and continue to write web applications, whether in
Perl, Python, or something else.)

I know that this isn't an accurate use of the term, but it conforms
with how the term seems to be used by the world at large.

CC

 

[Eli Morris-Heft]

That's exactly what I've seen.  Of course, none of us are claiming there
aren't skilled coders in PHP, but far more unskilled people code in PHP
than in Perl.

Ack. A bit of a self-fulfilling prophecy there. While it's not
guaranteed that most of the PHP coders out there have PHP as their
first language, I'll be a lot of them do - which is why they code in
PHP. From my professional experience, the closest most web designers
get to a programming language is either HTML (and many don't learn
that too well) or Flash (which isn't a programming language, though
Actionscript is, and 3.0 is actually a fairly respectable one - did I
write that?).

Both PHP and Perl suffer from being languages that are easy to start
learning, but difficult to build out in. They tend to breed
overconfidence from novice programmers, as sometimes exhibited on this
very newsgroup and its sister comp.lang.php. The fault, though, isn't
in the languages. It's in the programmer who fails to design, fails to
recognize limits, fails to ask before leaping. (And, really, which one
of us hasn't been that programmer at one time or another?)

From my experience, there's a lot of unskilled people out there coding
in Perl too - it's just that their scripts stay in their home
directories on their servers rather than being served for the whole
world to judge.

EMH, JAP{HP,erl}H