Example of web application done right?

Discussion in 'Perl Misc' started by Glen Pfeiffer, Feb 15, 2004.

  1. Hi all. I am building a new web application and am evaluating different
    technologies. I was wondering if anyone can recommend a large web
    application in Perl that was "done right" so I can look at the source
    code to evaluate it and determine if I want to use Perl.

    This is not a "What is the best language?" question. I merely want to
    look at an application that is easy to extend/modify/maintain which
    separates logic from display code, and compare it to some other
    applications done in other languages. I have experience building large
    sites in other languages, but not in Perl. Some thoughts I have had are
    WebGUI, YaBB and YAWP.

    Thanks,
    Glen
     
    Glen Pfeiffer, Feb 15, 2004
    #1
    1. Advertising

  2. Glen Pfeiffer <> wrote:

    > Hi all. I am building a new web application and am evaluating different
    > technologies. I was wondering if anyone can recommend a large web
    > application in Perl that was "done right" so I can look at the source
    > code to evaluate it and determine if I want to use Perl.



    How about SlashDot?


    http://sourceforge.net/projects/slashcode/


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
     
    Tad McClellan, Feb 15, 2004
    #2
    1. Advertising

  3. Glen Pfeiffer

    Chris Guest

    Glen Pfeiffer wrote:
    > Hi all. I am building a new web application and am evaluating different
    > technologies. I was wondering if anyone can recommend a large web
    > application in Perl that was "done right" so I can look at the source
    > code to evaluate it and determine if I want to use Perl.
    >
    > This is not a "What is the best language?" question. I merely want to
    > look at an application that is easy to extend/modify/maintain which
    > separates logic from display code, and compare it to some other
    > applications done in other languages. I have experience building large
    > sites in other languages, but not in Perl. Some thoughts I have had are
    > WebGUI, YaBB and YAWP.
    >


    Moveable Type. It's written entirely in Perl, makes excellent use of
    Perl OOP and modules and uses my personal favorite web templating system
    -- HTML::Template. You can download the full source from
    http://www.moveabletype.org.

    I would consider any web site written in Perl that made use of
    HTML::Template or Template Toolkit to be at least a step in the right
    direction and for the author of such to have some understanding of
    separating logic from presentation layer which I consider absolutely
    necessary. I seldom write *test* code in Perl apart from HTML::Template
    for that very reason. I would think you would definitely want to look
    at those two tools.

    There was also very recently -- within the last year -- a sample web
    application in Perl written up in either _Linux Magazine_ or _Linux
    Journal_ that made use of Template Toolkit that looked pretty good too.
    If I have a chance to look it up later I will and report here.

    Randall Schwartz uses Template Toolkit also so you might check out his
    StoneHenge site: http://www.stonehenge.com

    Chris
    -----
    Chris Olive
    chris -at- --this-is-bogus-- technologEase -dot- com
    http://www.technologEase.com
    (pronounced "technologies")
     
    Chris, Feb 15, 2004
    #3
  4. Glen Pfeiffer

    Guest

    Glen Pfeiffer <> wrote:
    > Hi all. I am building a new web application and am evaluating different
    > technologies. I was wondering if anyone can recommend a large web
    > application in Perl that was "done right" so I can look at the source
    > code to evaluate it and determine if I want to use Perl.
    >
    > This is not a "What is the best language?" question. I merely want to
    > look at an application that is easy to extend/modify/maintain which
    > separates logic from display code, and compare it to some other
    > applications done in other languages. I have experience building large
    > sites in other languages, but not in Perl. Some thoughts I have had are
    > WebGUI, YaBB and YAWP.


    I can't give you the code, as it was proprietary :-( But I can explain the
    design:

    We used an 'N'-tier model, with perl daemon(s) running on several machines.
    The web portion used a distributed directory to look for a list of back-ends
    that were currently available, contacted the backend, submitted the
    operation(s) and processed the results. (here operations would refer to
    database inserts, user add, queries, fetching mirrored file data, etc..)

    The idea was that these back-ends would do all the heavy lifting, provide
    additional scale (need more processing? add more machines) and completely
    abstract the database from the web front.

    The back-end contacted a database for a great deal of it's work. To
    accomplish portability (and improve maintenance) we used an XML format for
    each SQL statement the system used. (this was of course, cached. we didn't
    parse the document every time we needed a database operation) this XML
    document of SQL statements proved very useful, we were able to port
    everything from postgresql to DB2 in about 45 minutes. :) It also proved
    extremely useful for tuning later on as the database grew in size, the
    SQL was clearly visable and easily modified by altering this one document.

    Keeping database handles around on the web side proved to be very
    problematic, even with mod_perl. I would not recommend it. We ended up
    ripping every bit of code that referred to a database handle out of the web
    front.

    We were successfully able to take down any machine for maintenance without a
    service outage, with the exception of the database server. (and we did take
    down the servers 1 by one on a few occassions w/out service outages)

    This was a very important feature for this project, because as you know,
    servers need upgrades and maintenance. Having redundancy is important, but
    everything has to be designed to work in that type of environment.

    In theory, we would have been able to take down the database server too
    if we had the license and hardware to mirror it, and/or had designed some
    of these back-ends to "queue" data inserts.

    As far as the web front.. I would really not recommend perl for this
    portion. (at this time) But, if you're going to do it, keep it a very simple
    model-view-controller, and use a templating system that makes sense for your
    needs. Be very wary of any session handling, as soon as you need session
    data you'll have a mess in keeping synchronized sessions across multiple web
    hosts. (session handling was by far the biggest issue we had with this
    thing)

    Jamie
     
    , Feb 15, 2004
    #4
  5. Glen Pfeiffer

    Ian Cass Guest

    Glen Pfeiffer wrote:

    > This is not a "What is the best language?" question. I merely want to
    > look at an application that is easy to extend/modify/maintain which
    > separates logic from display code, and compare it to some other
    > applications done in other languages. I have experience building large
    > sites in other languages, but not in Perl. Some thoughts I have had
    > are WebGUI, YaBB and YAWP.


    I always use Apache::ASP for my web stuff. www.apache-asp.org See their
    Testimonials section for examples of large websites using Apache::ASP.

    --
    Ian Cass
     
    Ian Cass, Feb 16, 2004
    #5
  6. <> wrote in message
    news:iySXb.43206$_44.39503@attbi_s52...

    <snip>

    > As far as the web front.. I would really not recommend perl for this


    What would you recommend? and what did you use?

    > portion. (at this time) But, if you're going to do it, keep it a very

    simple
    > model-view-controller, and use a templating system that makes sense for

    your
    > needs. Be very wary of any session handling, as soon as you need session
    > data you'll have a mess in keeping synchronized sessions across multiple

    web
    > hosts. (session handling was by far the biggest issue we had with this
    > thing)


    Did you manage without session data? or how did you handle this problem?

    Simon
     
    Simon McCaughey, Feb 17, 2004
    #6
  7. Glen Pfeiffer

    Guest

    Simon McCaughey <> wrote:
    >
    > <> wrote in message
    > news:iySXb.43206$_44.39503@attbi_s52...
    >
    > <snip>
    >
    >> As far as the web front.. I would really not recommend perl for this

    >
    > What would you recommend? and what did you use?


    We used perl, and regretted it for the web interface.

    I might go with java servlets if I were to do it again, or try and find a
    way to get PHP to do it. mod_perl wasn't very stable IMO.

    > Did you manage without session data? or how did you handle this problem?


    No.. the sessions were a constant struggle. Here is how it was supposed to
    work (as I recall):

    Session ID was a string that contained info about which machine it was on.

    If Session ID is from another machine, fetch session across network
    and make it local, use new session ID.

    Our web server load balancing was done by DNS, and, thankfully most browsers
    didn't rotate the IP, so we didn't have to copy the session across very often,
    when we did, it usually worked but on occassion it dropped them on the
    floor. :-(

    I much perfer the way Apache + tomcat handle the problem, even if you use
    perl to do the web interface, it's a technique that is worth looking at.

    Basically the way they do it is the web server is like a proxy that forwards
    the HTTP request to whoever is responsible for that that session ID. I
    suppose we could have used this approach in our code as well but didn't
    think of it then. :)

    It's kind of a matter of cost -vs- cost of failure type of problem. It's a
    hassle, but not a complete outage because (ideally) the (current_users /
    (number of servers - 1) ) are unaffected by it, and anyway, affected users
    could restart their session. Considering most of your "heavy lifting" is
    done on a backend server, taking a web server down is relatively rare anyway
    so this isn't that huge of a concern. If you were a stock broker it might be
    though. (in that case you should probably have dedicated servers that do
    nothing but manage sessions)

    The most important part of doing large-scale web applications (IMO) is
    preparing for the maintenance issues, second is hardware failures. Ideally,
    you can work it out so that any machine can be taken out with minimal
    or no impact.

    A big part of that was dynamic configuration, when you take a machine out or
    add a new one in, all the other machines need to know about it. We "abused"
    an LDAP server for this purpose. Using LDAP was overkill, but we could take
    a backend out of the loop, wait for it to finish what it was doing then stop
    it, perform an upgrade, start it, and tell the other machines about it
    via LDAP. (repeat process foreach backend until everything is updated) This
    was actually kind of cool because it meant we didn't have to login to each
    machine to update it's config. :)

    By keeping as much logic as possible outside of the web space and on
    back-end(s), most maintenance was handled 100% transparently.

    Jamie
     
    , Feb 17, 2004
    #7
  8. Glen Pfeiffer

    Chris Guest

    wrote:
    > Simon McCaughey <> wrote:
    >
    >><> wrote in message
    >>news:iySXb.43206$_44.39503@attbi_s52...
    >>
    >><snip>
    >>
    >>>As far as the web front.. I would really not recommend perl for this

    >>
    >>What would you recommend? and what did you use?

    >
    >
    > We used perl, and regretted it for the web interface.


    What exactly does that mean? If by this you mean that you used 'print'
    statements to your HTTPD output stream (eg. print qq{<table border=0
    cellpadding=0 cellspacing=0>}) then, yes, I would agree with you. On
    that count NO language that outputs to an HTTPD stdout stream gets along
    very well, Perl "here" documents are an improvement and better most
    languages, but still aren't the answer in large-scale web applications.

    If by "we used Perl" you mean you used something like HTML::Template or
    Template Toolkit, then I am surprised at your statement. IMO,
    HTML::Template coupled with Perl, separates the logic from the
    presentation about as well as anything and provides much flexibilty and
    power. I much prefer this technique to PHP. It sounds like you didn't
    use a templating solution.

    I'm not trying to sound bigoted because this is c.l.p.m. I'm *in*
    c.l.p.m because I've *been* bigoted *by* Perl. (I hope you see the
    difference). I've used a lot of so called "web solutions" out there --
    ASP, PHP, JSP, CF, Ruby, even .NET etc. -- and I keep coming back to
    Perl. It rocks them all AFAIC.

    >
    > I might go with java servlets if I were to do it again, or try and find a
    > way to get PHP to do it. mod_perl wasn't very stable IMO.
    >


    I'm very surprised at this statement as well; mod_perl is "the bomb" --
    it's rock solid. mod_perl on an Apache server can do ANYTHING. I'm not
    trying to cast a shadow on you or the resources you had available to
    you, but I can't help but express surprise at some of your statements,
    because entirely the opposite has been true for me.

    Give me Perl/mod_perl, Apache, MySQL, and HTML::Template and I'll put
    together a multi-tiered, multi-server, load-balanced web solution that
    would better anything (probably faster) put together by Java, PHP or
    JSP. My opinion, but I'll bet many here would second it.

    >
    >>Did you manage without session data? or how did you handle this problem?

    >
    >
    > No.. the sessions were a constant struggle.


    The most pain-free solution I have come up with for "sessions" in a
    load-balancing situation is to take this data to the back-end or add
    this as a separate tier as well. The extra time and resources taken for
    lookups is worth the pain it removes esp. if your back-end solution is
    tuned properly.

    I'm hoping soon to see the Berkeley XML DB put through it's paces in
    just such a scenario. For session coupling as as well as backend object
    persistence.

    Chris
    -----
    Chris Olive
    chris -at- --spammer-speed-bump-- technologEase -dot- com
    http://www.technologEase.com
    (pronounced "technologies")
     
    Chris, Feb 17, 2004
    #8
  9. Glen Pfeiffer

    gnari Guest

    "Chris" <> wrote in message
    news:MLuYb.36950$...
    > wrote:
    > > Simon McCaughey <> wrote:
    > >
    > > I might go with java servlets if I were to do it again, or try and find

    a
    > > way to get PHP to do it. mod_perl wasn't very stable IMO.
    > >

    >
    > I'm very surprised at this statement as well; mod_perl is "the bomb" --
    > it's rock solid.


    I agree. mod_perl is pretty stable.
    unless you do not use strict. then all bets are off.

    gnari
     
    gnari, Feb 17, 2004
    #9
  10. Glen Pfeiffer

    Guest

    Chris <> wrote:
    > wrote:
    >> We used perl, and regretted it for the web interface.


    [snip]

    > If by "we used Perl" you mean you used something like HTML::Template or
    > Template Toolkit, then I am surprised at your statement. IMO,
    > HTML::Template coupled with Perl, separates the logic from the
    > presentation about as well as anything and provides much flexibilty and
    > power. I much prefer this technique to PHP. It sounds like you didn't
    > use a templating solution.


    We used a template system, I'll admit, the template system wasn't ideal, but
    that was not where the problems were. The template system did keep the
    logic away from presentation, but it was awful to use because it was _very_
    strict about how it did it, NO perl allowed in the templates at all,
    everything was either static HTML or had to come from a function outside of
    template space. I perfer the jsp syntax, (and because I like perl, later
    wrote a template system using JSP syntax w/out the tags, but it wasn't
    available for this project.)

    > I'm not trying to sound bigoted because this is c.l.p.m. I'm *in*
    > c.l.p.m because I've *been* bigoted *by* Perl. (I hope you see the
    > difference). I've used a lot of so called "web solutions" out there --
    > ASP, PHP, JSP, CF, Ruby, even .NET etc. -- and I keep coming back to
    > Perl. It rocks them all AFAIC.


    I quite like perl too actually. Even though I'd been burnt by mod_perl (this
    was 4+ years ago) I still might try and find some way to use it because I
    really do like it.

    >> I might go with java servlets if I were to do it again, or try and find a
    >> way to get PHP to do it. mod_perl wasn't very stable IMO.

    >
    > I'm very surprised at this statement as well; mod_perl is "the bomb" --
    > it's rock solid. mod_perl on an Apache server can do ANYTHING. I'm not
    > trying to cast a shadow on you or the resources you had available to
    > you, but I can't help but express surprise at some of your statements,
    > because entirely the opposite has been true for me.


    It was 4 years ago. (During the tail of the dot-com era) mod_perl *was* nice
    for the simple things, but when I used it for stuff like storing database
    connections (and yes, I used Apache::DBI) I had terrible memory leaks. Tried
    different variations of them but couldn't weed them out. Removed DBI and the
    leaks went away.

    Now I *know* others didn't have this problem, in fact, most people probably
    didn't. So, maybe it had something to do with the other modules I was using,
    maybe it had to do with the order they were loaded in, could have been
    a buggy database driver, I really don't know. (was using DB2 if that matters)

    We also employed DBM as part of the session handeling, again.. problem area
    periodic segfaults on the web server. The purely-perl stuff was absolutely
    wonderful, though.

    > Give me Perl/mod_perl, Apache, MySQL, and HTML::Template and I'll put
    > together a multi-tiered, multi-server, load-balanced web solution that
    > would better anything (probably faster) put together by Java, PHP or
    > JSP. My opinion, but I'll bet many here would second it.


    I know they would. (well, less the MySQL bit, mysql is not for "real"
    database work)

    I'll remind you at this point that I was _very_ happy with perl for the
    "middle tier" stuff, just not mod_perl. You would not believe the amount of
    flack I got for using perl on this project (I was the project mgr.) But, one
    person (me) was able to write 90% of the code on this. In Java we would have
    had to use several java developers. So, yes, I can definately appreciate
    perl. :)

    > The most pain-free solution I have come up with for "sessions" in a
    > load-balancing situation is to take this data to the back-end or add
    > this as a separate tier as well. The extra time and resources taken for
    > lookups is worth the pain it removes esp. if your back-end solution is
    > tuned properly.


    Yes, if the cost of having extra servers do sessions is justified, it'd be
    the best idea. A lot of it boiled down to "how much are sessions worth?" In
    our case, they weren't worth that much. Have to be careful not to make a
    central point of failure though, simply using NFS or MySQL would have been
    by far the easiest.. until that server goes down.. :)

    One thing I'd really like to see (and maybe this is done already, haven't
    looked) is mod_webapp or some such working with a perl back-end much the way
    servlet/tomcat does. I hate the java language but do like some of it's
    ideas.

    If there were were something akin to "out of process" servlets written in
    perl I'd seriously consider it. In fact, if I had 4 months of spare time,
    I'd code one myself becase it'd be a fun project. :) (Hmm.. maybe perl
    compiled to support multiplicity would be a requirement, with support for
    each 'webapp' getting it's own instance of perl?)

    Hmm.. and if this 'perlette' were taken down in an orderly fashion it could
    have the option of mirroring it's sessions some place so that when clients
    are directed to other servlets, the sessions are restored.

    Anyhow, we were talking about large scale perl apps. I'd hold my ground on
    keeping perl away from apache. Both perl and apache are complex software
    packages and binding perl to apache would be a lot like embedding perl into
    an OS's kernel - (kernel == good), (perl == good), (perl + kernel == very bad)

    If PHP keeps on it's current track, I'd say the same for it. It needs to
    stay fairly simple or it'll create problems with apache.

    Jamie
     
    , Feb 18, 2004
    #10
  11. Glen Pfeiffer

    Guest

    gnari <> wrote:

    > I agree. mod_perl is pretty stable.
    > unless you do not use strict. then all bets are off.


    I partially agree.

    use strict;

    It's super important in any project that involves more than 1 file. (well,
    that's my general policy, if it takes more than 1 package or more than 15
    minutes of devel time, it deserves strict, and really should have a main()
    sub.) The "15 minute rule" is due to the fact that if you wait longer than
    that, you'll probably have a mess adding strict later on. Best to do it
    right away if you're unsure. :)

    I big project w/out strict is like building a house with gasoline treated
    lumber. :)

    Jamie
     
    , Feb 18, 2004
    #11
  12. wrote:

    > It was 4 years ago. (During the tail of the dot-com era) mod_perl *was* nice
    > for the simple things, but when I used it for stuff like storing database
    > connections (and yes, I used Apache::DBI) I had terrible memory leaks. Tried
    > different variations of them but couldn't weed them out. Removed DBI and the
    > leaks went away.
    >
    > Now I *know* others didn't have this problem, in fact, most people probably
    > didn't. So, maybe it had something to do with the other modules I was using,
    > maybe it had to do with the order they were loaded in, could have been
    > a buggy database driver, I really don't know. (was using DB2 if that matters)


    There have been other comments on the net alluding to memory leaks with
    certain versions of DBD::DB2. Did you report it to DB2 support? IIRC,
    the DBI driver is a supported application.
     
    Steven N. Hirsch, Feb 20, 2004
    #12
  13. Glen Pfeiffer

    Guest

    Steven N. Hirsch <> wrote:

    > There have been other comments on the net alluding to memory leaks with
    > certain versions of DBD::DB2. Did you report it to DB2 support? IIRC,
    > the DBI driver is a supported application.


    Yea, I noticed. :)

    Actually, I was kind of thankful for the bug, because stripping out DBI from
    the web front turned out to be one of the best moves. Whether you use
    mod_perl OR php for web interface, I'd recommend avoiding any database
    connection from the web interface. (Only reason I tried it was that it
    seemed convenient for simple reports and the like)

    When I left, it looked something that looks like this:
    (standard n-tier I guess)

    Apache[] <------> Backend[] <------> Database
    | |
    +-------------------+--> Directory[] services, SMTP[], etc..

    The []'s mean "more than 1 of", with DNS giving revolving IP's to Apache.

    (with Database[] if you can manage it, we couldn't justify it, $$$)

    Isolating any database access to the backends means we could replace them
    with backends that only support queries and queued inserts if for some
    reason we had to take down the database. (To swap to another vendor for
    example)

    Storing core business logic in a backend is easier to change because the
    backends are completely transparent to the users. With dynamic, directory
    based configuration we were able to easily take them in and out of the loop.
    (Directory could have been an rsync'd conf file, in our case it just
    happened to be LDAP, we needed authentication anyway.)

    The backends spoke to the front ends using custom XML, (the "schema" was
    specifically designed to parse fast, avoiding cdata and using attributes
    make parsing XML trivial and FAST with XML::parser, using xml meant we could
    easily add functionality to a backend w/out disrupting existing
    functionality and we could 'boxcar' several business-logic functions
    together)

    Obviously these services reside on physically different machines.

    Had I left the DBI code in the web interface, this would not have been
    practical, maintenance would have been harder. In a way I was glad for
    that bug. :)

    Jamie
     
    , Feb 20, 2004
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Tom
    Replies:
    0
    Views:
    444
  2. Curious Student

    Have I done them right?

    Curious Student, Jun 19, 2004, in forum: C Programming
    Replies:
    2
    Views:
    3,903
    Jack Klein
    Jun 20, 2004
  3. =?Utf-8?B?QmlzaG95?=
    Replies:
    0
    Views:
    1,048
    =?Utf-8?B?QmlzaG95?=
    Dec 28, 2006
  4. Stef Mientki
    Replies:
    2
    Views:
    287
    Stef Mientki
    Nov 23, 2008
  5. Zhidian Du
    Replies:
    0
    Views:
    178
    Zhidian Du
    Feb 21, 2004
Loading...

Share This Page