Simple script execution problems (newbie)

[Paul E. Schoen]

I am trying to set up a webspace so I can use CGI, and I'm getting errors
and/or it is not behaving as it should using IE8 or Firefox. I read the FAQ
and it suggests that this should be asked in
comp.infosystems.www.authoring.CGI, but that NG has been removed. So, at the
very least, please change the perl FAQ to reflect this and provide a NG that
is appropriate.

I have a simple perl script which I copied from documentation here:
http://blog.dreamhosters.com/kbase/index.cgi?area=144

#!/usr/bin/perl
use CGI;
my $query= new CGI;
print $query->header;
print "hello people in my head\n";

Here are the problems I'm having:

On the dreamhost server http://www.pauleschoen.com/cgi-bin/first.pl, I get
Internal Server Error 500 and the error log reports "Premature end of script
headers:". On my other server, www.smart.net/~pstech/cgi-bin/first.pl, the
IE8 browser tries to download the file, and Firefox just shows the source
code text. If I use telnet to execute the script in smart.net, it seems to
echo the correct header and text. I have also used the pico editor to
eliminate possible problems with CR+LF newlines. I have not been able to log
on to the dreamhost server with telnet.

I have tried changing the permissions as recommended, and I have the
..htaccess file configured as:

Options +ExecCGI

My next step will probably be to contact the support or community
discussions for these servers, but I was hoping that someone here might be
able to help. I am planning to do some less trivial programming in Perl once
I get this simple script working as it should.

Thanks!

Paul

 

[Steve]

I am trying to set up a webspace so I can use CGI, and I'm getting
errors and/or it is not behaving as it should using IE8 or Firefox. I
read the FAQ and it suggests that this should be asked in
comp.infosystems.www.authoring.CGI, but that NG has been removed. So, at
the very least, please change the perl FAQ to reflect this and provide a
NG that is appropriate.

I have a simple perl script which I copied from documentation here:
http://blog.dreamhosters.com/kbase/index.cgi?area=144

#!/usr/bin/perl

# EEEK!!!

use CGI;
my $query= new CGI;
print $query->header;
print "hello people in my head\n";

Keep it *really* simple:

#!/usr/bin/perl

use warnings; # just do it, don't worry about why for now
use strict; # you'll save oodles of time debugging code later

print "Content-type: text/html; charset=utf-8\n\n

print "Hello World\n";

exit;

Here are the problems I'm having:

On the dreamhost server http://www.pauleschoen.com/cgi-bin/first.pl, I
get Internal Server Error 500 and the error log reports "Premature end
of script headers:". On my other server,

Mime-type, permissions, ownership, combination of.... sounds like
cgi-bin might be set up more or less correctly though.

Assuming the webserver thinks the script owner and user configured to
run the script match up. (IF SUEXEC is enabled. If it isn't: run screaming)

www.smart.net/~pstech/cgi-bin/first.pl, the IE8 browser tries to
download the file, and Firefox just shows the source code text.

Sounds like cgi-bin is NOT configured correctly in httpd.conf.

Ah... did *you* create the cgi-bin directory or was it already there?

If I use
telnet to execute the script in smart.net, it seems to echo the correct
header and text. I have also used the pico editor to eliminate possible
problems with CR+LF newlines. I have not been able to log on to the
dreamhost server with telnet.

I have tried changing the permissions as recommended, and I have the
.htaccess file configured as:

Options +ExecCGI

The above may or may not do any good depending on Server configuration.

My next step will probably be to contact the support or community
discussions for these servers, but I was hoping that someone here might
be able to help. I am planning to do some less trivial programming in
Perl once I get this simple script working as it should.

Thanks!

Paul

Here is a link to a page you may find useful:

http://brian-d-foy.cvs.sourceforge.net/viewvc/brian-d-foy/CGI_MetaFAQ/CGI_MetaFAQ.html

And you really should set up a web server on your local box. The first
time you fire off a script with an infinite while(){} loop in it, your
providers will not be happy..... and it *will* happen.

hth,

\s

 

[Paul E. Schoen]

There is no guarantee that there is one. (If the FAQ points to a
non-existent NG that should be removed, of course.)

It appears that comp.infosystems.www.authoring.misc is still supported, but
very little activity. Last post 7/6/2010. I'll try there.

Is /usr/bin/perl a valid path to perl on that machine?

I can't access it using FTP, and I can't use Telnet, so I'm not 100% sure.
But that is the standard location and it is from the documentation on the
server.

Is the script executable?

I have permissions set as 755, so it should be executable.

Does the webserver user (or whatever user is used to run the
CGI) have read and execute access to the script and execute access to
all directories down from the root?

The directory permissions are all 755

Does that perl have CGI installed?

It should be. Here is the documentation: http://wiki.dreamhost.com/CGI

Is mod_perl involved?

I don't know.

Do you see any other errors?

There is only a "suexec failure: could not open log file", "fopen:
Permission denied", "Premature end of script headers: test.cgi", "File does
not exist: /home/pes1949/pauleschoen.com/internal_error.html"

I could not set the permissions for the error.log higher than 644, but it is
being updated, so that can't be it. I couldn't find an suex log.

Have you looked for
any Dreamhost documentation on running Perl CGI scripts?

Yes. See above.

Then you haven't got CGI enabled for that directory, or the script isn't
executable, or .pl is mapped to something other than CGI, or...

Maybe that's something I must request from the ISP/host?

How did you execute it--by executing the file directly, or by running
perl script.pl? Were you doing this as the webserver user or as
yourself?

I logged on with my username and password. Then I went to the cgi-bin
directory and typed "first.pl". The header and text were printed as text
(stdout?)

This won't necessarily have any effect. It depends on how the server is
configured (which is OT for clpmisc).

I renamed .htaccess to dothtaccess and nothing changed.

That would be a good idea.

I just now got a reply on the forum, asking for more information. I can't
believe this should be so difficult. Thanks for your time. I'll report back
if and when I get this working. Maybe it will help someone else.

Paul

 

[Paul E. Schoen]

# EEEK!!!

Why EEEK?

I made a second test script and I executed it using telnet in my smart.net
host.

#!/usr/bin/perl
use strict;
use warnings;
use CGI;
my $query= new CGI;
open my $out, '>', "output.txt" or die "File error: $!";
print $out $query->header;
print $out "\r\nHello, World.\r\n";
close $out;

It produced the correct output.txt file:

Content-Type: text/html; charset=ISO-8859-1


Hello, World.

Mime-type, permissions, ownership, combination of.... sounds like cgi-bin
might be set up more or less correctly though.

Assuming the webserver thinks the script owner and user configured to run
the script match up. (IF SUEXEC is enabled. If it isn't: run screaming)


Sounds like cgi-bin is NOT configured correctly in httpd.conf.

Ah... did *you* create the cgi-bin directory or was it already there?

I created it in the dreamhost.com server, but it was already set up in
smart.net.

The above may or may not do any good depending on Server configuration.


Here is a link to a page you may find useful:

http://brian-d-foy.cvs.sourceforge.net/viewvc/brian-d-foy/CGI_MetaFAQ/CGI_MetaFAQ.html

And you really should set up a web server on your local box. The first
time you fire off a script with an infinite while(){} loop in it, your
providers will not be happy..... and it *will* happen.

I do have a localhost that got set up when I installed the DotCMS demo. The
home page appears when I use http://localhost/ in my browser. I don't know
how to set it up for a local copy of my website(s). It resides in C:\DotCMS.

Tomorrow I hope to hear from someone at smart.net about the setup there. I'm
using a free demo of www.dreamhost.com and I was happy to chat with someone
at 3 AM Sunday, but when I asked a technical question today the tech said it
was only for sales and tech questions are to be referred to the forum. It
seems like a really good deal for about $2 or $3 per month for unlimited
storage and domain name registration included, while my 100 MB on smart.net
is $99/year and my domains cost about $9/year from www.mydomain.com.

Thanks for the information. Maybe eventually I'll get something running and
then I can ask "real" perl questions. What I really want to do is set up a
website and be able to have specified users log on and contribute simple
content in a form for inclusion in HTML files. Basically a very limited CMS.

Paul

 

[Steve]

Syntax error.

Yep. You'd think I could cut-n-paste accurately. Apparently not.

Don't lie about the content type. Use text/plain if that's what you're
actually sending.

Nope. Makes no difference for what the OP is trying to do and when he
starts fiddling with printing out HTML it will just 'work'.

\s

 

[Steve]

Why EEEK?

I made a second test script and I executed it using telnet in my
smart.net host.

#!/usr/bin/perl
use strict;
use warnings;
use CGI;
my $query= new CGI;
open my $out, '>', "output.txt" or die "File error: $!";
print $out $query->header;
print $out "\r\nHello, World.\r\n";

^ ^
Just the \n is fine.

close $out;

It produced the correct output.txt file:

Content-Type: text/html; charset=ISO-8859-1


Hello, World.


I created it in the dreamhost.com server, but it was already set up in
smart.net.

Figured. I'm thinking your account is not set up (properly?) to allow
cgi-script execution. Have to talk to the dreamhost folks about that.

Thanks for the information. Maybe eventually I'll get something running
and then I can ask "real" perl questions. What I really want to do is
set up a website and be able to have specified users log on and
contribute simple content in a form for inclusion in HTML files.
Basically a very limited CMS.

Paul

The proverbial can o' worms. There are a *lot* of security issues to
deal with when accepting user input. The page I linked to in a previous
post contains some required reading.

If you're thinking about using some sort of database back end, read up
on SQL Injection attacks. Easy enough to prevent, (especially with
Perl) but a serious threat when accepting user input.

hth,

Steve

 

[Paul E. Schoen]

Figured. I'm thinking your account is not set up (properly?) to allow
cgi-script execution. Have to talk to the dreamhost folks about that.

I got a reply on the forum and the "code monkey" said my first.pl script had
a semicolon after the first line, and also the script had DOS-type line
endings. And my second.pl script would not work because it did not print to
the stdout.

But when I used their net2ftp application to edit the file, it apparently
used DOS type line endings, even when I used their "codegear" editor. And I
can't use telnet to log onto my account at dreamhost, so I suppose I'll have
to edit the script locally with a special UNIX compatible editor and then
upload to the site. That may even be preferable so that my local copy
matches what's on the remote server.

I just heard back from smartnet/sovenix, and it was a problem with the
setup. It's odd because the cgi-bin directory was included, but I've had
that account since 1996 and this is the first time I've used the cgi-bin
directory. But now my script runs OK. However, I also had to access the CGI
by using the URL http://www.smart.net/pstech-cgi-bin/first.pl, rather than
http://www.smart.net/~pstech/cgi-bin/first.pl, which still tries to download
the script. Actually that might be handy.

The dreamhost server must be set up differently, as I can use
http://www.pauleschoen.com/cgi-bin/first.pl. Now I can go nuts playing with
server side scripts. The Sovenix/smartnet guy said I can run the scripts
from a text based lynx browser using telnet, and I can use Ctrl-C to escape
from wild code. He said the server is already set up to limit CPU or
execution time to about 2 minutes.

I rewrote the second.pl script as follows:

#!/usr/bin/perl
use strict;
use warnings;
use CGI;
my $query= new CGI;
print $query->header;
print "\nStarting\n\n";
open my $out, '>', "output.txt" or die "File error: $!";
print $out $query->header;
print $out "\nHello, World.\n";
close $out;
print $query->header;
print "\nFinished!\n";

Running it on Telnet shows both headers and their text. But if run on the
browser it only shows "Starting". Probably just shows how much I need to
learn!

The proverbial can o' worms. There are a *lot* of security issues to deal
with when accepting user input. The page I linked to in a previous post
contains some required reading.

If you're thinking about using some sort of database back end, read up on
SQL Injection attacks. Easy enough to prevent, (especially with Perl) but
a serious threat when accepting user input.

If I use a database backend such as MySQL, I'll probably use a canned CMS
such as Wordpress. It should have decent security set up. If I "roll my
own", I will probably just use a simple form with data that can be emailed
to me for review, and then I can update the local HTML file and upload it.
The only content that needs to be updated frequently is a list of activities
and events, which will be organized by fields for Date/time, Title, and
Description, which may include HTML links or images or other more
interesting content. Hopefully that will be secure enough.

Now I just need to get the log-on setup for the actual live Sierra Club
site.

Thanks a lot!

Paul

 

[Steve]

On 08/23/2010 03:34 PM, Paul E. Schoen wrote:

I rewrote the second.pl script as follows:

#!/usr/bin/perl
use strict;
use warnings;
use CGI;
my $query= new CGI;
print $query->header;
print "\nStarting\n\n";
open my $out, '>', "output.txt" or die "File error: $!";
print $out $query->header;
print $out "\nHello, World.\n";
close $out;
print $query->header;
print "\nFinished!\n";

Running it on Telnet shows both headers and their text. But if run on
the browser it only shows "Starting". Probably just shows how much I
need to learn!

Probably because of an error trying to open that file.

Now I just need to get the log-on setup for the actual live Sierra Club
site.

Thanks a lot!

Paul

You're welcome.

You've outlined a massive amount of work for yourself, but as long as
you're having fun.

\s

 

[Paul E. Schoen]

On 08/23/2010 03:34 PM, Paul E. Schoen wrote:


Probably because of an error trying to open that file.

Actually, running it in Telnet produces both headers and text, and the
"output.txt" file is updated. I wanted the browser to show "Starting" and
then "Finished" after the file had been written. It does the same thing as
edited above, without the second header. I thought it should show both lines
in the browser. So, how does the browser know that the text it is reading on
stdout has "finished"? Would it work if it actually generated HTML?

You're welcome.

You've outlined a massive amount of work for yourself, but as long as
you're having fun.

Yes, I have fun once I learn enough to get at least a simple script working.
Now I have made a form which POSTs data to a mailer.pl script I got from
http://www.jmarshall.com/easy/cgi/ and modified it to work with my sendmail
application on the Smartnet server. It works OK except that the ReplyTo
appears as "nobody". The perl script that invokes it is:

$mailprog= "/usr/bin/sendmail" ;
$recipient= "paul\@peschoen.com" ; # make sure to \ escape the @

# Get the CGI input variables
%in= &getcgivars ;

open(MAIL, "|$mailprog $recipient")
|| &HTMLdie("Couldn't send the mail (couldn't run $mailprog).") ;

# Print the header information
$ENV{'HTTP_REFERER'} || ($ENV{'HTTP_REFERER'}= "www.peschoen.com") ;
print MAIL "Subject: Form data from the Web\n\n",
"The following data was entered at $ENV{'HTTP_REFERER'}:\n\n" ;

There is another form emailer that can be called from the dreamhost server
http://formmail.dreamhost.com/cgi-bin/formmail.cgi and it uses what I set as
"email" in the form data.

I have installed an editor http://www.indigostar.com/perledit.php that seems
pretty good, and I've made it my default for perl script files. It can even
edit the remote file directly using FTP.

OK, my fun is done for the day. But here is an outline of what I intend to
do:

1. Have an HTML document that will read the contents of another HTML file
which contains dates, titles, and descriptions of events. It should list
these entries in order of date, and possibly delete events that are older
than the present date, or put them in a "Past Events" file.

2, Have a form that will allow a user to enter a new event, and POST this to
a CGI script that will format the data and add it to the file. Then the
updated file should appear in the document. There should be some sort of
password protection and also a check for duplicate entries, although
duplicate dates may be allowed.

3. The events list will appear in the home page www.smart.net/~pstech/SCGBG,
which will remain mostly static except for changes I will make. The
important thing is to have other members be able to update events as needed.

4. All changes to the list will be automatically emailed to me so I know
what's going on and who is doing what.

I think I can do this! But that will be for another day...

Thanks for putting up with a newbie.

Paul and Muttley
www.peschoen.com
www.muttleydog.com

 

[RedGrittyBrick]

I find it is a mistake to get into the habit of lying to software about
data types. What is harmless today may trip you up tomorrow. Cultivating
good habits should save you grief in the long run.


Which is why I usually redirect errors to the HTTP client.
use CGI::Carp qw(fatalsToBrowser);

Actually, running it in Telnet produces both headers and text, and the
"output.txt" file is updated. I wanted the browser to show "Starting"
and then "Finished" after the file had been written. It does the same
thing as edited above, without the second header. I thought it should
show both lines in the browser. So, how does the browser know that the
text it is reading on stdout has "finished"? Would it work if it
actually generated HTML?

I believe the web-server (e.g. Apache) is waiting for your CGI script to
exit before it *starts* sending a reply to the web-browser. So any
"Started" and "Finished" messages will arrive simultaneously and only
after the output.txt file is written and closed.

 

[Scott Bryce]

Actually, running it in Telnet produces both headers and text, and
the "output.txt" file is updated. I wanted the browser to show
"Starting" and then "Finished" after the file had been written. It
does the same thing as edited above, without the second header. I
thought it should show both lines in the browser. So, how does the
browser know that the text it is reading on stdout has "finished"?
Would it work if it actually generated HTML?

Well, you are sending headers to the browser telling the browser that it
is being sent HTML, but what you are sending is not HTML. What the
browser does with the text you send it is not predictable.

BTW, you will want to include a line in your script, somewhere before
the first print statement, that reads:

$|=1;

Perl buffers output. Your browser is likely to get the "Starting" and
"Finished" at the same time after the file write on the server. The line
above will tell Perl to flush the buffer after every print statement, so
that the "Starting" will get to the browser before the file write. What
the browser does with it at that time is entirely up to the browser.

 

[Steve]

Actually, running it in Telnet produces both headers and text, and the
"output.txt" file is updated. I wanted the browser to show "Starting"
and then "Finished" after the file had been written. It does the same
thing as edited above, without the second header. I thought it should
show both lines in the browser. So, how does the browser know that the
text it is reading on stdout has "finished"? Would it work if it
actually generated HTML?

This is starting to sound like Apache running SUEXEC where there is a
permissions/ownership conflict somewhere.

You might look at:

http://httpd.apache.org/docs/2.0/suexec.html

which will outline how SUEXEC works.

I'm wondering if cgi-bin is owned by someone else.

Oh.

It is almost always a bad idea to write a file to cgi-bin or any
directory your server considers a script enabled directory.

hth,

\s

 

[Peter J. Holzer]

There is another form emailer that can be called from the dreamhost server
http://formmail.dreamhost.com/cgi-bin/formmail.cgi and it uses what I set as
"email" in the form data.

Ouch. That looks a lot like Matt Wright's infamous formmail script
(especially the "send to any email address the user cares to enter"
feature/bug). Don't use this and tell dreamhost to get rid of it. Once
upon a time this was a major spam vector (by now most spammers have
probably forgotten that it ever existed).

hp

 

[Peter J. Holzer]

As others already suggested, your script probably dies after the failed
open. You should see the error message in the error log. It is possible
(even likely) that the script doesn't have permission to write to the
current directory when it is started by the webserver.

The browser knows that the text has finished when the web server closes
the connection or when it has read as many bytes as the server indicated
(via a content-length header or chunked encoding).

No.


Well, you are sending headers to the browser telling the browser that it
is being sent HTML, but what you are sending is not HTML.

This is true only in a very strict sense. Many tags in HTML are
optional, so

---%<------%<------%<------%<------%<------%<------%<------%<---
Starting ...


Finished
---%<------%<------%<------%<------%<------%<------%<------%<---

is exactly equivalent to
---%<------%<------%<------%<------%<------%<------%<------%<---
<html>
<head>
</head>
<body>
Starting ...


Finished
</body>
</html>
---%<------%<------%<------%<------%<------%<------%<------%<---

which still is not conforming because the title element is missing
(which strangely is the only element in HTML which is not optional and
not implied, but afaik there isn't any browser which will complain about
a missing title element) and also because of the missing doctype (but
that only means the the browser doesn't know the exact HTML version).

What the browser does with the text you send it is not predictable.

For a standards-conforming browser it is predictable: It will display a
single paragraph with the text "Starting ... Finished". Of course IE may
do something different .

hp

 

[Peter J. Holzer]

It might and it might not.

A CGI program's output goes to the *web server*. The server may
decide to pass output through right away, or it may decide to wait
until all of the output is available anyway.

True. But without the $| = 1; the first line won't even get to the web
server before the script finishes, so the web server doesn't get a
chance to pass it on.

hp

 

[C.DeRykus]

It might and it might not.

A CGI program's output goes to the *web server*. The server may
decide to pass output through right away, or it may decide to wait
until all of the output is available anyway.
...

IIRC, a no-parsed header (NPH) script bypasses the server's
header parsing/insertions as well as its buffering. One of
the key NPH script advantages is a scenario in which backend
processing is slow and you want to display unbuffered results
to the client more quickly. The server has to be configured
for NPH scripts though.

 

[James]

I am trying to set up a webspace so I can use CGI, and I'm getting errors
and/or it is not behaving as it should using IE8 or Firefox. I read the FAQ

Modify conf/httpd.conf as

DocumentRoot "whatever"
Options FollowSymLinks
Options All

and restart apache.

JL

 

[Paul E. Schoen]

I am trying to set up a webspace so I can use CGI, and I'm getting errors
and/or it is not behaving as it should using IE8 or Firefox. I read the
FAQ

Modify conf/httpd.conf as

DocumentRoot "whatever"
Options FollowSymLinks
Options All

and restart apache.
-----------------------------------------------

I think that's what the administrator at Smart.net did at my request. I also
had to change the way I refer to files in cgi-bin. The main home page is
www.smart.net/~pstech, and for subdomains I simply added directories with
/subdir. But for the CGI scripts I have to use www.smart.net/pstech-cgi-bin
even though the actual directory is /cgi-bin.

Thanks for the information, but I have things under control, at least until
I start writing some non-trivial code where I'll have the opportunity of
making non-trivial mistakes

Paul (the 61 year old perl virgin)