Forum Moderators: coopster & phranque

Message Too Old, No Replies

Perl Newbie Question

Hrmph

         

Cynthia Blue

6:41 pm on May 16, 2005 (gmt 0)

10+ Year Member



I am attempting to get a .cgi script running on my webserver at work. I apologize in advance for being such a newbie, but I've never worked in Perl before, and my knowledge of servers is limited.

I'm assuming that Perl scripts come in .cgi and .pl files. Is this correct?

I'm getting a 500 Internal Server Error. This says to me that the .cgi file is being recognized, but is not executing properly. I tried putting basic HTML into a file, saving it as test.cgi, and loading it onto the server. No Perl code in there. And still I get the 500 ISE error. Is this a valid test?

The server admin has done a perl -v, and he tells me Perl v5.8.0 is installed on the server. So it is installed.

I've tried changing the permissions on the file to 755 so that it'll execute. I've made sure it's being FTPd to the server in ASCII mode instead of Binary.

As for the http.conf file, the directory that it specified in the ScriptAlias configuraiton is not a directory I have write access to, but I can see the files there. I can also see http.conf, but I cannot change it.

ScriptAlias /cgi-bin/ "/usr/local/apache2/cgi-bin/"

<Directory "/usr/local/apache2/cgi-bin">
AllowOverride None
# Options None
Options +ExecCGI
Order allow,deny
Allow from all
</Directory>

I do want the ScriptAlias to be changed to a subdir I have access too, but the server admin is gone at the moment, probably out to lunch.

Anyway, I would love more ideas. I've spent 2 days googling this and haven't been able to fix it.

Thanks!

moltar

7:45 pm on May 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I'm assuming that Perl scripts come in .cgi and .pl files. Is this correct?

Correct. Those are the two primary extensions that are used for Perl scripts. Conventionally .cgi is used for the web, and .pl is used for shell. It really does not matter which one you use though. The server can be configured to parse any extension as Perl file. You can have .xyz if you wish.

I'm getting a 500 Internal Server Error.

Have you printed the Content-type line first? You need to specify which content type you are going to use for your output. For text and html you should state the following before you print anything:

print "Content-type: text/html\n\n";

wruppert

7:48 pm on May 16, 2005 (gmt 0)

10+ Year Member



The suffix does not matter. Make sure the first line is "#!/usr/bin/perl -wT" or whatever is appropriate for your site so that perl will be used. Make sure the program compiles - "perl -c prog.pl". If possible, try to run it from the command line to see if it outputs some html.

Try a very simple program to make sure the cgi-bin is setup correctly, something like:

#!/usr/bin/perl -wT
print "Content-type: text/plain\n\n";
print "Great, the cgi-bin environment is working!";

Look at server error log, it may contain useful error messages. On my FreeBSD system it is at /var/log/httpd-errors.log.

I just use a ScriptAlias to point cgi-bin to the right place, past that I'm not too good at configuration stuff.

Lord Majestic

7:48 pm on May 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Before you output anything consider switching off buffering:

$¦ = 1; # don't buffer (to avoid server error on Unix)

Please, please use correct CRLF line endings, not just \n's:

print "Content-type: text/html\r\n\r\n";

CRLF is required part of standard, here is quote:

"HTTP/1.1 defines the sequence CR LF as the end-of-line marker for all protocol elements except the entity-body (see appendix 19.3 for tolerant applications)."

While RFC recommends HTTP clients to be lax, its a good idea to stick to standard.

[edited by: Lord_Majestic at 7:55 pm (utc) on May 16, 2005]

moltar

7:53 pm on May 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Add this as your first line of code to see the run time errors on the screen.

use CGI::Carp 'fatalsToBrowser';

moltar

8:00 pm on May 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Please use correct CRLF line endings, not just \n's:

That is wrong.

\n
contains the correct new line character sequence on each platform.

From perlport man (Writing portable Perl):

In most operating systems, lines in files are terminated by newlines. Just what is used as a newline may vary from OS to OS. Unix traditionally uses \012, one type of DOSish I/O uses \015\012, and Mac OS uses \015.

Perl uses \n to represent the logical newline, where what is logical may depend on the platform in use. In MacPerl, \n always means \015. In DOSish perls, \n usually means \012, but when accessing a file in text mode, STDIO translates it to (or from) \015\012, depending on whether you’re reading or writing. Unix does the same thing on ttys in canonical mode. \015\012 is commonly referred to as CRLF.


...

A common misconception in socket programming is that \n eq \012 everywhere. When using protocols such as common Internet protocols, \012 and \015 are called for specifically, and the values of the logical \n and \r (carriage return) are not reliable.

print SOCKET "Hi there, client!\r\n"; # WRONG
print SOCKET "Hi there, client!\015\012"; # RIGHT

Lord Majestic

8:03 pm on May 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



That is wrong. \n contains the correct new line character sequence on each platform.

On Unix \n is not CRLF but LF, where as HTTP requires CRLF specifically, so for Unix that the original poster appears to be using \r\n is necessary and it will produce the same result as \015\012 -- using octets is more portable though

I almost conceded defeat but then I realised I am right afterall :)

moltar

8:06 pm on May 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



so for Unix that the original poster appears to be using

I don't see where the poster specified the platform...

In any case, if perl documentation tells us to use \015\012 then we should do so. Perl developers know better.

Even I made the mistake in my first post. ;) To be honest, I always use \n\n.

Lord Majestic

8:09 pm on May 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



3 pointers to platform being some flavour of Unix:

1) usage of chmod
2) directory for Apache is /usr/local etc
3) he mentioned he used ASCII mode rather than binary for FTPing -- it matters when transferring from Win to Unix that needs line ends to be fixed

:)

To be honest, I always use \n\n.

Yes and thats the problem on Unix because \n will just be LF where as HTTP spec requires CRLF, not just LF, even though they recommend that client software to be lax and expect LF's -- I had to change my HTTP library just now to take this into account as it became apparent to me that its easy to make mistake and browsers are just very forgiving HTTP clients :)

[edited by: Lord_Majestic at 8:12 pm (utc) on May 16, 2005]

moltar

8:10 pm on May 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



To make it even easier, we should just stick to using CGI.pm


use CGI;
my $cgi = CGI->new();
print $cgi->header();

moltar

8:19 pm on May 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Just checked CGI.pm to make sure the correct syntax is used and found out that the problem is even more complex than I thought it was. This is a quote from CGI.pm code:

# Define the CRLF sequence. I can't use a simple "\r\n" because the meaning 
# of "\n" is different on different OS's (sometimes it generates CRLF, sometimes LF
# and sometimes CR). The most popular VMS web server
# doesn't accept CRLF -- instead it wants a LR. EBCDIC machines don't
# use ASCII, so \015\012 means something different. I find this all
# really annoying.
$EBCDIC = "\t" ne "\011";
if ($OS eq 'VMS') {
$CRLF = "\n";
} elsif ($EBCDIC) {
$CRLF= "\r\n";
} else {
$CRLF = "\015\012";
}

Lord Majestic

8:22 pm on May 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Yeah its more complicated than I thought too :)

Cynthia Blue

8:32 pm on May 16, 2005 (gmt 0)

10+ Year Member



Thanks so much for the help.

For a moment there, we got it working, with the small test .cgi file as you stated. But now, it's broken again. =P Not sure why.. I changed some file permissions, but I swear that can't be only why. Gave everything in the subdir a 0775, and then it stopped.

Drawback of not having direct access to the server, he changes something, I change something, and we cross our changes.

Grrr.. what a pain. I almost give up. :p

Cynthia Blue

8:54 pm on May 16, 2005 (gmt 0)

10+ Year Member



One more thing.

There are two ScriptAlias tags in the httpd.conf file now... not sure if they were there earlier. Will one overwrite the other? Or can they both work.

ScriptAlias /FIS/adv3x/dev/cgi-bin/ "/home/FIS/public_html/adv3x/dev/cgi-bin/"
<Directory "/home/FIS/public_html/adv3x/dev/cgi-bin/">
AllowOverride None
Options +ExecCGI
Order allow,deny
Allow from all
</Directory>

and

ScriptAlias /cgi-bin/ "/usr/local/apache2/cgi-bin/"
<Directory "/usr/local/apache2/cgi-bin">
AllowOverride None
##### Options None
Options +ExecCGI
Order allow,deny
Allow from all
</Directory>

Or what happens if you delete that subdir, and then recreate it? I would think it wouldn't really matter...

It is really really really really bugging me that this is being such a pain. Especially since it worked for five minutes, and now it is not working again. :(

It is just displaying the file as text, not running it as a script. :(

wruppert

12:21 am on May 17, 2005 (gmt 0)

10+ Year Member



On the "\n\n" question...

The O'Reilly book "CGI Programming with Perl, 2nd Edition" on page 53 has a box titled "Line Endings" that discusses this issue, explaining why they use "\n\n" in their examples.

They summarize the whole OS difference thing, the fact that CRLF is required, etc, and then state:

"CGI requires that the web server translate your operating system's conventional line ending into a CRLF for you. Thus for the sake of portability, it is always best practice to print a simple line feed ("\n"): Perl will output the operating system's default line ending, and the web server will automatically convert this to the CRLF required by HTTP."

Cynthia Blue

7:11 pm on May 17, 2005 (gmt 0)

10+ Year Member



Just to answer my own question... I spent some time on google and found the answer.

When, in httpd.conf, there are Alias as well as ScriptAlias, the order in which they are listed matters. I think my server admin put the ScriptAlias first, and viola, it worked. (Might be Alias first I forget, but either way, if anyone cares in the future, the order matters.)

Lord Majestic

7:28 pm on May 17, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



CGI requires that the web server translate your operating system's conventional line ending into a CRLF for you

What's the source of that? First time I hear that it's webserver's responsibility to ensure headers printed by a CGI script have proper endings.

wruppert

8:41 pm on May 17, 2005 (gmt 0)

10+ Year Member



Lord Majestic -

The source is in the original post - O'Reilly's CGI Programming with Perl, 2nd edition, p53. I have no idea what their source is - I generally take O'Reilly books as being authoritative.

I went looking because I was curious where I got the idea to use "\n\n" - I was copying how they do it.

Lord Majestic

8:42 pm on May 17, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I suppose if web server is indeed is responsible for making sure that line endings in headers are conforming with HTTP spec then its okay :)

moltar

8:40 am on May 18, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Indeed! The official Common Gateway Interface Documentation [hoohoo.ncsa.uiuc.edu] on the CGI Script Output [hoohoo.ncsa.uiuc.edu] page states:

The output of scripts begins with a small header. This header consists of text lines, in the same format as an HTTP header, terminated by a blank line (a line with only a linefeed or CR/LF).

If I understand correctly we can use LF or CRLF, but not CR (Macintosh).

gnotellaluvr

2:24 am on May 24, 2005 (gmt 0)

10+ Year Member



I agree with Moltar. If you have access to this little tool:

use CGI::Carp 'fatalsToBrowser';

then you will be much better off. It has helped me many times to get a feeling for the area where the problem was happening rather than just a "woops it just died" type message.

jatar_k

4:11 pm on May 24, 2005 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



Welcome to WebmasterWorld gnotellaluvr