error log entries

how to interpret error logs

[J64sqs]

I recently looked at my error logs, and I found the following errors repeated over and over again. What is the robots.txt file and comment.php file? And why would someone be requesting these pages repeatedly?

[Tue Sep 28 06:27:23 2004] [error] [client 207.46.98.52] File does not exist: /home/username/public_html/404.shtml
[Tue Sep 28 06:27:23 2004] [error] [client 207.46.98.52] File does not exist: /home/username/public_html/robots.txt
[Tue Sep 28 06:27:12 2004] [error] [client 66.249.65.242] File does not exist: /home/username/public_html/404.shtml
[Tue Sep 28 06:27:12 2004] [error] [client 66.249.65.242] File does not exist: /home/username/public_html/comment.php

The following error message only showed up once, and I'm really confused about what it means.

[Tue Sep 28 06:28:12 2004] [error] [client 207.46.98.52] attempt to invoke directory as script: /home/username/public_html/cgi-bin/pollscript

[Birdman]

Hello,

Robots.txt is an optional file to tell search engine crawlers which files/directories they are allowed to read.

404.shtml is the file your server redirects requests to when a file was not found(404).

To stop these errors, you can create a blank robots.txt file and put it in your root folder. Also, create a file named 404.shtml and put an appropriate message on that page with a link to your homepage.

As for the other errors, you probably have a couple broken links somewhere on your site.

Regards,
Birdman

[J64sqs]

I recently found another error continuously repeated in my error log. It says...

File does not exist: /home/username/public_html/favicon.ico

I'm completely lost as to why someone would be trying to find that path on my website again and again, as I have no idea what favicon.ico is. Does anyone else know?

[fiestagirl]

The favicon is shown in the address bar, in the bookmark list and in the tabs of some browsers. You can read about them here: [favicon.com...]

[cgrantski]

I don't know about pollscript, but your visitors will never see an error message because of the other two events. You don't have to have a robots.txt or favicon.ico file if you don't need one. Many, many sites have the same issue and don't know about it unless they look in their logs.

[webnerd]

favicon.ico is requested by the browser when someone is bookmarking your website. It is a very positive sign that people plan on returning to you. The move there are of them the merrier.

[balam]

favicon.ico is requested by the browser when someone is bookmarking your website.

100% correct, but that is not the only reason why a web browser may grab favicon.ico.

Originally, favicon.ico was a Microsoft creation for Internet Explorer. Websites could create their own custom icon that would appear beside your bookmark when you added their site to your "Favorites."

This made requests for favicon.ico (whether successful or not, based on whether you had an icon or not) a decent estimate of how many people were bookmarking your site (given the fact that most people use IE) - for awhile...

With the advent of "tabbed" browsers such as (newer versions of) Netscape, Opera & Firefox, tracking requests for favicon.ico as a metric for visitor bookmarking has become totally useless. (Many/most/all) of the tabbed browsers have caught onto the idea of using favicon.ico, and now grab the file (if it exists) as soon as you visit a page - whether you (plan to) bookmark the page or not. The icon is grabbed so that it may be displayed on the tab for that page.

With Internet Explorer slowly but steadily losing market share to these newer browsers, you can expect more & more of your visitors to be requesting this file.

[grandpa]

tracking requests for favicon.ico as a metric for visitor bookmarking has become totally useless

For the sake of argument, not so. It makes that metric more difficult to come by. You have to weed out all the non-IE requests and calculate from there. Once MS catches up with tabbed browsing, favicon.ico will become totally useless as a metric.

[balam]

For the sake of argument, not so. It makes that metric more difficult to come by.

Touche, grandpa, you are right. Indeed, at least one web stats program - AWStats - does filter out non-IE requests to supply an estimated number of bookmarking visitors.

(Since some of this thread has been a "favicon.ico 101" crash course, I sort of consider(ed) the filtering of non-IE requests as a "102 course" topic...)