Robots.txt Error in Stats

But don't have a robots.txt?

[invisible]

Hi

When looking at my error pages in my basic stats I see two error pages.

1 example.com/robots.txt 189 57.62% (Page Views/Percentage)
2 example.com/favicon.ico 139 42.38%

As far as I know I don't have a robots.txt file as I do not want to exclude any robots. As for No. 2 I have no idea what this is.

Is this a problem and what do I need to do to stop these error pages. Any help would be much appreciated.

Thanks

[Philosopher]

Those are simply requests that are being made for those files.

All search engine spiders will request a robots.txt even if you don't have one. The only way they will know you don't have one and don't have any special restrictions is to try and request it.

They will generally make a request for it before each crawl in case you have added one since the last time.

The favicon.ico reqeust is for a icon file some sites have. It creates the little picture next to the URL in the address bar. Many sites use it to help stand out.

Many browsers automatically request that file when accessing your site. If you don't have one, it generates a 404 error.

Nothing to worry about on either one. perfectly normal.

[edited by: Philosopher at 4:00 pm (utc) on Aug. 29, 2006]

[stapel]

I will guess that the "error page" in question is the "404 Page Not Found" error.

The first log listing just mean that robots looked for a robots.txt file for instructions, but didn't find one.

Note: The robots.txt file is not only for exluding bots from your site, and bad-bots don't obey it anyway. There is no harm in having a robots.txt file. Since good-bots look for this file, you might want to have it, even if it's blank. And this will prevent this error from showing up.

The second log listing means that people bookmarked your site in Internet Explorer. The browser looked for an icon to include with the "favorites" listing, but didn't find one.

Neither error is a "problem". You don't have these two files, so user-agents receive a 404 reply when they look for them. This is the correct response. To prevent this response, create and upload the two files.

(For specific information and instructions, do searches on "robots.txt" and "favicon".)

Eliz.

[mack]

you might want to have it, even if it's blank

I agree, When I don't have need for a robots.txt file on a site I still place a blank file there. It reduces the number of 404 error messages and makes my log files a lot easer to work with.

Mack.