helenp - 12:21 am on Jan 18, 2004 (gmt 0)

Thanks
Trying to read the rawlog, because everytime I saw inktomi in my stats, (not to good) it always stated my robots.txt file.
As far as I can see, reading it, itīs dificult without a program, it stops at robots.txt.....?

Thinking maybe something is wrong with my robotfile...........
this is the text in it:
User-agent: *

Disallow: /404.shtml
Disallow: 404.shtml
Disallow: espanol/404.shtml
Disallow: /espanol/404.shtml
Disallow: svenska/404.shtml
Disallow: /svenska/404.shtml
Disallow: /cgi-bin/
Disallow: /scgi-bin/

and here were inktomi explain about the robots file: http://www.inktomi.com/slurp.html
donīt understand to much, but I donīt think I done anything incorrect?