Page is a not externally linkable
youfoundjake - 1:00 am on Nov 16, 2007 (gmt 0)
User-agent: * not that complex of a file but for what ever reason, but i get this error... Possible problem with their validator?
I did determine that 5 green boxes is the max, as I have one of my domains which is only 1 green box, the other 4 are white.
Whats odd about the robots.txt validator is that it gives errors, when every other place says it passes fine
User-agent: Slurp
User-agent: Googlebot
User-agent: msnbot
User-agent: Mediapartners-Google
User-agent: Adsbot-Google
User-agent: Googlebot-Image
User-agent: Googlebot-Mobile
User-agent: Teoma
User-agent: ia_archiver-web.archive.org
Disallow: /keepout/
Disallow: /goaway/
Disallow: /
Error: MSNBOT isn't allowed to crawl the site.
**************************************************
Line #2: User-agent: Googlebot
Error: 'user-agent' tag should be followed by a different tag.
**************************************************
Line #3: User-agent: msnbot
Error: 'user-agent' tag should be followed by a different tag.
**************************************************
Line #4: User-agent: Mediapartners-Google
Error: 'user-agent' tag should be followed by a different tag.
**************************************************
Line #5: User-agent: Adsbot-Google
Error: 'user-agent' tag should be followed by a different tag.
**************************************************
Line #6: User-agent: Googlebot-Image
Error: 'user-agent' tag should be followed by a different tag.
**************************************************
Line #7: User-agent: Googlebot-Mobile
Error: 'user-agent' tag should be followed by a different tag.
**************************************************
Line #8: User-agent: Teoma
Error: 'user-agent' tag should be followed by a different tag.
**************************************************
Line #9: User-agent: ia_archiver-web.archive.org
Error: 'user-agent' tag should be followed by a different tag.
**************************************************
Warning: 'sitemap' - tag isn't specified.
**************************************************
Because i do see them crawling and indexing my sites.