Page is a not externally linkable
walrus - 6:40 pm on Nov 17, 2007 (gmt 0)
This one below finally validated but no matter what i do i cant get it to validate with the User-agent: Mediapartners-Google* It says cant use 2 user agents in same line, likeit is counting the asterisk as another agent. User-agent: * Asa matter of fact it said msn could not crawl my site. is this why i was pulledfromtheir index? This is what ive used the last 2 years, though it validated but msn says it cant crawl. Where is the error? User-agent: Mediapartners-Google* PS AAAAAAUUUGHGHHHH!
Went very smooth for me except for one crucial detail.....
Try validating your robots file. Been trying for one hour, no luck.
The only time it validated was when i took out User-agent: Mediapartners-Google*
Sitemap: [yoursite.com...]
User-agent: *
Disallow: /stats/
Disallow: /cgi-bin/
Disallow: /private/
Disallow: /demos/
Disallow: /tester/
Disallow:
User-agent:63.148.99.*
Disallow: /
User-agent:211.157.36.4
Disallow: /
User-agent:213.42.2.29
Disallow: /
User-agent:211.152.14.92
Disallow: /
User-agent:203.111.194.11
Disallow: /
User-agent:*
Disallow: /stats/
Disallow: /cgi-bin/
Disallow: /private/
Disallow: /demos/
Disallow: /tester/