Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Dumbest question eva: Do all major SEs support the Google guidelines?

I am testing mine on G's WebmasterCentral

6:28 pm on Mar 26, 2008 (gmt 0)

Senior Member

joined:Dec 29, 2003
votes: 0

So I am testing my robots on Google's webmaster site and I got it down. Do I have to worry about other SEs or MSN, Y, Ask support the same standards? I know there was some controversy with wildcards, filenames etc so better safe than sorry. Thoughts, experiences?
6:42 pm on Mar 26, 2008 (gmt 0)

Junior Member

5+ Year Member

joined:Apr 19, 2007
votes: 0

Both Yahoo and MSN does not obey(sometimes) robots.txt. A few days before I put a bot-trap in my site, I disallowed the url in my robots.txt. But both of them are following those links sometimes and getting trapped.