| Welcome to WebmasterWorld Guest from 188.8.131.52 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Pubcon Platinum Sponsor 2014|
I haven't looked at my robots.txt in years, and it suddenly dawned on me that I should check it out:
User-agent: * # directed to all spiders, not just Scooter
Am I inadvertandly excluding all robots with that first line???? Yipes!
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved