| Welcome to WebmasterWorld Guest from 126.96.36.199 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Pubcon Platinum Sponsor 2014|
im getting lots of visits from the inktomi bot.
it keeps requesting for the robots.txt
what should i add in the robots file to make inktomi index my pages?
You don't need to add anything (or even have a robot.txt). Just make sure that if you do have a robot.txt it's written properly and not blocking spiders.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved