| Welcome to WebmasterWorld Guest from 126.96.36.199 |
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
|Pubcon Platinum Sponsor 2014|
| 2:39 pm on Apr 15, 2003 (gmt 0)|
im getting lots of visits from the inktomi bot.
it keeps requesting for the robots.txt
what should i add in the robots file to make inktomi index my pages?
| 2:58 pm on Apr 15, 2003 (gmt 0)|
You don't need to add anything (or even have a robot.txt). Just make sure that if you do have a robot.txt it's written properly and not blocking spiders.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved