Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

content removal via robot.txt

8:46 pm on Aug 17, 2003 (gmt 0)

New User

10+ Year Member

joined:July 18, 2003
votes: 0

I just notice inktomisearch bot index some area on my site that I don't want to. Now I changed the robot.txt to exclude that area, but what about the existing index that inktomi has? will they remove automatically based on the new robot.txt?
10:46 pm on Aug 17, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
votes: 0


I'd guess that Ink will remove your excluded pages faster if you are a paid subscriber. Otherwise, it may take awhile, possibly several months. It all depends on how often Slurp visits your site and re-indexes the disallowed pages.



Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members