- - Search Engines
- -- Sitemaps, Meta Data, and robots.txt
- ---- How to ban YahooSeeker?
sidyadav - 3:59 am on Jan 1, 2004 (gmt 0)
|All other crawlers obey my robots file, except yahooseeker. |
Thats not true, there are more than a 1000 bots out there that don't obey robots.txt. Only the good ones do.
YahooSeeker is used for Yahoo's product search service: [products.yahoo.com...] , if your site doesn't sell any products, you should ban it.
|Are they both Yahooseeker? |
They both come from the same batch of IP 66.196.93 which resolve to yj1028.inktomisearch.com, and since Yahoo owns Inktomi, They indeed belong to Yahoo.
If you want to block the whole YahooSeeker IPs, enter this code in your .htaccess file:
|order allow,deny |
allow from all
deny from 18.104.22.168
It should work.
BTW - Welcome to WebmasterWorld!
Thread source: http://www.webmasterworld.com/robots_txt/223.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com