Forum Moderators: goodroi
YahooSeeker is still crawling my site like crazy!
the ip is:
66.196.93.32 I checked this ip, it never read robots file.
the log file is like:
YahooSeeker/1.0+(compatible;+Mozilla+4.0;+MSIE+5.5;+http://help.yahoo.com/help/us/shop/merchant/) - 200 335
but another ip: 66.196.93.5 reads the robots file then left.
the log file is like:
HTTP/1.0 YahooSeeker/1.0+(compatible;+Mozilla+4.0;+MSIE+5.5;+http://search.yahoo.com/yahooseeker.html) - 200 233
Are they both Yahooseeker?
Anyone met this before and any suggestions is really appreciated.
BTW, since my site doesn't sell any products, there should be no negative effects on my site if I ban yahooseeker?
Thank you all, Happy new year!
Feng
All other crawlers obey my robots file, except yahooseeker.
YahooSeeker is used for Yahoo's product search service: [products.yahoo.com...] , if your site doesn't sell any products, you should ban it.
Are they both Yahooseeker?
How to ban YahooSeeker?
order allow,deny
allow from all
deny from 66.196.93.32
It should work.
Sid
BTW - Welcome to WebmasterWorld!