homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Gold Sponsor 2015!
Home / Forums Index / Search Engines / Ask - Teoma
Forum Library, Charter, Moderator: open

Ask - Teoma Forum

does ask read robots.txt?

WebmasterWorld Senior Member 10+ Year Member

Msg#: 553 posted 6:48 am on Jun 28, 2006 (gmt 0)

on ask.com faq I found this:

User-agent: Teoma
Disallow: /

, added it to my robots.txt (months ago), but they still read my website.

I really like the SE, but 0.006% of traffic sent from ask IMO doesn't justify them using my bandwidth over and over again.

I traced the spider-IP address and it traces back to some ask-address.

Any idea - apart from IP-Blocking and the other things meant for the bad boys?


Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Ask - Teoma
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved