Page is a not externally linkable
- Search Engines
-- Ask - Teoma
---- Ask Teoma bot crawling again


Angonasec - 12:28 am on Nov 24, 2010 (gmt 0)


We still specifically ALLOW the Ask Teoma bot for nostalgic reasons, and they haven't stopped crawling our site, but look at this weird UA, which got a speedy 403 of course...

204.236.147.79 - - [23/Nov/2010:15:30:56 -0500] "GET /robots.txt HTTP/1.0" 403 294 "-" "\"Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml)\""

They visited using that format and IP.

Same day they used this format and got in:

66.235.124.15 - - [23/Nov/2010:03:30:47 -0500] "GET /robots.txt HTTP/1.1" 200 1221 "-" "Mozilla/5.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml)"


Deliberate, ham-fisted, or desperate?


Thread source:: http://www.webmasterworld.com/ask_jeeves_teoma/4234253.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com