Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- UNTRUSTED in Nokia User Agent


lucy24 - 12:51 am on Aug 20, 2012 (gmt 0)


1.202.aaa.bbb - - [18/Aug/2012:03:06:22 -0700] "GET /robots.txt HTTP/1.1" 200 874 "-" "Mozilla/5.0 ()"

Heck, why bother to put anything at all inside the parentheses? We both know you're a robot.

P.S. It followed this request by putting on :: yawn :: a simple jikespider costume, eating two 301's in quick succession, and then quietly leaving. Shrug.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4440895.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com