New Spider? statcrawler (at) gmail.com - Crawler, Spider, and User Agent ID forum at WebmasterWorld - WebmasterWorld

Forum Moderators: open

Message Too Old, No Replies

New Spider? statcrawler (at) gmail.com

no url given

uncle_bob

6:38 pm on Oct 2, 2004 (gmt 0)

10+ Year Member

Noticed a new robot, using slightly different ua for robots.txt requests to index page requests


robots.txt: stat (statcrawler@gmail.com) 
index.html: stat statcrawler@gmail.com

IP resolves to SFO BRIDGED CIRCUITS

volatilegx

12:57 am on Oct 3, 2004 (gmt 0)

WebmasterWorld Senior Member

10+ Year Member

see [webmasterworld.com...]