Forum Moderators: open

Message Too Old, No Replies

SindiceFetcher/sig.ma

         

keyplyr

8:55 am on Aug 30, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Took one secondary html page. No request for robots.txt (this visit anyway) even though their info page says they support it.

140.203.155.** - - [29/Aug/2009:17:57:58 -0700] "GET www.example.com/page.html HTTP/1.1" 200 8617 "-" "SindiceFetcher/sig.ma (http://sindice.com/developers/bot)"

keyplyr

9:20 am on Aug 30, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Came back a few minutes later, fetched same page from different D class and different UA:

SindiceFetcher/Ping Manager (http://sindice.com/developers/bot)