homepage Welcome to WebmasterWorld Guest from 54.211.100.183
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
bloggsi
(incl. link to post-Tweet bot recap)
Pfui

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 4160364 posted 10:09 pm on Jun 27, 2010 (gmt 0)

bloggsi.com
-

robots.txt? NO

UA: None
Ref: None
Req: HEAD

Obviously bare-bones on one of my sites. But also travels using the above Host's IP and UA:

88.198.69.11*
Bloggsi/1.0 (http://bloggsi.com/)

robots.txt? Yes

Source:
Detailed, post-Tweet bot-by-bot observation/deconstruction a.k.a. --
A tale of a Tweet [java-community.de] (June 8, 2010)

P.S.
If that link gets axed for going to a blog, search for: A tale of a Tweet - Mike's Blog

 

windermann



 
Msg#: 4160364 posted 8:43 pm on Jul 1, 2010 (gmt 0)

what is this?

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4160364 posted 11:22 pm on Jul 1, 2010 (gmt 0)

what is this?


A "heads up".

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved