homepage Welcome to WebmasterWorld Guest from 54.167.177.180
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Clushbot/3.1-BinaryFury
The newest Clush bot not checking robots.txt
pendanticist

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 2706 posted 1:45 am on Jan 15, 2005 (gmt 0)

209.249.80.242 - - [14/Jan/2005:01:16:20 -0800] "GET / HTTP/1.0" 200 20407 "-" "Clushbot/3.1-BinaryFury (+http//www.clush.com/bot.html)"

209.249.80.242 - - [14/Jan/2005:05:39:56 -0800] "GET / HTTP/1.0" 200 20407 "-" "Clushbot/3.1-BinaryFury (+http//www.clush.com/bot.html)"

[webmasterworld.com...]

[webmasterworld.com...]

((These are the only two threads I could find. I know there was at least one more, ( because I participated in it ) but Google isn't picking it up.))

At any rate, no robots.txt.

 

pendanticist

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 2706 posted 4:00 pm on Jan 28, 2005 (gmt 0)

Three Version updates in just a day.

209.249.80.242 - - [27/Jan/2005:20:21:11 -0800] "GET / HTTP/1.0" 200 20407 "-" "Clushbot/3.7-BinaryFury (+http//www.clush.com/bot.html)"

209.249.80.242 - - [27/Jan/2005:23:58:36 -0800] "GET / HTTP/1.0" 200 20407 "-" "Clushbot/3.8-BinaryFury (+http//www.clush.com/bot.html)"

209.249.80.242 - - [28/Jan/2005:06:04:37 -0800] "GET / HTTP/1.0" 200 20407 "-" "Clushbot/3.9-BinaryFury (+http//www.clush.com/bot.html)"

These are the fastest version changes that I have ever seen.

Still not checking robots.txt though.

wkitty42

10+ Year Member



 
Msg#: 2706 posted 8:05 pm on Feb 1, 2005 (gmt 0)

FWIW: i dropped by the clush website and spoke with their livehelp folk... their /bot.html file doesn't state the name of the bot to use in the robots.txt file... their livehelp folk told me to use "clushbot" sans quotes... i mentioned that it had been seen in three+ different versions in one day and they responded that it was evolving ;)

pendanticist

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 2706 posted 9:49 pm on Feb 1, 2005 (gmt 0)

Thanks wkitty42. :)

They didn't happen to indicate if they're ever going to go the robots.txt route did they?

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved