Welcome to WebmasterWorld Guest from 54.145.13.215

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

Bot, Robot, Spider, Crawler from Brazil

     
10:25 pm on Apr 27, 2014 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12714
votes: 244


This was so adorable, I had to share.

201.81.177.130 - - [25/Apr/2014:07:49:30 -0700] "GET /robots.txt HTTP/1.1" 200 779 "-" "WIRE/0.22 (Linux; x86_64; Bot,Robot,Spider,Crawler)"
201.81.177.130 - - [25/Apr/2014:12:48:13 -0700] "GET /robots.txt HTTP/1.1" 200 779 "-" "WIRE/0.22 (Linux; x86_64; Bot,Robot,Spider,Crawler)"


... and that was all she wrote. D'you think it might possibly be a robot?
3:45 am on Apr 28, 2014 (gmt 0)

Moderator from US 

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:2570
votes: 48


Covering all the bases.
3:59 am on Apr 28, 2014 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Aug 1, 2013
posts:1295
votes: 8


It's a human trying to make you think it's a robot. Don't be fooled. Write an algorithm to let her in. If I'm right, she is sleepwalking and will walk straight into a wall. Then you can sign her up for your newsletter and sell her a bottle of aspirin.
4:00 am on Apr 28, 2014 (gmt 0)

Moderator from GB 

WebmasterWorld Administrator brotherhood_of_lan is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 30, 2002
posts:4842
votes: 1


Perhaps a test to discover that 99.9% of sites would happily let that continue crawling.
2:36 pm on Apr 28, 2014 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Aug 1, 2013
posts:1295
votes: 8


Perhaps a test to discover that 99.9% of sites would happily let that continue crawling.


I was thinking of the exceptions I'd have to configure to be able to let that in for a crawl. ;)
7:31 pm on June 10, 2014 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12714
votes: 244


Hey, I just met its brother (URL lightly obfuscated because it's Brazil and might be an infected human machine):

177.32.69.abc - - [09/Jun/2014:17:26:02 -0700] "GET /robots.txt HTTP/1.1" 200 802 "-" "WIRE/0.22 (Linux; x86_64; Bot-Robot-Spider-Crawler)" 

Y'know, sometimes it's just so hard to figure out if something is a robot or not...