Welcome to WebmasterWorld Guest from 23.20.230.24

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

Bot, Robot, Spider, Crawler from Brazil

     
10:25 pm on Apr 27, 2014 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



This was so adorable, I had to share.

201.81.177.130 - - [25/Apr/2014:07:49:30 -0700] "GET /robots.txt HTTP/1.1" 200 779 "-" "WIRE/0.22 (Linux; x86_64; Bot,Robot,Spider,Crawler)"
201.81.177.130 - - [25/Apr/2014:12:48:13 -0700] "GET /robots.txt HTTP/1.1" 200 779 "-" "WIRE/0.22 (Linux; x86_64; Bot,Robot,Spider,Crawler)"


... and that was all she wrote. D'you think it might possibly be a robot?
3:45 am on Apr 28, 2014 (gmt 0)

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month



Covering all the bases.
3:59 am on Apr 28, 2014 (gmt 0)

WebmasterWorld Senior Member Top Contributors Of The Month



It's a human trying to make you think it's a robot. Don't be fooled. Write an algorithm to let her in. If I'm right, she is sleepwalking and will walk straight into a wall. Then you can sign her up for your newsletter and sell her a bottle of aspirin.
4:00 am on Apr 28, 2014 (gmt 0)

WebmasterWorld Administrator brotherhood_of_lan is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Perhaps a test to discover that 99.9% of sites would happily let that continue crawling.
2:36 pm on Apr 28, 2014 (gmt 0)

WebmasterWorld Senior Member Top Contributors Of The Month



Perhaps a test to discover that 99.9% of sites would happily let that continue crawling.


I was thinking of the exceptions I'd have to configure to be able to let that in for a crawl. ;)
7:31 pm on Jun 10, 2014 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



Hey, I just met its brother (URL lightly obfuscated because it's Brazil and might be an infected human machine):

177.32.69.abc - - [09/Jun/2014:17:26:02 -0700] "GET /robots.txt HTTP/1.1" 200 802 "-" "WIRE/0.22 (Linux; x86_64; Bot-Robot-Spider-Crawler)" 

Y'know, sometimes it's just so hard to figure out if something is a robot or not...
 

Featured Threads

Hot Threads This Week

Hot Threads This Month