Forum Moderators: open

Message Too Old, No Replies

T-h-u-n-d-e-r-s-t-o-n-e

         

wilderness

6:19 am on Feb 27, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



This bot has never been complaint, at least that I'm able to recall.
For the longest time it ran from a Road Runner IP.

70.90.215.8z - - [26/Feb/2008:23:51:43 -0600] "GET /robots.txt HTTP/1.0" 200 4549 "-" "Mozilla/2.0 (compatible; T-H-U-N-D-E-R-S-T-O-N-E)"

read robots (which has included the UA for an eternity and grabbed a single page.
No images.

The Class D website is active, although without any active pages.

keyplyr

7:32 pm on Feb 27, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



T-H-U-N-D-E-R-S-T-O-N-E has been around a long time, predating AltaVista.

There are several builds. One is a 3rd party hosted free site-search utility that I use on one of my sites. This one does follow the robots.txt standard and there is no way to alter that.

Other releases of this bot are packaged as a back-end directory builders similar to FAST and are available for public/cooperate purchase. Since this version is a commercial product, I would imagine the modules can be customized. I block this version since it has the potential to be used as a scraper.

wilderness

6:20 pm on Feb 29, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



unreal!

70.90.215.85 - - [29/Feb/2008:07:25:13 -0600] "GET /robots.txt HTTP/1.0" 200 4623 "-" "Mozilla/2.0 (compatible; T-H-U-N-D-E-R-S-T-O-N-E)"
70.90.215.85 - - [29/Feb/2008:07:25:14 -0600] "GET /MyFolder/MyPage.html HTTP/1.0" 403 - "-" "Mozilla/2.0 (compatible; T-H-U-N-D-E-R-S-T-O-N-E)"