Forum Moderators: open

Message Too Old, No Replies

Maybe a spider? "Crawl_Application"

Has not been brought up for a few months.

         

werty

3:57 pm on Jul 9, 2003 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Checked the logs today and got hit by this:

129.34.20.16 - - [09/Jul/2003:07:52:14 -0700] "GET /xxxxxxxx.html HTTP/1.1" 200 12434 "-" "Crawl_Application"
129.34.20.16 - - [09/Jul/2003:07:53:05 -0700] "GET /xxxxxxxx1.html HTTP/1.1" 200 10681 "-" "Crawl_Application"

Stayed for less than a minute only hit those two pages, did not request robots.txt.

Did a search on here and only came up with 2 posts from 2001 and 2002.

fiestagirl

5:48 pm on Jul 9, 2003 (gmt 0)

10+ Year Member



It's an experimental crawler from IBM. Been around for a while - no clues as to what they will be doing with the results yet..