Forum Moderators: DixonJones

Message Too Old, No Replies

Lycos_Spider crawling again

Haven't seen this one in a long time

         

jdMorgan

6:44 am on Feb 28, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I just had my first visit from Lycos_Spider(mod_spider) in a *long* time -- 2 1/2 years, I'd guess.

I'm guessing that this is in response to the recent shake-ups in the search provider world -- With Fast's recent sale of AllTheWeb to Overture/Yahoo!, and Fast's apparently-sole focus on enterprise search, Lycos may now have to re-develop their own search index.

Anyway, the first log entry was:
209.202.2**.* - - [27/Feb/2004:23:14:55 -0600] "GET /robots.txt HTTP/1.0" 200 2335 "-" "Lycos_Spider_(modspider)"

Jim

<added> And no, I don't know how to make sense of this in light of recent Lycos news: [webmasterworld.com...] </added>

keyplyr

8:44 am on Feb 28, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



They've come by my place a couple times also.

pendanticist

3:12 pm on Feb 28, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



209.202.***.* - - [27/Feb/2004:20:23:27 -0800] "GET /robots.txt HTTP/1.0" 200 1565 "-" "Lycos_Spider_(modspider)"
209.202.***.* - - [27/Feb/2004:20:23:28 -0800] "GET / HTTP/1.0" 403 480 "-" "Lycos_Spider_(modspider)"

Looks like an existing ban is in place.

What with all these new and or re-'mastered' bots coming around it might be prudent for folks to check traps and re-evaluate bans.

Congrats on going over four too Jim. Excellent work. :)