homepage Welcome to WebmasterWorld Guest from 54.204.94.228
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Fast's Mozilla spider
littleman

WebmasterWorld Senior Member littleman us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 199 posted 8:06 pm on Nov 29, 2000 (gmt 0)

Just in case you didn't know. It's been cralling a lot over the last 24 hours.
IP Address: 209.67.247.232
Hostname: rdcrawler3.uswal.alltheweb.com
User agent: Mozilla/4.0 (compatible; MSIE 4.01; Win32)

 

msgraph

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 199 posted 9:46 pm on Nov 29, 2000 (gmt 0)

Getting the same spider on about a 100 of my domains. All within a 10 minute period. Definitely not a common user.

Edited by: msgraph

msgraph

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 199 posted 9:46 pm on Nov 29, 2000 (gmt 0)

Getting the same spider on about a 100 of my domains. All within a 10 minute period. Definitely not a common user.

cirelle

10+ Year Member



 
Msg#: 199 posted 4:43 pm on Dec 7, 2000 (gmt 0)

had about 50 domains hit 1 each between 11-28 and 11-29 none since.

c

PeteU

10+ Year Member



 
Msg#: 199 posted 2:11 am on Jan 5, 2001 (gmt 0)

its at it again

littleman

WebmasterWorld Senior Member littleman us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 199 posted 2:18 am on Jan 5, 2001 (gmt 0)

Yes, it certainly is! Thanks for the headsup, Pete.

PeteU

10+ Year Member



 
Msg#: 199 posted 2:26 am on Jan 5, 2001 (gmt 0)

heh, Littleman, somehow I knew you'd be the first to reply ;)
seems like its a very organized crawl, its pulling alphabetically, all my hosts starting with 'a' are being requested nd only index pages

PeteU

10+ Year Member



 
Msg#: 199 posted 2:30 am on Jan 5, 2001 (gmt 0)

oops..

Air

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 199 posted 2:57 am on Jan 5, 2001 (gmt 0)

Yup, seeing it here too..

cirelle

10+ Year Member



 
Msg#: 199 posted 9:05 pm on Jan 5, 2001 (gmt 0)

saw that one on 1/4 at about 10:30pm wasn't alphabetical though

BoneHeadicus

10+ Year Member



 
Msg#: 199 posted 7:07 pm on Jan 6, 2001 (gmt 0)

Me too..

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved