homepage Welcome to WebmasterWorld Guest from 54.196.196.108
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
New Excite Bot?
storm.excite.com
Son_House




msg:398056
 8:52 am on Apr 9, 2001 (gmt 0)

storm.excite.com
libwww-perl/5.47

Grabbed 3 pages. Did not get robots.txt

 

tcmalaska




msg:398057
 2:16 pm on Apr 9, 2001 (gmt 0)

Not new they've been doing the lib thing for quite a while, I have them coming out of all over but they've been running them for a while .

Son_House




msg:398058
 5:02 pm on Apr 9, 2001 (gmt 0)

Thanks for the info tcmalaska, first time I have seen Excite use the lib ua. I know that they use ArchitextSpider so I thought they might be testing a new one and did not give it a real ua yet. But the real thing that caught my eye was the storm part. I took a look at SEW's page on the Excite spider [searchengineworld.com] and did not see storm listed. I looked up the IP address and it is 199.172.157.28

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved