homepage Welcome to WebmasterWorld Guest from 23.23.12.202
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Inktomi hitting pages over and over
what the heck?
Travoli

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 765 posted 3:33 pm on Jun 12, 2001 (gmt 0)

Hi all

So I finally got my spider tracking report going today, and on the first day of tracking I see that Excite visited, aswell as Northern Light and Inktomi. Weird thing is...Inktomi is going nutz-o and requesting the same pages over and over. I searched through some old threads, but was not able to figure out if this is just completely normal. Is this normal for spiders? Thanks.

 

toolman

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 765 posted 3:38 pm on Jun 12, 2001 (gmt 0)

inktomi.com (209.185.141.200) - Spider/Robot
09 Jun -- 04:57:36 -- -- /ohmygod/somethingbesideroot.html

inktomi.com (216.35.103.79) - Spider/Robot
09 Jun -- 05:01:15 -- 00:02 -- /robots.txt
09 Jun -- 05:01:17 -- -- /

inktomi.com (216.35.103.80) - Spider/Robot
09 Jun -- 04:56:53 -- 00:01 -- /robots.txt
09 Jun -- 04:56:54 -- -- /

inktomi.com (216.35.103.81) - Spider/Robot
09 Jun -- 04:57:40 -- 00:04 -- /robots.txt
09 Jun -- 04:57:44 -- -- /

inktomi.com (216.35.116.90) - Spider/Robot
09 Jun -- 04:57:49 -- 00:01 -- /robots.txt
09 Jun -- 04:57:50 -- -- /

inktomi.com (216.35.116.91) - Spider/Robot
09 Jun -- 04:57:34 -- 00:00 -- /robots.txt
09 Jun -- 04:57:34 -- -- /

inktomi.com (216.35.116.92) - Spider/Robot
09 Jun -- 04:58:08 -- 00:02 -- /robots.txt
09 Jun -- 04:58:10 -- -- /

inktomi.com (216.35.116.93) - Spider/Robot
09 Jun -- 04:57:24 -- 00:01 -- /robots.txt
09 Jun -- 04:57:25 -- -- /

Just another day.

Rusky

10+ Year Member



 
Msg#: 765 posted 3:46 pm on Jun 12, 2001 (gmt 0)

Hi

Experiencing exactly the same thing here, just the pages that make up the frameset of the index page.

Travoli

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 765 posted 3:46 pm on Jun 12, 2001 (gmt 0)

but the same for :

/welcome

100 times??

man! heh
you would think they could be SLIGHTLY more efficient. Then again, it is nice to know they have up to the SECOND information :)

Travoli

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 765 posted 1:04 pm on Jun 13, 2001 (gmt 0)

man, what the heck! 944 hits by Inktomi yesterday. most to /welcome. geeeeeez!

Can you say "Easy champ!"

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved