homepage Welcome to WebmasterWorld Guest from 54.243.23.129
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Website
Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
Forum Library, Charter, Moderators: Brett Tabke

Paid Inclusion Engines and Topics Forum

  posting off  
Inktomi bot's deep crawl?
inktomisearch, slurp, crawl
ircgeeks




msg:19462
 7:52 pm on Apr 11, 2003 (gmt 0)

Ok is this normal? I have never seen deep crawl from slurp like this. It has been going on for two hourís now they just keep coming... I am not complaining just surprised


66.196.72.68 j3158.inktomisearch.com Commercial 23 min : 25 sec
66.196.72.95 j3185.inktomisearch.com Commercial 27 min : 31 sec
66.196.72.71 j3161.inktomisearch.com Commercial 0 min : 18 sec
66.196.72.39 j3129.inktomisearch.com Commercial 0 min : 57 sec
66.196.72.11 j3101.inktomisearch.com Commercial 2 min : 49 sec
66.196.72.72 j3162.inktomisearch.com Commercial 16 min : 33 sec
66.196.72.81 j3171.inktomisearch.com Commercial 15 min : 35 sec
66.196.72.47 j3137.inktomisearch.com Commercial 17 min : 38 sec
66.196.72.62 j3152.inktomisearch.com Commercial 23 min : 48 sec
66.196.72.46 j3136.inktomisearch.com Commercial 25 min : 46 sec
66.196.72.40 j3130.inktomisearch.com Commercial 22 min : 34 sec
66.196.72.52 j3142.inktomisearch.com Commercial 7 min : 41 sec
66.196.72.60 j3150.inktomisearch.com Commercial 2 min : 50 sec
66.196.72.51 j3141.inktomisearch.com Commercial 12 min : 2 sec
66.196.72.35 j3125.inktomisearch.com Commercial 26 min : 39 sec
66.196.72.49 j3139.inktomisearch.com Commercial 6 min : 2 sec

 

pendanticist




msg:19463
 3:27 pm on Apr 12, 2003 (gmt 0)

si3005.inktomisearch.com - - [12/Apr/2003:05:00:49 -0700] "GET /robots.txt HTTP/1.0" 200 220 "-" "Mozilla/5.0 (Slurp/si; slurp@inktomi.com; http//www.inktomi.com/slurp.html)"
si3007.inktomisearch.com - - [12/Apr/2003:05:03:53 -0700] "GET /robots.txt HTTP/1.0" 200 220 "-" "Mozilla/5.0 (Slurp/si; slurp@inktomi.com; http//www.inktomi.com/slurp.html)"
j3122.inktomisearch.com - - [12/Apr/2003:05:04:52 -0700] "GET /Libraries.html HTTP/1.0" 200 14424 "-" "Mozilla/5.0 (Slurp/cat; slurp@inktomi.com; http//www.inktomi.com/slurp.html)"
j3122.inktomisearch.com - - [12/Apr/2003:05:05:32 -0700] "GET /Statistics.html HTTP/1.0" 200 22657 "-" "Mozilla/5.0 (Slurp/cat; slurp@inktomi.com; http//www.inktomi.com/slurp.html)"

I can't say too much about years past, but it's been like this long enough that to me, it is normal.

Pendanticist.

flexsez




msg:19464
 4:50 am on Apr 13, 2003 (gmt 0)

This deep crawl behavior seems unusual to me too.

Within the past month my interior pages are showing up in MSN.

Part of the problem with MSN is that they keep changing their minds on what direction they are headed. Their serps are all over the map on a day to day basis.

When do the deep crawl results seem to show up in the serps?

pendanticist




msg:19465
 10:23 pm on Apr 13, 2003 (gmt 0)

>When do the deep crawl results seem to show up in the serps?

I don't concern myself w/serps, ergo I don't know how to answer your question. Sorry.

Pendanticist.

coconutz




msg:19466
 1:43 am on Apr 14, 2003 (gmt 0)

>>When do the deep crawl results seem to show up in the serps?

Not sure when or if all of the pages crawled are added to the index, but Inktomi says the regular index is updated once every 2 weeks.

How often is Ink updating now? [webmasterworld.com]

futureX




msg:19467
 1:49 am on Apr 15, 2003 (gmt 0)

i've had the same on and off for the past few weeks...

At one point inktomi was sending a new spider every 20 seconds which continued for around 12 hours (9/10 times the same page)... Looks like every fortnight inktomi comes and hamers my site... Although i have yet to see the pages it has visited in the SERPs

btw does anyone know a command on any of the inktomi sites that is equal to google's allinurl:?

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved