homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
Forum Library, Charter, Moderator: open

Paid Inclusion Engines and Topics Forum

  posting off  
Is Slurp more powerful than Googlebot?
Is mothra going to overtake godzilla?

 4:44 pm on Feb 11, 2003 (gmt 0)

I have noticed significantly more hits (almost double) the hits from Slurp on a couple month old site as I have from Googlebot. Has anyone else experienced much higher volume spidering from Inktomi than from Gbot?

I'm curious if their has been lots of improvements in slurp since the acquisition, and if it has made their crawler more effective.


troels nybo nielsen

 5:00 pm on Feb 11, 2003 (gmt 0)

Slurp has become very active, but googlebot is still easily #1 on all my websites.


 6:20 pm on Feb 11, 2003 (gmt 0)

My site still has much more referral traffic from G, but I'm just wondering about hits from the actual spiders.

Has anyone else had Slurp hit their site harder than Gbot?


 6:24 pm on Feb 11, 2003 (gmt 0)

Usually for me slurp slowly creeps across the site. It goes just as deep just slower. GBot accosts my site in short bursts. Don't get me started on scooter the web server abuser.


 6:34 pm on Feb 11, 2003 (gmt 0)

Slurp has become much more active.

I think the biggest lesson G has taught all the other engines is that to be successful, you need to aggressively crawl the web and you must also be consistent about it.

Conducting a massive crawl every 6 months just doesn't cut it. I think Ink understands that. So I would expect to start seeing Slurp on a regular basis in the upcoming months.


 6:51 pm on Feb 11, 2003 (gmt 0)

INK has been steady, but probably not #1 for me. In the past month I have been heavily spidered by a new IP block assigned to DEC. Several very deep crawls. The spider is "Mercator-2.0".

I forget who ended up owning Alta Vista, but I assume that's where this spider is from.


 7:31 pm on Feb 11, 2003 (gmt 0)

I don't know about more powerful, but I'm finding that my site's new pages are entering into the Ink index faster than into Google's. Googlebot has the regular, dependable cycle, but Slurp seems to be stopping by more often, but with no real predictability.


 9:38 pm on Feb 11, 2003 (gmt 0)

I've seen much more activity from Slurp over the last month or so. I just assumed it must be part of Yahoo's grand plan and indicative of their future intent. No doubt someone with experience could give a reasonable analsis of the possible reasons for the increased activity which presumably is being orchestrated by Yahoo? Maybe I'm just a little too suspicious.


 6:20 pm on Feb 13, 2003 (gmt 0)

Is slurp supposed to be pay for inclusion?

I didn't pay for that spidering, and I haven't noticed it on my other sites.

Is Yahoo changing the model making it no longer pay for inclusion?

troels nybo nielsen

 8:15 am on Feb 14, 2003 (gmt 0)

you may want to read this thread if you haven't already.


I find it quite natural to expect Yahoo to make much more use of Inktomi, which might very well include creating a much larger and much more up to date database.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved