Welcome to WebmasterWorld Guest from 50.17.5.36

Forum Moderators: open

Message Too Old, No Replies

Is Slurp more powerful than Googlebot?

Is mothra going to overtake godzilla?

     
4:44 pm on Feb 11, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 22, 2002
posts:1807
votes: 1


I have noticed significantly more hits (almost double) the hits from Slurp on a couple month old site as I have from Googlebot. Has anyone else experienced much higher volume spidering from Inktomi than from Gbot?

I'm curious if their has been lots of improvements in slurp since the acquisition, and if it has made their crawler more effective.

5:00 pm on Feb 11, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Oct 23, 2002
posts:1052
votes: 0


Slurp has become very active, but googlebot is still easily #1 on all my websites.
6:20 pm on Feb 11, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 22, 2002
posts:1807
votes: 1


My site still has much more referral traffic from G, but I'm just wondering about hits from the actual spiders.

Has anyone else had Slurp hit their site harder than Gbot?

6:24 pm on Feb 11, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member korkus2000 is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 20, 2002
posts:3732
votes: 0


Usually for me slurp slowly creeps across the site. It goes just as deep just slower. GBot accosts my site in short bursts. Don't get me started on scooter the web server abuser.
6:34 pm on Feb 11, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 26, 2000
posts:2176
votes: 0



Slurp has become much more active.

I think the biggest lesson G has taught all the other engines is that to be successful, you need to aggressively crawl the web and you must also be consistent about it.

Conducting a massive crawl every 6 months just doesn't cut it. I think Ink understands that. So I would expect to start seeing Slurp on a regular basis in the upcoming months.

6:51 pm on Feb 11, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member jomaxx is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Nov 6, 2002
posts:4768
votes: 0


INK has been steady, but probably not #1 for me. In the past month I have been heavily spidered by a new IP block assigned to DEC. Several very deep crawls. The spider is "Mercator-2.0".

I forget who ended up owning Alta Vista, but I assume that's where this spider is from.

7:31 pm on Feb 11, 2003 (gmt 0)

Full Member

10+ Year Member

joined:Oct 22, 2002
posts:217
votes: 0


I don't know about more powerful, but I'm finding that my site's new pages are entering into the Ink index faster than into Google's. Googlebot has the regular, dependable cycle, but Slurp seems to be stopping by more often, but with no real predictability.
9:38 pm on Feb 11, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 31, 2002
posts:880
votes: 0


I've seen much more activity from Slurp over the last month or so. I just assumed it must be part of Yahoo's grand plan and indicative of their future intent. No doubt someone with experience could give a reasonable analsis of the possible reasons for the increased activity which presumably is being orchestrated by Yahoo? Maybe I'm just a little too suspicious.
6:20 pm on Feb 13, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 22, 2002
posts:1807
votes: 1


Is slurp supposed to be pay for inclusion?

I didn't pay for that spidering, and I haven't noticed it on my other sites.

Is Yahoo changing the model making it no longer pay for inclusion?

8:15 am on Feb 14, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Oct 23, 2002
posts:1052
votes: 0


stuntdubl,
you may want to read this thread if you haven't already.

[webmasterworld.com...]

I find it quite natural to expect Yahoo to make much more use of Inktomi, which might very well include creating a much larger and much more up to date database.