Welcome to WebmasterWorld Guest from 50.17.79.100

Forum Moderators: mademetop

Message Too Old, No Replies

Crawlers Out In Force.

     

Brett_Tabke

7:11 am on Nov 28, 2002 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



This week, we saw full crawls from:

Fast
Inktomi
Google
Wisenut
Altavista

That's all the major crawlers but Teoma. I think that is the first time in ages where 5 engines have all been actively crawling at the same time.

It's nice to see them all out fighting for it again!

dantheman

8:05 am on Nov 28, 2002 (gmt 0)

10+ Year Member



And inktomi just looped and looped while generating 20,000 404 hits. All this due to a handful of links that were out by one folder level. Just makes a huge mess of the logs!

But it sure was a busy week.

mayor

8:50 am on Nov 28, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Brett, I haven't seen Inktomi in my logs, except PFI pages. I have some older sites that have some PFI pages, and get only PFI crawls, and new sites that, despite having good links from other established sites, haven't been getting crawled by Slurp.

Can you tell if the Ink crawling you are seeing includes new sites (three months old or newer), or free pages on PFI sites?

Brett_Tabke

11:59 am on Nov 28, 2002 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Sorry, all older sites Mayor.

john316

3:20 pm on Nov 28, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Very nice activity from the major players, serps are improving as well.

I'm glad to see that there is still a race!

If AV would kill the poker/toner invitation links after each results page, they might get back faster, credibility is pretty well demolished with that nonsense.

[edited by: john316 at 3:36 pm (utc) on Nov. 28, 2002]

creative craig

3:27 pm on Nov 28, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I had Fast and Google taking near enough the same pages at the same time yesterday, and Wisenut is currently having a look around my site right now :)

freddoo

1:58 pm on Nov 29, 2002 (gmt 0)

10+ Year Member



How can I know that a full craw was done on my site.
I have an old site but optimized it in mid octobre, no changes on the engines so far.

My cached page in google is still the old site.

When can I expect to see my modifications?

thank you.

willtell

3:16 pm on Nov 29, 2002 (gmt 0)

10+ Year Member



We have Inktomi and LS on our site, every single day. Should I be concerned?

martinibuster

4:48 pm on Nov 29, 2002 (gmt 0)

WebmasterWorld Administrator martinibuster is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Even Robozilla stopped by to see if I was still around!

Namaste

11:52 am on Dec 3, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



All this is very nice, but the relevency of results from all except google is E grade.

Hands up all those that will use anything but google if looking for nightclubs and discount hotels for your next vacation to Spain.

Why don't the search engines understand that 1st it's relevancy . Just read all kinds of nonsense in Inktomi's press release about Geo relevance and blah blah. Typical kind of press release put out by an overpaid & disinterested PR department.

Then some user called "Inktomi" says "we're very excided about the future"...well my friend, we are not!

 

Featured Threads

Hot Threads This Week

Hot Threads This Month