homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Marketing and Biz Dev / General Search Engine Marketing Issues
Forum Library, Charter, Moderators: mademetop

General Search Engine Marketing Issues Forum

Crawlers Out In Force.

 7:11 am on Nov 28, 2002 (gmt 0)

This week, we saw full crawls from:


That's all the major crawlers but Teoma. I think that is the first time in ages where 5 engines have all been actively crawling at the same time.

It's nice to see them all out fighting for it again!



 8:05 am on Nov 28, 2002 (gmt 0)

And inktomi just looped and looped while generating 20,000 404 hits. All this due to a handful of links that were out by one folder level. Just makes a huge mess of the logs!

But it sure was a busy week.


 8:50 am on Nov 28, 2002 (gmt 0)

Brett, I haven't seen Inktomi in my logs, except PFI pages. I have some older sites that have some PFI pages, and get only PFI crawls, and new sites that, despite having good links from other established sites, haven't been getting crawled by Slurp.

Can you tell if the Ink crawling you are seeing includes new sites (three months old or newer), or free pages on PFI sites?


 11:59 am on Nov 28, 2002 (gmt 0)

Sorry, all older sites Mayor.


 3:20 pm on Nov 28, 2002 (gmt 0)

Very nice activity from the major players, serps are improving as well.

I'm glad to see that there is still a race!

If AV would kill the poker/toner invitation links after each results page, they might get back faster, credibility is pretty well demolished with that nonsense.

[edited by: john316 at 3:36 pm (utc) on Nov. 28, 2002]

creative craig

 3:27 pm on Nov 28, 2002 (gmt 0)

I had Fast and Google taking near enough the same pages at the same time yesterday, and Wisenut is currently having a look around my site right now :)


 1:58 pm on Nov 29, 2002 (gmt 0)

How can I know that a full craw was done on my site.
I have an old site but optimized it in mid octobre, no changes on the engines so far.

My cached page in google is still the old site.

When can I expect to see my modifications?

thank you.


 3:16 pm on Nov 29, 2002 (gmt 0)

We have Inktomi and LS on our site, every single day. Should I be concerned?


 4:48 pm on Nov 29, 2002 (gmt 0)

Even Robozilla stopped by to see if I was still around!


 11:52 am on Dec 3, 2002 (gmt 0)

All this is very nice, but the relevency of results from all except google is E grade.

Hands up all those that will use anything but google if looking for nightclubs and discount hotels for your next vacation to Spain.

Why don't the search engines understand that 1st it's relevancy . Just read all kinds of nonsense in Inktomi's press release about Geo relevance and blah blah. Typical kind of press release put out by an overpaid & disinterested PR department.

Then some user called "Inktomi" says "we're very excided about the future"...well my friend, we are not!

Global Options:
 top home search open messages active posts  

Home / Forums Index / Marketing and Biz Dev / General Search Engine Marketing Issues
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved