homepage Welcome to WebmasterWorld Guest from 54.167.138.53
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Marketing and Biz Dev / General Search Engine Marketing Issues
Forum Library, Charter, Moderators: mademetop

General Search Engine Marketing Issues Forum

    
Crawlers Out In Force.
Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 1613 posted 7:11 am on Nov 28, 2002 (gmt 0)

This week, we saw full crawls from:

Fast
Inktomi
Google
Wisenut
Altavista

That's all the major crawlers but Teoma. I think that is the first time in ages where 5 engines have all been actively crawling at the same time.

It's nice to see them all out fighting for it again!

 

dantheman

10+ Year Member



 
Msg#: 1613 posted 8:05 am on Nov 28, 2002 (gmt 0)

And inktomi just looped and looped while generating 20,000 404 hits. All this due to a handful of links that were out by one folder level. Just makes a huge mess of the logs!

But it sure was a busy week.

mayor

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 1613 posted 8:50 am on Nov 28, 2002 (gmt 0)

Brett, I haven't seen Inktomi in my logs, except PFI pages. I have some older sites that have some PFI pages, and get only PFI crawls, and new sites that, despite having good links from other established sites, haven't been getting crawled by Slurp.

Can you tell if the Ink crawling you are seeing includes new sites (three months old or newer), or free pages on PFI sites?

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 1613 posted 11:59 am on Nov 28, 2002 (gmt 0)

Sorry, all older sites Mayor.

john316

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 1613 posted 3:20 pm on Nov 28, 2002 (gmt 0)

Very nice activity from the major players, serps are improving as well.

I'm glad to see that there is still a race!

If AV would kill the poker/toner invitation links after each results page, they might get back faster, credibility is pretty well demolished with that nonsense.

[edited by: john316 at 3:36 pm (utc) on Nov. 28, 2002]

creative craig

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 1613 posted 3:27 pm on Nov 28, 2002 (gmt 0)

I had Fast and Google taking near enough the same pages at the same time yesterday, and Wisenut is currently having a look around my site right now :)

freddoo

10+ Year Member



 
Msg#: 1613 posted 1:58 pm on Nov 29, 2002 (gmt 0)

How can I know that a full craw was done on my site.
I have an old site but optimized it in mid octobre, no changes on the engines so far.

My cached page in google is still the old site.

When can I expect to see my modifications?

thank you.

willtell

10+ Year Member



 
Msg#: 1613 posted 3:16 pm on Nov 29, 2002 (gmt 0)

We have Inktomi and LS on our site, every single day. Should I be concerned?

martinibuster

WebmasterWorld Administrator martinibuster us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 1613 posted 4:48 pm on Nov 29, 2002 (gmt 0)

Even Robozilla stopped by to see if I was still around!

Namaste

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 1613 posted 11:52 am on Dec 3, 2002 (gmt 0)

All this is very nice, but the relevency of results from all except google is E grade.

Hands up all those that will use anything but google if looking for nightclubs and discount hotels for your next vacation to Spain.

Why don't the search engines understand that 1st it's relevancy . Just read all kinds of nonsense in Inktomi's press release about Geo relevance and blah blah. Typical kind of press release put out by an overpaid & disinterested PR department.

Then some user called "Inktomi" says "we're very excided about the future"...well my friend, we are not!

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Marketing and Biz Dev / General Search Engine Marketing Issues
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved