homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Yahoo / Deprecated - Altavista, Alltheweb.com
Forum Library, Charter, Moderator: open

Deprecated - Altavista, Alltheweb.com Forum

fast spider reading http code 304
jeremy goodrich

 5:34 pm on Sep 28, 2001 (gmt 0)

That is the apache not modified since last visit header, correct? I haven't actually bothered with their spider much recently...though they do send a tiny bit of traffic.

How long have they been looking at things other than 200? This is news to me...and I looked through the forum for a few minutes before posting, didn't notice anyone else mentioning this...makes sense that they are doing it, to keep a fresh database of info, and not bothering to reprocess the same exact data twice.

Anyone else seeing this?



 6:08 pm on Sep 28, 2001 (gmt 0)

The way it works is Fast's bot comes in with a IF_MODIFIED_SINCE = some date and then it is up to the server to return the page or serve up the error message. The bot doesn't know what is going to be returned, the server checks for the IF_MODIFIED_SINCE header and makes the call.

stephen baker

 9:24 pm on Sep 28, 2001 (gmt 0)

Hey there. This is Stephen from FAST. Stupid question for you...when you say that we send you a tiny bit of traffic, are you referring to AllTheWeb only, or to Lycos, SOL, Wind, etc..all of the other portals that we power? Thanks!

jeremy goodrich

 4:44 am on Oct 1, 2001 (gmt 0)

:) Must apologize for that, Stephen from FAST..

What I meant was that my personal search engine related site has only recieved, since inception, 2 clicks from Fast Search and Transfer powered search engines and portals. Then again, it's only been around since June.

Sorry, but it seems that people don't search much for what I wrote within your database. Or I'm doing something wrong, and optimizing poorly :) One of the two referrals I recieved was actually from Lycos. It was quite flattering, because I didn't think my site would come up in a search so, how do I say, scientific?.

Then again, the site has only recieved referrals from Google, Alltheweb, Lycos, AOL, DMOZ, Wisenut and partners who have linked to it. So actually, being last out of 6 isn't too bad, considering Inktomi, Altavista, Excite, etc. have never sent a single referral, nor have Northernlight, IWon, directhit, ask jeeves, teoma, looksmart, goto, findwhat...etc.

BTW, IMHO, the stupid questions are the ones we don't ask. Cheers.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Yahoo / Deprecated - Altavista, Alltheweb.com
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved