homepage Welcome to WebmasterWorld Guest from 54.163.91.250
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
Forum Library, Charter, Moderator: open

Paid Inclusion Engines and Topics Forum

  posting off  
Inktomi Web Robot - Clarification!
bashyam




msg:19995
 5:12 am on Mar 1, 2003 (gmt 0)

Hi,

I am quiet new to this forum. I have a doubt regarding Inktomi web robot.

I launched my new website few weeks back. Today I found these 2 records in my web log (inktomi web robot has visited my site).

------------------------------------------------------------
66.196.65.26 - - [28/Feb/2003:22:51:14 ] "GET /robots.txt HTTP/1.0" 404 5908 "-" "Mozilla/5.0 (Slurp/si; slurp@inktomi.com; [inktomi.com...]

66.196.72.59 - - [28/Feb/2003:22:52:03 - "GET /accessibility.htm HTTP/1.0" 404 5908 "-" "Mozilla/5.0 (Slurp/cat; slurp@inktomi.com; [inktomi.com...]

------------------------------------------------------------

Though I did not opt for pay for inclusion option with inktomi, does this mean that inktomi web robot has located and registered my website for free indexing...?

pls clarify.

Thanks.

Balaji Bashyam.

 

jdMorgan




msg:19996
 5:33 am on Mar 1, 2003 (gmt 0)

Balaji,

Welcome to WebmasterWorld [webmasterworld.com]!

Yes, quite possibly, Ink has found a link to the page shown in your log, and has indexed that page. It also tried to read your robots.txt file [robotstxt.org], but you don't have one (You might consider uploading at least a blank robots.txt to prevent the 404 errors, but you don't have to).

If you read the paid offer on the Ink site (and others), you will see that for a fee, they offer to spider and index your pages often. Though they do not emphasize it strongly, that is what the fee is for - frequency of updates, and not necessarily fee-for-inclusion.

We had another short discussion of this today in this thread [webmasterworld.com].

Jim

bashyam




msg:19997
 6:10 am on Mar 1, 2003 (gmt 0)

thanks ..Jim.

daroz




msg:19998
 6:15 am on Mar 1, 2003 (gmt 0)

Actually what's interesting is that your page was 404 -- Not found. If those are the only 2 refrences to the Inktomi spider I don't think you're going to make the index.

Perhaps your IP was recently used for another site - and that one was in the index?

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved