homepage Welcome to WebmasterWorld Guest from 54.227.11.45
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
Forum Library, Charter, Moderators: Brett Tabke

Paid Inclusion Engines and Topics Forum

  posting off  
Inktomisearch spider... what does it do?
coffeelover




msg:21974
 4:43 pm on Mar 14, 2003 (gmt 0)

I, like lots of people, have been hit several times lately by the inktomisearch spider (slurp). What does it do? Why aren't my other sites that are linked to my site getting spidered? Does it only check for new content, or what?
Thanks,
Theodore

 

Alphawolf




msg:21975
 10:31 pm on Mar 14, 2003 (gmt 0)

Well, I never paid for inclusion, but my index page is listed.

Maybe because I am linked from a client site and he does pay for inclusion?

What does it do- well, someone with raw logs can report what files it crawls...

Maybe there is a hidden index somehere being prepared for yahoo with all this crawling.

AW

bsmithag




msg:21976
 12:19 am on Mar 16, 2003 (gmt 0)

I paid for Inktomi back in November. Usually it would grab my robots.txt and my index page every couple of days. Now it seems to be crawling deeper through my site.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved