homepage Welcome to WebmasterWorld Guest from 54.227.141.230
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Website
Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
Forum Library, Charter, Moderators: Brett Tabke

Paid Inclusion Engines and Topics Forum

  posting off  
Will inktomi deep crawl for one fee?
does the $39 fee mean slurp will (likely) index all pages?
matthew




msg:27489
 2:53 am on Dec 9, 2002 (gmt 0)

I was wondering if I pay for the inktomis fast spidering for my index page, does this mean it will deep crawl most likely? Also, off topic a bit, but does having no robot.txt file hurt my "likeability" or chance of being deep crawled/crawled by a spider? thanks.

 

JayC




msg:27490
 7:05 am on Dec 9, 2002 (gmt 0)

1) No. The "free" crawl won't happen as a result of your paying for spidering of the index page. Wouldn't make sense, would it? They want you to pay for all of your pages!

2) No. If you don't have a robots.txt you're allowing everybody. No search engine will penalize you for it.

Big_Balou




msg:27491
 7:20 am on Dec 9, 2002 (gmt 0)

Welcome to Webmaster World.

By paying for the index page INK will only spider that page on the 48hr refresh setup. The rest will be indexed in the normal round of updates. I'm not sure if there is any evidence of having a page in the express search submit will speed up the indexing of the rest of your site. If there is someone will come along to correct me.

At one time there was a "problem" with sites that paid the fee for the quick refresh not having the rest of their pages spidered and indexed in the normal INK update but that seems to have gone away now.

Not having a robots.txt file will not hurt you but is a good idea if there are any files or areas that you do not want indexed. Without one basically a spider sees anything there, at the site, is fair game to be crawled and indexed.

If you want to find out more about the robots.txt file click on the site search link at the top of the window and do a search for it.

You'll find that a lot of the questions you might have have already been covered in some detail and you can find those discussions using the site search....plus a lot of other goodies ;)

tigger




msg:27492
 8:20 am on Dec 9, 2002 (gmt 0)

I do remember sometime ago that index pages were not ranking well under the paid inclusion program, is anyone still finding this?

Big_Balou




msg:27493
 11:24 pm on Dec 9, 2002 (gmt 0)

Actually for me it is just the opposite. The index page of one site that I used the express deal on is ranked high on multiple KW's while another page, also submited in the express deal, on the same site is buried.

ulstrup




msg:27494
 12:05 am on Dec 10, 2002 (gmt 0)

For me it's subpages ranking good, index page low, but the competition on the index page keyword is much higher, so I don't think it's because it's the index page

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved