Will inktomi deep crawl for one fee?

does the $39 fee mean slurp will (likely) index all pages?

[matthew]

I was wondering if I pay for the inktomis fast spidering for my index page, does this mean it will deep crawl most likely? Also, off topic a bit, but does having no robot.txt file hurt my "likeability" or chance of being deep crawled/crawled by a spider? thanks.

[JayC]

1) No. The "free" crawl won't happen as a result of your paying for spidering of the index page. Wouldn't make sense, would it? They want you to pay for all of your pages!

2) No. If you don't have a robots.txt you're allowing everybody. No search engine will penalize you for it.

[Big_Balou]

Welcome to Webmaster World.

By paying for the index page INK will only spider that page on the 48hr refresh setup. The rest will be indexed in the normal round of updates. I'm not sure if there is any evidence of having a page in the express search submit will speed up the indexing of the rest of your site. If there is someone will come along to correct me.

At one time there was a "problem" with sites that paid the fee for the quick refresh not having the rest of their pages spidered and indexed in the normal INK update but that seems to have gone away now.

Not having a robots.txt file will not hurt you but is a good idea if there are any files or areas that you do not want indexed. Without one basically a spider sees anything there, at the site, is fair game to be crawled and indexed.

If you want to find out more about the robots.txt file click on the site search link at the top of the window and do a search for it.

You'll find that a lot of the questions you might have have already been covered in some detail and you can find those discussions using the site search....plus a lot of other goodies ;)

[tigger]

I do remember sometime ago that index pages were not ranking well under the paid inclusion program, is anyone still finding this?

[Big_Balou]

Actually for me it is just the opposite. The index page of one site that I used the express deal on is ranked high on multiple KW's while another page, also submited in the express deal, on the same site is buried.

[ulstrup]

For me it's subpages ranking good, index page low, but the competition on the index page keyword is much higher, so I don't think it's because it's the index page