why does google etc only index my home page

do spiders only take a few pages at a time?

[freshoutthebox]

Apologies if the question is a bit basic but this is my first post and you gotta start somewhere.

Do the search engine spiders just index a few page at a time as they visit your site

If so can anything be done to get them to fully index a site

Do robot.txt files have anything to do with this?

Do they index a few more next time they visit
How often do they visit?
Can you get them to visit more often?

Thanks for helping out

[SlyOldDog]

Hi there

Google will spider "more important" pages first. The most important pages are the ones with the best "reputation".

The google score for reputation is a metric called pagerank. There's plenty on the internet about it and how to earn it for your pages, and I can't explain it in one short post.

But the upshot is this: If your pagerank of any particular page exceeds 5 it is most likely to get spidered frequently.

The robots.txt is used primarily to stop spiders in the case you prefer content not to be listed, so don't worry if you don't understand how to use it.

[hitchhiker]

O and now there's BlockRank, the new one.

All my new sites tend to start of with just one page, it's pretty normal, also check your urls aren't too long, or weird.

(Especially if they're aspx)

[Robber]

Having plain html links to the pages on your site from the home page will help if you currently rely on something like javascript navigation.