Which pages of my site can Google spider?

and how can I find out?

[dodger]

I've noticed that some of the pages in my site have no page rank at all which means they haven't been spidered I guess.
Is there any propram out there or tool that can run through my site and show me what can/will be spidered and what won't, or even better identify problems.?

Thanks in advance.

[JoeyBall]

My sites have a pr6,pr6,pr4 over the last few weeks googlebot activity has dissapeared and consequently my site has been taken of google.

Is there anyway for my site to be indexed again. it still holds a few hundred links.

[plasma]

1. disable SessionIDs (at least for googlebot)

2. make sure you have enough PR

3. Search for: site:www.mysite.com -gfdsgfdsgfdsg
this will give you a list of all spidered pages.

4. There's a difference between grey-PR and white-PR
White means page is spidered but has no PR (or not yet, e.g. a newly indexed page, wait 1 update cycle). Could be a penalized site, too. (e.g. goatse ;)
Grey means Google doesn't even know that this page exists.

[DaveAtIFG]

Xenu is a free link checker, dodger. If Xenu can spider all your pages, surely the Googlebot can.

[victor]

I was going to suggest Xenu too - it's a great product.

But remember that Googlebot may be able to index pages that Xenu can't find.

You may have pages that are not reachable from your root page (the usual place you point Xenu to start) but are linked to by other sites. Maybe they are old pages for which you've removed all internal links. Googlebot may find them, and any other redundant pages they link to.

[nileshkurhade]

I think it is much better to create a Sitemap and be sure that all the pages get indexed.

[dodger]

Thanks all,I've got Xenu, I'll give it a try -

By sitemap do you mean a proper site map with the expanding menus or just headings and subheadings with links?

plasma - I'm not sure what you mean -

1. disable SessionIDs (at least for googlebot)

How?

2. make sure you have enough PR

Page rank? PR?

3. Search for: site:www.mysite.com -gfdsgfdsgfdsg
this will give you a list of all spidered pages.

With the garble at the end? Doesn't work for me.

[dodger]

I hate to draw away from my own post but I've got another question - responses to the above will still be very welcome -

When I do a search on "mysite.com" it retrieves sites that link to me - my competitor much to my horror shows twice as many as me even though we are more popular (official stats)

I noticed as I went through the links though that when I got to page 10 (100 links)for my competitor I got the message -

repeat the search with the omitted results included.

I didn't get that message for my site till I reached page 18 or 180 links.

Does this mean more of my links are of a better quality than my competitors even though I don't have as many?

[abates]

Note that Xenu doesn't honour robots.txt or meta tags (at least, the version I have doesn't!), so it will find pages that Googlebot can't...