Forum Moderators: open
However a lot of the other sub pages dont seem to be indexed. First question: is there any way to find out what pages are being spidered?
Second issue is that I'm running a CSS menu as a PHP include on every page. The menu is an expanding/collapsing one that has links to every other page (there are over 700 of them).
Do the SE's ignore internal links after a certain number? Would this explain why some pages dont seem to be showing?
Is there an easy fix for this without changing the menu system which took me months to design?
Sorry for all the Qs!
To find out what pages are being spidered you can do 2 things...
1. Look at the webserver logs for your site and look for any "spider" user agents (googlebot for google).n You can get software to analyse your site logs for you and get some real useful info about how users (and bots) use your site.
2. In google do a search a follows site:www.mysite.com this will show you all pages google has indexed for your site.
As for menus... if you have a script based menu search engines can find it hard to spider. You should always have a simple HTML menu (perhaps at the bottom of the page) linking to your most important pages.
As for number of menu items... I have read somewhere that you should limit each page to linking to about 100 items or so.
Hope that helps...