Forum Moderators: open

Message Too Old, No Replies

Large Subdirectory Not Getting Crawled

900 link site map.

         

rrl

1:48 pm on Nov 8, 2003 (gmt 0)

10+ Year Member



I added a subdirectory to a smallish site. The subdirectory contains about 900 pages of unique content. I linked to it from one of the root level pages. That link goes to an index page within the subdirectory that has a search function and also a link to a site map of all 900 pages. Each of the pages has the same wrapper as the root level pages (same navigation items). I also linked to the index page of the sub directory from the home page.

The index page of the subdirectory has been picked up by google, but that's it. When I run the spider sim, it also stops there. Any thoughts? Thanks.

HarryM

2:53 am on Nov 9, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



If I understand you correctly, Google has indexed your subdirectory index page and that has only one link on it to the sitemap, and a search facility.

If so, Google is only aware that there is one page (your sitemap) to be indexed, and knows nothing of the other 900. It will probably give your sitemap the same priority as any other unindexed page, and how quickly it gets indexed probably depends on the PR of the index page. And until it is indexed the others will stay in limbo.

Why not add a link fron your home page to the sitemap which may encourage googlebot to look it over.

Harry

marcs

3:04 am on Nov 9, 2003 (gmt 0)

10+ Year Member



Google also recommends no more than 100 links per page. You may wish to consider splitting up the site map into multiple pages.