Forum Moderators: open
I really don't like a site with all content on root, and it's kind of unordered, but it seems is the best way to
get crawled at least by google...
Any comments, experiences?
How can I encourage spiders to crawl inside
mysite.com/folder/mydocuments...
Before I move all documents to mysite.com/...
Thanks!
For search engine indexing, the directory structure and URL-path structure of your site do not matter -- at least they do not matter very much.
What matters is the length of the click-path from your pages which have good incoming links to those pages which you want to have indexed. How many clicks to get from your very popular pages to those that you want to get indexed -- that is what counts. Search engine spiders follow links, just like users do.
I believe the URL and directory structure of your site should be determined with maintainability and site security features (i.e. password-protected areas) in mind. Some small benefits can also be gained from keyword-in-URL techniques, but not enough to override the other two considerations.
Jim