Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: open
e.g. one of my domains file structure is something like:
which links to many pages in the form
all of these pages where spidered properly.
Each of above pages again links to
these pages did never appear in the index. Either to much parameters or no good links from static pages.
When I switched the structure to .html-urls (widget_123_456_1.html) using mod_rewrite on a similar project all the catalog pages where spidered soon and appeared in the next update.
Last month my website was full of dynamic pages, so I put a site map in place with a link from the index page to the site map, and each month GoogleBot hit my site up to 100 times!
Last week I implemented mod_rewrite and replaced all my dynamic links (mydomain.com/Nav.do?par1=string&par2=string2) from the site map to mydomain.com/dir/blah.html GoogleBot visited me yesterday and hit over 900 pages! It is true that GoogleBot can trawl dynamic pages with a limited query length, but I would say that it prefers static pages!