Forum Moderators: open
Nonone of the results apges ahve been index, I am assuming because spiders can't select from the form.
How can I fix this site so search engines can find the pages?
the english version
sorry
I have recently taken the management of a site where the main info pages can only be reach through a form. it is a travel site, so users choose things like destination, type of accommodation they want, length of stay etc from drop down lists then press enter to get the results.
Not one of the results pages have been indexed, I am assuming because spiders can't select from the form.
How can I fix this site so search engines can find the pages?
First, focus your pages down into the "basic" pages that you can incorporate into some sort of a hierarchical data structure that makes sense, possibly by not using all of your variables when you create these pages.
It helps to clean up your urls too. Here's a great thread about eliminating "allergic symbols" that I've bookmarked...
[webmasterworld.com...]
...and I know there are others.
You will then need to build some site map type pages with text links to your basic pages for the spiders to follow, and have your most important site map pages not too many clicks from your home page.
The choice of the basic pages to which you're building the hard links is going to be important. They need to be representative of the other possible dynamic pages you haven't specifically linked to... should give a visitor navigation or search access to these other pages... and the hierarchical structure, page titles, etc, need to make sense to both the user and the search engines.
Good choice of dynamic page titles is important. Regarding organizing the page structure, familiarize yourself with the idea of a "theme pyramid" structure (do a site search, or use Google to do a site search). Look at these threads, among others:
Search Engine Theme Pyramids and Google
Optimising the Pyramid for PageRank
[webmasterworld.com...]
(covers organization of material for PageRank)
Directory structure of theme pyramid
[webmasterworld.com...]
(has some material on dynamic urls)
I've really lumped a great many concepts together here and probably glossed over a lot, but I hope this gets you started.
will have to build into the database, a sitemap which is database generated based on possible options from the form.... for which their are going to a couple of million....
The second part of the above is what I'm saying you should not do.... You don't want to try to index all the possible combinations.
Instead, from your database, you've got to come up with a skeleton framework, so to speak, of main pages to be indexed, around which all the other pages (not to be indexed) exist. You'll need a conceptual scheme, to classify what pages are most important and to generate titles and assign some relevant content in key places to them.
Among other things, you've got to think about about distribution of PageRank... If you have too many pages, either in width or depth, they're not going to get crawled, and if they do get crawled, they're not going to rank and/or they may be basically duplicating each other. Read the theme pyramid threads, and keep in mind that Google will probably follow only 100 links per page.
Dupe content considerations also will be a limiting factor regarding the number of pages you want crawled. Engines don't like lots of dupe content.
Keep in mind that no one is going to be searching... on a search engine... specifically for a hotel room (smoking/non-smoking, view/non-view) for 3 people in Ravenna for between US$75 to US$125/ night.
They may be searching for "discount Ravenna hotels" or "cheap Ravenna hotel rooms" or whatever, and that's what you need to serve up for search engines. But you should only try to index several main pages for each area, covering the wording variations... not hundreds of pages serving every booking variable. Google is a search engine, not a booking engine.
How you get your database to do that is going to be tricky... but you need to limit the pages you deliver to the search engines. Save the rest for your search form on the site.
Eventually, on Froogle and the shopping search engines, indexing more pages with specific prices may become an issue. I don't know exactly how they will handle this... but, again, I'd think you'd want a representative sample, not all possible combinations. Infinity or a little less is not a satisfactory number of pages to expect to get indexed.
