Welcome to WebmasterWorld Guest from 220.127.116.11
My question: What exactly should appear on the crawler page? I read that I should omit all text and tags and to include only links. What is the syntax of those links? Should I use anchor tags or something else? An example would be very helpful.
I make hallwaypages with text, tags and a little graphics.
I group my pages after content and make one hallway for each group (optimized for one keyword). All the hallways are interlinked and I have links to my hallways on most of my pages.
Since my hallways are human freindly, all the links to the hallways are visible (preferably with keywords in the links)
Here is a breif summary of what I believe is a good strategy:
Make your hallway page into a "site map" type of page. Each link to a page should include a description (keyword rich, of course) of what you will find on that page. It is also a great idea to arrange the page in a directory fashion. For example
-this site is designed to bring you the best information about cool widgets. Yadda Yadda...
-we have a large selection of cool-blue widgets. Whether you are looking for light blue or dark blue widgets, we can help you find what you are looking for.
-check out our new cool-light-blue-widgets. These are great for widgeting on special occasions. We have sizes S-XL.
Basically it becomes a directory page with real value to your customers/users. They should be able to quickly and easily navigate the page.
And....you get a page that spiders will (hopefully) spend lots of time jumping from.
HERE [webmasterworld.com] is lots of great info, and where I learned a lot of this stuff.
Thanks for the response and for the link to addtional info. I have briefly looked at the topic that you referred me too, but have not had a chance to read it in detail.
Your approach - and that of others who have commented - appears to be a little different than theone I originally read (by the way, from an article by Paul Boutin). You advocate making the page more than a gateway to other pages by including keyword rich descirptions for the links, correct? If this hallway page is chock full of keywords is it going to appear on search engine results and, if so, is that an issue? If it will appear in results should I include text beyond the index to give users an idea of what my site and product is about? Will the hallway page help other pages (where the real content is located) get ranked higher?
By the way, I also submitted a question about separate pages for major keyword phrases. Any input to that question is also greatly apprecited.
I haven't read the article you mentioned, but I'd have to disagree with the idea of only including links on the page. First of all, having a site map listed in a search engine isn't a bad thing. Having someone enter on a page that provides a clear, annotated list of all the site's content can result in the visitor being able to quickly find what they're looking for.
Also, some search engines clearly avoid indexing and following links on pages that contain no other text. Adding the additional content greatly increases the chances of a spider actually crawling all the pages listed.