Welcome to WebmasterWorld Guest from

Forum Moderators: brotherhood of lan & mack

Message Too Old, No Replies

Building for the user and the bot

Finding the Balance

4:41 pm on Jan 14, 2006 (gmt 0)

Moderator This Forum from GB 

WebmasterWorld Administrator mack is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:June 15, 2001
votes: 3

Building for the user and the bot.

All to often web site developers design with search engines as the number one priority. This may get them good traffic, but very often such websites are little or no use to the site visitor. The web was designed as a way to publish your material to the masses, sometimes is is good to bear this in mind when you design a site.

If you build purely to please the search engines then you will probably succeed in generating search engine referrals, but as a user arrives on your site they are unlikely to stay for very long if the site is not built with a user in mind. On the other hand if you build purely with the user in mind then each visitor will have more of an impact and will be likely to stay on your site and perhaps return. Generating return users is very important in this business.

Lets imagine you have a site that generates 100 new users daily from search engines. If 15% of those users like your site enough to bookmark it just think how much traffic your site could generate long term. Over time you will have your search engine referrals plus repeat users.

To an extent it is a bit of a balance. You need to build a site that has enough user designed aspects to retain and satisfy your users yet still be able to hold it's own in the search engines.

Your main goal when you build a site should be to please the end users. I believe you should bear this in mind right through the entire design process, then re-assess your pages from a search engine point of view. Having a site that is incredibly simple to use, provides the user with the information they require and is attractive in appearance will make it stand out from the crowd. In my opinion the key to good traffic is to be patient.

Concentrate on your content, not just quantity but quality. If your site provided really good articles on your subject then people who share an interest will be very likely to return to your site in the future. It is then very important to satisfy your repeat users by regularly updating your content. Fresh content is the key to retaining repeat users. If your site only contains static content that never updates then there really is little or no point in users returning. Fresh content that is very useful, updates regularly and is well written could set your site well on it's way to becoming an authority in it's industry.

One term that you will hear a lot in the web development industry is "content is king" lets go into this in some more detail to explain just why this is so. As we have already said content is the main reason users will visit your pages, and fresh content gives them a reason to return. Search engines also love content. Search engine spiders (bots) are effectively addicted to content, they just cant get enough of it. If you have a site that is very rich in unique content then you can be reasonably assured it will be sought after by search engine bots. As search engines crawl your content pages they index your content and try to determine just what your page is about. If your content has been written specifically about "something" then the search engines will be able to work out just what that "something" is.

Because of this it is important to write your content and articles as specifically as possible. If for example you have a website about cars is is very difficult to write an article about cars and be specific. If however you have a page about "Ford" you have the scope to really write targeted information about the ford motor company. You may then have a page about the "ford focus" on this page you can be really specific and write a lot of very very targeted content based on the specific make and model.

Using content that is targeted to something specific per article works very well for both users and search engines. Search engines can analyse your pages and easily decide the exact topic, your users will also have really detailed information to digest. The key here is to not dilute your content and to try and keep all your individual articles as specific as possible.

Many websites will choose to have their site broken up into many different topic areas. As an example we will use the car website we described above. We have our homepage that has links to the topical areas. In the case of the car site lets say for example our main topic areas where car manufacturers. We couch then set up web folder for example /Ford, /Volvo and /Honda and so on. Within each folder we have an index page containing information about the specific car maker. We could then have a folder for each model produced my a manufacturer and use content within these folders that is related to the model in question.

You would end up with a folder structure something like


By assembling your folders into this logical structure you are in effect creating a pyramid with the homepage at the top and the sub folders spreading out below. By doing this we have created a very simple site structure that works for us in two ways. It makes the site very easy to maintain and it also makes it nice and simple to work on the sites navigation. When building a site with the users in mind navigation is very important.

On our car site we want to make it very easy for users to find their way about. Our homepage really only acts as an entry point to our real content, but it is important to make the homepage useful to the end user. In this case we will use it to post links to all new articles., so if ford bring out a new version of the Probe for example, there will be a brief summery and a link to the page containing the full article (content). When a users visits the content page they will be able to read the article and then be able to navigate the site. In this example I believe the best form of navigation will be the "bread crumb" This form of navigation works very well within a site following the pyramid structure. It allows users to visualise the site structure and move back down through the levels or go back to the homepage. For example of a users is on the web page with information on the Honda Civic the bread crumb would be something like this...

Home > Honda > Civic

There are two main reasons I like this sort of navigation. The first is as I mentioned before the ability to drill down through the levels or go right to the homepage. The second reason I like this form of site navigation is because it allows for slightly un-targeted search engine referrals. Lets imagine someone is looking for information about a Ford fusion and they are directed to the page about the Ford Focus they will almost immediately realise they are on the wrong page and may click the link to Ford from the bread crumb and then follow the correct link from within the ford page. Search engines are not perfect and being able to offer users a choice on arrival can be very beneficial to end users.

Good navigation is very user orientated but it also works like a road network that search engine spiders can follow as they index your website. This is an aspect of site construction that works very well from both a user and search engine point of view. You should design and test your site navigation well because if there is a flaw and a user cant get to a page he/she wants to view easily then they may choose to leave. For example the search engine referral we mentioned above where the user is send to a slightly off topic page. If there was no link to Ford why would they hang about. It would be a simple case of back button and they return to the search engine they came from.

Page headings.

On a web page it is very important to use page headings. The heading is what a user will glance when they arrive on your page. To an extent what the user perceives when they glance over your heading will reassure them that they are on the correct page. For this reason it is very important to make sure you use well thought out short, but concise headings. A page heading is also something that interests search engines. There are several different types of headings. On a web page the headings are generated using "H tags" For example H1 will be a main heading, H2 will be a sub heading and H3 will be a less important heading and so on. Lets imagine on our car site that we are placing a heading for the "Honda Civic" Our H1 tag would look something like the following

<H1>Honda Civic Information</H1>

This Will be seen as large text on the web page and will clearly stand out as a page heading. Obviously it is vital that you set your page up properly so that the heading appears above the actual content. You can then specify subheadings within your text, for example...

<H2>About the Honda Civic<H2>

The H text will appear smaller than H1 and will look like a perfect text for a subheading. You can then place heading for the various sub-sections of the article.


The H3 tag is perfect for almost labelling smaller areas within the text. In the example it will act as a header for a piece of text outlining the vehicles specification.

When you use H Tags is is very important to nest them correctly. You should only use one instance of H1, be very conservative with H2, only use it to divide very definitive main areas of the text. You can be a bit more liberal with H3 tags but it makes since only to use then as a minor headings, and not just for text. H Tags are designed to be used as heading, using them for any other purpose is going against their intended use.

Using page headings is great for the end user, if the are used correctly. They help to break up the page into manageable chunks, and if they are just glancing the page they will be able to locate the information they are looking for. It also works well with regards to search engines. The heading tags help the search engines work out your pages content by supplying it with clues. As long as you use well worded headlines this can really help you rank well for keywords that are related to your pages content. If you try and trick the search engines by overly using the H tags then all you are achieving is making the site a lot less useful for your end users.

Page titles

In print media, to an extent page titles and page heading mean very much the same thing. On the web things are a lot different in this respect. The page heading is an item that is a visible on page component. The page title is to a large extent hidden from the user. The page title is contained within the head section of an html document and is displayed at the very top of a users web browser. It is also the piece of text that many search engines will show within their results page. The title tag is all you have at your disposal to convince a user to click. It is important to get the page title as spot on as possible. It needs to be short and specifically related to the page it refers to. If you have a good well worded title then you stand a good chance of getting users to click through.

The title tag is something that is also very highly regarded by the search engines as a ranking tool. The title tag is arguable the most important ranking aspect of any html document from a search engines point of view. Because we have kept each of our on page components (content, headings, navigation and titles) specific to the page they will all be very relevant to each other. Now if a search engine indexes your page and reads your content. it will find title headings and text that are all very related, it will then be able to work out your page based in this information you have provided. Like I said before it is all about supplying the search engine with as many clues as possible to help it work out your page and rank it accordingly.

Unfortunately this is not realistically enough to get your site to rank well. In order to rank well you will need to get links pointing to your site. Many search engines class a link to your site as a vote for your site, the more voted you can get the better. If a search engine discovers you have a lot of votes (links) then it is more likely to rank you highly because it has reason to believe your site is a high quality resource.

Getting links is never going to be am easy task but it is an essential part of promoting a website that has to be done. I mentioned before that "content was king" this is also very true when it comes to finding links. If you supply great content on your website then you are more likely to receive a incoming links than a site that offers little or no good usable content.

If you build your site with the user in mind the search engine will follow.


6:37 am on Feb 3, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 7, 2004
votes: 0

Great post!

Regarding the balance between user and bots, I think that the most important element is the correct use of <meta name="robots" content="">. Be clever and do not hesitate to use "none" for pages that bots will dislike but visitors will use.

Indeed, there are tons of pages that you may want to give to your users - like the possibility of reordering items (e.g. last pics, best pics, user with most posts, etc...) - and in PHP/ASP, making such pages it very simple, a variable in the URL (via GET) will do the job. So easy! But don't forget, bots hate them!

Also, when launching a huge websites, you better use "none" in some pages and a bit by a bit change it to "index" so bots see new contents.

This 31 message thread spans 2 pages: 31