Forum Moderators: open

Message Too Old, No Replies

Database Data to Static Pages

Producing static html pages from a database.

         

Frank_Rizzo

11:04 pm on Jun 30, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Say I have a site which contains a database on animals. I have a few key html pages describing animals, but the main function of the site is a database search of animals.

e.g. enter Tiger and you get a full descripton of lions.

Trying to optimise this for SE's is tricky. The main function is an animal database but very few people will ever think about searching on SE's for "animal database". What they will search for is Lion or Tiger or Elephant right?

So how about I use the database to extract data and generate one off static html pages.

e.g.

----------
www.mysite.co.uk/tiger.html

<title>tigers..
<meta tag description="tigers....
.
.
<h1>tigers...
The tiger is found in a variety of habitats: from the tropical evergreen and deciduous forests of southern Asia to the coniferous, scrub oak, and birch woodlands of Siberia
----------

So what I am doing is creating static pages extracted from the database. Doing this will allow me to create keyword rich pages for each animal which will mean that it is more likely I'd be listed in search engines for that animal.

The only problem I can see is that it will probably increase the size of "the site" in html pages many times over (but then surely that is good to for internal link count?). Say there are 1000 animals, in effect I could create 1000 pages.

I guess 1000 pages is not practical so maybe I should just do it for key animals such as "tiger, lion, elephant, zebra etc."

Note that this whole task is not to disolve the database function but to attract visitors to the site in the first place. If they read the tiger.html page they would be encouraged to use the database to find more data on other animals.

Anyone done anything like this? How does it fare on the search engines, and more importantly with visitors?

Symbios

11:14 pm on Jun 30, 2003 (gmt 0)

10+ Year Member



Page names are important and I think its a good way to serve up relevant content, also from what I've read on [webmasterworld.com...] its a good way to build your site.

rcjordan

11:20 pm on Jun 30, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



> 1000 pages is not practical

Why? If you're creating them from a database, what's the problem? You'll just overwrite the pages when you add/delete/modify. Navigation? Create it from the database, too.

olwen

11:21 pm on Jun 30, 2003 (gmt 0)

10+ Year Member



It's relatively simple to build a whole heap of "pseudo-static" pages from a database, so the 1000 pages isn't necessarily silly.

Frank_Rizzo

11:34 pm on Jun 30, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



It's relatively simple to build a whole heap of "pseudo-static" pages from a database, so the 1000 pages isn't necessarily silly.

Sure, it's easy from a programming point of view. But what about search engines. Will they determine the site quality to be lowered?

rcjordan

11:42 pm on Jun 30, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



It's been my experience that the bots will grade them according to the page's individual merits. The problem, imo, comes from repetitive templates which is a very common problem in db-fed sites.

Frank_Rizzo

11:47 pm on Jun 30, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



The problem, imo, comes from repetitive templates which is a very common problem in db-fed sites.

That's the thing that worries me. Especially with 1000 pages.

I could throw some randomness into the tags, but the main meat on the page will be the same: location, size, eating habits etc.

rcjordan

11:55 pm on Jun 30, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



But the location, size, eating habits etc. will not be all that repetitive. I have a large-ish (12k pages, 3 sets of templates per animal) db-fed static site that does very well. It was designed to bring them in at the "lion" level, as a reference/guide. Get your titles and backlinks in order and I don't think you'll be sorry you went static.

jmccormac

12:05 am on Jul 1, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



The only problem I can see is that it will probably increase the size of "the site" in html pages many times over (but then surely that is good to for internal link count?). Say there are 1000 animals, in effect I could create 1000 pages.
%<snip>%
Anyone done anything like this? How does it fare on the search engines, and more importantly with visitors?

Yep I do this kind of thing on a regular (weekly or so) basis with all the domains and websites registered in the .ie cctld. It ends up generating a site of about 65000 webpages. The pages are generated from a number of databases. And the data collection and parsing is highly automated as it is done via nine spiders.

The advantage is that it puts a far lower load on the webserver as it only has to serve static webpages. A few pages are live though and the effect of search engines hammering them in quick succession is apparent. However search engines as a rule tend to prefer static webpages. With a dynamically generated webpage, the 'last updated' data tends to be the date the page was generated unless the developer was smart and included a fix for this.

I am not sure that visitors would be able to discern the difference between a dynamically generated page and a static page. However you will quickly see page popularity patterns develop. This would enable you to work on the pages that are less popular.

The whole database backed website argument is one that can cause long and sometimes heated discussions amongst developers and publishers. However there is one underlying fact that is very important - the frequency of update. If the data is rarely changed (every six months or year) then converting the site to static webpages is probably better because there is very little to be gained from having a db backed site. However from your side of things, you have to make sure that your update procedures are straightforward. If the site you are talking about is frequently updated then it may be a good thing to look at a content management system like Midgard or similar.

Regards...jmcc

pageoneresults

12:06 am on Jul 1, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Question...

Why would you want 1,000 static pages sitting there when you can serve up virtual pages? I mean, on one site that I've built, we use 3 .asp template pages that generate over 1,000 pages from the database. These pages are assembled from the database based on the request.

rcjordan

12:35 am on Jul 1, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>why

Because I can... ...and swap templates on-the-fly, too, right down to the url if need be. hhh!

Also, as mentioned above, cpu overhead is reduced. And lots of little nagging worries about which spider can do what disappear.

pageoneresults

12:52 am on Jul 1, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Because I can... ...and swap templates on-the-fly, too, right down to the url if need be. hhh!

Humph! I can do that too. Na, na, na... ;)

Hmmm, we've not seen any problems with cpu overhead. And, it appears that the spiders enjoy our templated structure. The pages look and act as if they were static. The content changes so much from one request to another that the page would not appear to be a duplicate.

Sure, the underlying structure of the page is consistent, but there are various tags and attributes being applied based on the request and the data served so there is constant change in the document structure.

P.S. When data is entered into the database, we also include various structure tags depending on the amount of content and formatting.

rcjordan

1:22 am on Jul 1, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>Sure, the underlying structure of the page is consistent

Not if I set the internal "rank" of the content sub-sections as the individual pages are built. (Na, na, na...) For some animals, the habitat should be near the top, for others their food source, etc.



>there are various tags and attributes being applied based on the request and the data served so there is constant change in the document structure.

>P.S. When data is entered into the database, we also include various structure tags depending on the amount of content and formatting.

Same here, I just pre-process the request and write it out to static.



>we use 3 .asp template pages

Perl and text files. Goes anywhere.



>cpu
I also process multiple (up in the teens now) ssi calls (headers, footers, nav, etc) during the static build. That leaves the ssi available for more visit-sensitive calls.

I also serve one script in particular that is a cpu hog, and it's popular. I have to off-load everything else that I can.



>it appears that the spiders enjoy our templated structure

Same here, but it's been a long while since I've even considered which spiders like dynamic or pseudo-static pages. I don't have to consider mod_rewrite, etc. ...just write html. Far simpler to keep all the pieces in my head.

Frank_Rizzo

8:11 am on Jul 1, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



OK, you convinced me its a good idea guys. Just reading through the threads though, I hope there is no confusion of the goal here.

I'm not trying to do a "dynamic database driven site to static page conversion" at all.

The reason for doing this is because I have an extensive animal database and yet visitors are not finding me because no one searches for "animal database".

Therefore, I do a one-shot extract of the database to pull off profiles of all animals. These would be split into seperate pages with a master index linking to each animal page.

These are in effect doorway pages right? Whilst they will show useful information about each animal their sole purpose is to be flyers to get visitors to the site.

I wouldn't expect them to sit there looking at page with 1000 links on it, they are encouraged to find these doorway pages and then go to the main database search page where they can do fancy stuff like find animals where continent = Asia, Food chain = plant eater, mammal = yes, size = small

Aren't doorway pages bad things?

PS
Would it be better if I got a totally new domain to do this kind of thing? i.e. main site is vvv animalwidgetworld.com so create the static pages on vvv.widgetanimals.com with links such as "you've read about tigers now search an interactive database over at animalwidgetworld.com"

WebDevInfo

10:07 pm on Jul 4, 2003 (gmt 0)

10+ Year Member



If you want people to be able to find details of each and every animal in your database, you should setup your database your create virtual pages ... but ... make sure they have different title tags, alt tags on any images, headings etc.

You could also have whatever programming language you'll be using to explode the URL from something like (only an example) www.yoursite.com/animal.php?id=123 to something like this instead www.yoursite.com/animal/tiger/

This way, each of your animal pages will be indexed on the SEs, therefore they'll be searchable on the SEs.

Does this make sense?