Forum Moderators: open

Message Too Old, No Replies

How to SEO a database driven site?

         

astoller

11:04 am on Dec 14, 2002 (gmt 0)

10+ Year Member



Should important pages be made static or can engines successfuly crawl d/base driven sites?

Woz

11:41 am on Dec 14, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



The question is not whether the engines can spider dynamic sites, rather whether the engines can spider dynamic URLs?

There has been a lot of discussion around this, if you do a search for [dynamic site] or [dynamic url] you should find plenty.

Onya
Woz

wruk999

3:49 pm on Dec 14, 2002 (gmt 0)

10+ Year Member



hi astoller,

I run a website which has a page for each individual category. There are lots of categories, and all are PHP, with a select statement, built into the page - though there is only 1 paramter!

I use a makestatic script which is a php script, and runs from the browser. it "indexes" the folder with all my dynamic php pages in, and writes a .sh (shell) script.
This shell script when run, takes the php pages, and copies them into a static html page.
If you have ssh/telnet acces like i do, you can set it up on cron (mine is set to run every hour as database is being updated quite often).
if you dont have access to cron, you can still run it from a basic ssh/telnet command.
The script was used by a client who has a very very large dynamic site, and had to be able to have static html pages to send out on CD-ROM.
I use the script because of the large volume of pages on the site.
If you don't have that many pages. you can use the following UNIX command:

lynx -source [dynamic.yourdomain.com...] > /www/where/you/want/it/page.html

This should all be on one line! change the dynamic bit to wherever yuor dynamic pages are (url) and the second bit to the REAL ROOT PATH of where you want them to go.
This will take a "snapshot" of page.php and copy it to page.html...makes it static.
If your pages aren't changing that often, you could write your own .sh script, using the example code above, without the makestatic php script.
Just make sure that every page you wnat converting is on a line of its own, and make sure that it doesnt put the code onto 2 lines!

don't know how this would work with paramters being passed thorugh the url - never tried it ;)

let me know if this helps,
Will.

andreasfriedrich

4:08 pm on Dec 14, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



As Woz wrote indexing dynamically generated sites is not a problem at all. No SE spider cares nor do they have any way of knowing how the body of a http message they receive as an answer to their request was generated. That being said an approach as the one explained by wruk999 is not neccessary at all to make spidering possible. It might help reduce the load on the server.

The real problem lies with URIs that contain (too many) parameter in their query string. And by no means does having a query string is sufficient evidence that the page identified by that URI will be generated dynamically.

I do admit that it is usually. But this is just a possibility or a question of probability. Anything derived from that will only express a probablity itself.

To answer your questions:

Should important pages be made static

Not at all. There is no need to do that for the reasons explained above.

can engines successfuly crawl d/base driven sites

As long as the documents that are served to the spider contain spiderable links engines crawl d/base driven sites just like any other sites.

I have a new database driven site in the tourism industry that was spidered by GoogleBot as soon as it went online. This site is totally database driven, is XHTML compliant and uses nice looking URIs without any query strings.

Andreas

wruk999

5:56 pm on Dec 14, 2002 (gmt 0)

10+ Year Member



Andreas,
The static script that I use is used for two reasons, one being what you outlined in your post: to reduce load on the server, and the other, which I did explain, and that was to be able to send out the static files on a CD-ROM.

This is only possible if the files are static html, as most home computeres cannot display php pages properly - especially when there is a database being used aswell!

The script works excellently for me, google picked it up in a matter of days, and its ranking well!

Regards,
William

andreasfriedrich

6:04 pm on Dec 14, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



All I was saying is that your approach is not necessary to ensure spidering. I did not mean to imply that it did not solve your particular problems.

There is, however, nothing that should prevent you from Putting a dynamic PHP/MySQL site on a cd-rom [webmasterworld.com].

Andreas

wruk999

6:16 pm on Dec 14, 2002 (gmt 0)

10+ Year Member



As it happens - I was just reading that post - interesting!

The way I built my site, was that the paramteres were actually on the page (in the coding) - not in the URL.
I never tried making pages static which have parameteres in the url's.
Though the makestatic script has the provision for passing parameters when it makes the static page.

Though as I said, Ive never tried it - so I dont know how well it would work.
Though the system has dramatically reduced the load on the server! :)

Regards.

daamsie

8:49 am on Dec 18, 2002 (gmt 0)



I have found that my dynamic site (written in coldfusion) is spidered very nicely by Google at least. In fact, because of the dynamic nature of the pages, I have been able to register well for some pretty obscure search terms by making a dynamic title tag.

For example.. the title on the index page might be 'find widgets' .. then on each of the dynamic pages it says 'find widgets in australia' or 'find widgets in usa' etc.. one for each country that widgets are available (100 or so)

In this way, even though I haven't been ranking high yet on my prime keywords (site is still young), I have been getting a lot of hits from people searching for more specific pages. I get some pretty obscure ones, like 'find slovak widgets' :-) Seems to me far easier than doing the same thing on a static site!

So take advantage of your dynamic pages and make your header tags dynamic as well!

astoller

11:50 am on Dec 18, 2002 (gmt 0)

10+ Year Member



Can you please explain how I make my header tags dynamic?
Does this mean you have an online database of header tags.eg 100 titles one for ech country, for example.

widgets for in australia
widgets in ireland
.
.Then how does you software generate 100 php or coldfusion pages?
and how are they all linked, because if they are not linked, then how can the s/e spider them all?

Confused.

daamsie

7:59 am on Dec 21, 2002 (gmt 0)



Hi Astoller..

I've replied to this one in your other post, so check there for more code.

All links work just like normal dynamic links.. ie..


<a href="country.cfm?country=#country#">

.. you then use the variable "url.country" to query the database in your country.cfm page.. ie..


<CFQUERY DATASOURCE="#1#" NAME="country">
SELECT * FROM
Countries
WHERE Country = '#url.country#'
</CFQUERY>

Output that on your page in whatever way you like..but the header tags can either be done in the way i mentioned in your other post.. or if you don't use includes.. you could just make a title like so:


<CFOUTPUT>
<title>widget in #country#</title>
</CFOUTPUT>

There is no need to do that manually, as all the information you need is already in your database.

Good luck with it.