Create Your Perfect Internet Directory

Building and promoting an Internet directory.

[elguiri]

I have a new client that already runs a successful regional print business directory, similar to yellow pages. They now want build a portal for the region that is fully indexed by the search engines and will achieve good results wherever possible for search terms related to the directory categories.

As they’re doing this from scratch, it’s the perfect opportunity to get it right the first time, making it fully accessible to search engines and users, and automating the optimisation to some degree.

Although this site is being built in ASP on an IIS server (not my first choice, but I can’t change that), I’d be very grateful to hear the opinions of the wise citizens of WebmasterWorld on architecture and promotion strategies.

Here are a few ideas of mine to kick off with:

Database structure

• The database itself should be normalised to avoid duplicated data.
  
• One database table houses the category structure for the directory
  
• Every advertiser is assigned to a subcategory
  
• The record for each advertiser includes:
  a) Company name
  b) Address, Tel, www, email
  c) Sub-category
  d) Activity description
  e) Related keywords

Portal structure

My gut feel is there should be three access points to the content:

1.A search form with 3 boxes: Company name, activity, location
The activity search should search, in order of priority: categories/subcategories, activity description, related keywords, company name

2.There should be a directory tree, like Y! or the ODP. (Does this get rid of the need for a one page site map?)

3.Searches should be recorded and listed on a “recent searches” and/or “popular searches” page. This allows a broader reach of keywords and increases the size of the site.

Optimisation issues

1. URL’s should be static and include keywords from the category/subcategory name. That means in this case using the ISAPI module (or mod_rewrite had it been an Apache server).

2. HTML tags should be created dynamically. Search results should include the search term in the title at a minimum.

What do you think of this structure? Can it be done better?

What are the key issues in promoting a regional hub?

The company is quite aggressive, and, in time, will add more regions to the same portal. Does that present any issues that should be considered at this point?

Thinking caps on. All contributions greatly appreciated.

[Dave_A]

I would consider this from my angle, I run a search engine in New Zealand and if you store results in a MYSQL database then our search engine wouldn't be able to read the data, it can only pull or read data from the page content.
I have tried many times to try and spider a web site held inside a directory type structure and it is almost impossible for the spider to lock into it and index it. We can visit it by surfing the web but getting a web spider into it simply doesn't work.
Sub domains are almost a waste of time trying to lock into even if they are given a domain name, then we have a problem getting a web spider to follow the redirects.
That is why Google doesn't like redirections either so be very careful how you organise it.
Other problems we have are when we index web sites full of flash graphics, intro's with a small html link somewhere on the page that says "Press here to enter the site!" PHp web spiders can't read and follow the directions they often only index a few lines and leave.
So perhaps try and look into it more fully before you go and start designing it, consider it from the search engines angle.
Many regards
Dave Andrews

[clearvision]

Hello Dave You said, "I have tried many times to try and spider a web site held inside a directory type structure and it is almost impossible for the spider to lock into it and index it." Does this mean that a page in a website that looks like http:// mywebsite.com/directory/stuff mean that a spider doesn't visit the "stuff" page? Sorry if this is a basic question...just trying to get a grasp on this idea :)

[figment88]

Don't try to get rid of your site map, but think of ways to make more.

listings by location
listings by activity
etc.

Make sure you have a structure that allows a listing to be found by more than one category (unless it is impossible with topic).

home > cat1 > subcat1 > listing
home > cat2 > subcat2 > same listing

I use mod_rewrite extensively with my directories. I don't really know if you can get the same functionality out of ISAPI module.

[elguiri]

Thanks for your replys, guys.

figment88 - unfortunately the ASP/IIS/ISAPI is one of the few things I can't influence, so I'll have to live with it. That aside, the idea of providing multiple access points is one I've decided on.

Do you find that having different maps provides more sense to the search engines about directory pages?

For example, if I have
a)a map of all pages relating to locationA, and
b)a map of all pages relating to activityB,

will my page dealing with activityB in locationA be rewarded when someone searches for "activityB in locationA"? Will the search engine relate location and activity more easily?

Also, when using mod_rewrite on an Apache server, can you use a rewrite map on a shared server or must it be dedicated?

Dave, does your spider still have problems when the URL's are rewritten?

[figment88]

will my page dealing with activityB in locationA be rewarded when someone searches for "activityB in locationA"? Will the search engine relate location and activity more easily?

I don't think it will necessarily be easier for the search engines to put two together, but people usually start with simple searches. For example, if someone wants to find red widgets in london they will probably start their search for widgets, red widgets, or widgets london.

Also, if they really do put in the specific red widgets in london they might want specifis sites instead of a directory (depending on what widgets we're really talking about).

Also, when using mod_rewrite on an Apache server, can you use a rewrite map on a shared server or must it be dedicated?

Yes, it can be configured using the .htaccess file which can be site-wide or different for each directory. What I do is route all URL's to a php script that does db queries based on the terms in the URL.

For example,
www.widgets.com/city/london

will run a php script that has a query
$sql="select * from widgets where city='london'";

[eZeB]

I am not sure what scale you are thinking of, but you will want to avoid google adwords if you getting really big.

Lots of people have built spammy directories, SEO'd them, got indexed with loads of top spots, slapped adwords on all the pages and made a bundle until Google gives them the boot.