Welcome to WebmasterWorld Guest from

Forum Moderators: martinibuster

Message Too Old, No Replies

A question of hubs

Bringing posts together and answering questions...



6:11 pm on Jun 13, 2001 (gmt 0)

In response to the email I get from my posts on hubs Iím starting this discussion to bring them together and answer some of the terrific questions I receive that I feel will benefit anyone interested. Iím also really hoping that others of you with experience on these issues will jump in and share your views, experiences and ideas with the group. All technical support available will truly be appreciated.

A disclaimer right up front. I do not want to turn this into a promotional list of products people use to make this happen but I do encourage participation from companies with products. There are rules governing this so read them first before posting. Be sure that if you do join in your URL is in your profile. Also, sticky mail and email are great but Iím hoping we can keep much of the information in front of the public with this discussion. If you are gameÖ here goes.

To even begin I suggest you become familiar with previous posts that Iíve participated in that relate to this topic.

Themes part 1
Themes part 2
Discussions on canonicals and links
DMOZ Clones

Iím sure there some here that Iíve missed so if you find one that should be included, please post.

After you read through these youíll see that I use ODP for my search and creating my directory although I do customize the results. To find out more about using ODP for this must-reads include:

Many of the questions I get revolve around using ODP and technical issues. I am not a technical guru but I do have access to one professional and he is worth every penny. If you purchase a product make sure they have great technical support. Here are random answers to questions Iíve received.

It appears to be a googlebot thing as I've seen it now on different hubs I've created, on different servers, using different scripts. It appears that Googlebot leaves a cache behind with every hit. I have never seen anyone anywhere talk about this. Last spider I was getting hit with 25,000 to 38,000 hits a day from googlebot. This can be a huge problem. Through Anaconda and with technical support we were able to set up a purge so I can click a link and purge the cache left by googlebot. Not knowing this can cause severe memory problems on your server. Googlebot hits me nearly every day. Last month there were only four days I wasn't visited by googlebot. I found my sister site was also receiving more googlebot visits than others appeared to be. This appears to be a good thing.

I've created two hubs now using dmoz. My present hub has 43,700 pages indexed and nearly all of those pages are due to the dmoz info. I think it's perfect because you can recreate your own directory to point to related categories but they appear to be your own. I have dynamic URLís for the search function but we were able to modify the categories to include cgi but no Ď?í.

To prevent unrelated oddball searches coming up in the log files we have been narrowing the search options we offer using the dmoz info. This is important if we want to stay on theme. I would rather have 4,000 pages of the hub on Google that are on theme than the 40,000 plus with many unrelated. This is an area I am still working on and welcome any technical support from people experienced with this.

I do not actually store the dump on my server. That's what Anaconda is about or any of the POD or ODP++ scripts. I'm unfortunately not as techie as I would like to be. Here are a few more resources you can check out for this:

Upload_Tools [dmoz.org]

I tried to go through my email and collect answers to most of the common questions I receive. I hope this helps and we can continue to discuss this. I am not an expert on this but I have over a year of experience now with research into development. There are experts among us and I certainly hope they jump in.


6:18 pm on Jun 13, 2001 (gmt 0)

WebmasterWorld Senior Member agerhart is a WebmasterWorld Top Contributor of All Time 10+ Year Member

excellent post Paynt!

I am going to read it about three more times so that I can actually ask you a good question about this


6:49 pm on Jun 13, 2001 (gmt 0)

10+ Year Member

Excellent post - it's been difficult keeping track over the last few weeks.


10:31 pm on Jun 13, 2001 (gmt 0)

I'm going to toss in my two cents. In in the interests of fair disclosure I should point out that I am a co-founder of Anaconda Partners.

There are really three general methods one can approach to integrate the Open Directory Project database into their website.

The first and oldest method is also the most time and resource consumptive. Originally the only way to integrate the Open Directory Project into your website would be to download the massive Open Directory database file to your server and host it yourself. In addition to the space required a suite of tools are needed - a database and a searching mechanism.

The database and search problems can be solved with any number of different tools. A free tool called "Senga" (I believe) has been available for several years to help with this process. It should be noted, however, that local handling of the ODP database is not for the faint of heart - if only because of the time required to download the several hundred megabyte compressed file and the 1+gig required for it once its uncompressed and another gig or so required for its index...

While the locally hosted method is very unwieldy and out of reach for most small websites, many larger sites choose it for the level of control it provides them over the data.

The second method was an Anaconda innovation - that of proxying content as-needed live from DMOZ.ORG directly onto your website. In brief, this method involves installing a small piece of software on your webserver which presents a gateway to the Open Directory database on your website. As your visitor clicks on a link (/Arts/Television, for example) the software remotely fetches /Arts/Television data from Dmoz.org and re-presents it on your site - and also temporarily caches it. As it is cached, if another visitor clicks on /Arts/Television a moment later, they are presented with the same data, this time served directly from the cache.

Subsequent clicks on subsequent links repeat the same process. Searching the database causes the software to perform the search at Dmoz.org, retrieve and cache the results and present them from your site as well.

This method's advantages over the locally hosted method are significant: Instead of 2+ gigs of local space to host the data file, only a small amount for the software and cache are required. The software itself requires insignificant CPU resources, and instead of consuming the bandwidth required to download the entire database, data is downloaded on an as-needed basis only, saving a great deal of bandwidth. The data that is presented on your site is always up to date as well as it doesn't depend on the latest creation/downloading of the database dump.

This innovation appeared in our Anaconda's "Open Directory" and "Open Directory Pro" products which have seen many imitators, several of which have been mentioned in this and related threads. The current Anaconda product is called "Foundation Directory."

The third and newest method of integrating Open Directory content into your site is another Anaconda innovation called DigitalWindmill.

This method involves a webmaster pasting three lines of javascript into the page on his or her site where he/she wants the Open Directory to appear.

When this page is loaded in a visitor's browser, the javascript contacts an Anaconda/DigitalWindmill server and fetches one screen's worth of Open Directory data. To the visitor, the data appears on your website's page on your website's domain, but its source is actually the AP/DW server. A subsequent click on a link (/Arts/Television again, for example) causes the page to load again from your server, with the javascript requesting /Arts/Television data from DigitalWindmill this time. Again it all appears on your site.

While this method provides you as the webmaster with less flexibility in terms of the "look and feel" of the data being handled, it has several advantages, notably that there is no software to install - the javascript is all that is required. No bandwidth is consumed, as the transfer of data is between your visitors browser and the DigitalWindmill server only. Since the service is being administered by a 3rd party there is zero administrative overhead on your part - no updates to download and install. And finally, it is free! ;)

The feedback we have received over the years from webmasters regarding the value that Open Directory data on their websites has added has been overwhelming. This is the same TYPE of data that launched Yahoo as the world's most successful website. Since the depth and scope of the database is so vast, it can lend itself to virtually any topic. If your website is about vacationing in Paris, why not present the directory's Paris or French language categories to your visitors? If your site is about Video Games then use ODP to present thousands of game related links to your users.

Adam Stanhope


1:07 am on Jun 14, 2001 (gmt 0)

Welcome to WmW Adam,

Thank you for taking the time to visit and answering questions that I've wondered about and have received emails on. I wish I were more techie. I'm too dependent on saying, "This is what I need", and getting someone to do it.

I do have a question about the free DigitalWindmill script. Does that show up as cgi without ?. Also can you customize it so you can work with themes or is it the whole ODP mirror.

Thanks again for joining and answering these questions.


3:36 am on Jun 14, 2001 (gmt 0)

Hello Paynt:

Thanks for the feedback.

Regarding the ?: Yes, the ? appears in the URL. It must be there in order for the javascript to work properly on the page as served from your server.

Regarding themes: Indeed DW can be "rooted" to a specific category/subcategory within the Open Directory database. An example of this would be to start the "tree" at /Home/Gardens. Simple instructions for this appear at:


Adam Stanhope


9:40 pm on Jun 14, 2001 (gmt 0)

10+ Year Member

paynt - Thanks again for this excellent material.

Here is another one. The linking structure you have described in your
posts is: outgoing link from a links page within the themed canonical;
incoming links to the home page (I cant remember if the home page of the hub
domain or the canonical homepage of the theme, but it is not the same page
as the outgoing link.) Is there a reason for this, or is it just your
personal preference?

Also, should I be requesting links from a "links page" or a "themed page"?


10:48 pm on Jun 14, 2001 (gmt 0)

Thanks Budterm, for bringing the question to the forum. Iíve been giving my link strategies a lot of thought lately so I welcome the opportunity to talk a bit more about them.

To answer your question, yes that is how I have been working my linking and itís been very successful. The key point that I doubt Iíll ever change is having more than one link page, connected to itís themed counterpart. Although itís not been the case until recently I am now concentrating on not allowing any of my link pages to run over 40 links. Why? I think if I can find 40 high quality link partners for each themed page, Iím doing pretty well. I also believe that the more links on a page the less punch they carry.

While weíre looking at it, who actually benefits from that page? The people the links lead to, obviously. Itís a courtesy we offer in exchange for a similar link. To capture the last part of your question, I would much prefer and I believe that it works more in my benefit, if people will add a link to my site from a non-link page on their site. If fact, if weíre thinking Google and I believe many of us are, then the best page that someone can link to me from is one that has a high page rank, is within a complimentary theme as my own and has very few outgoing links on it. It can be really ugly to clutter a page with reciprocal links so I would definitely save these for special link partners.

Having said that I still believe we need these link pages. Why not just have the link partners link back directly to the link page itself? Well, that isnít going to help me unless Iím trying to build the weight of the link page up and I would prefer that they donít get much notice. I certainly donít want someoneís first visit to my site to be on my link page. If it does happen, and it does, I want the first link they see to lead them to the main themed page they are looking for. This could be a www page, a canonical or a sub page.

To sum this up, as it pertains to my strategy Budterm, the ultimate is the link coming in from a highly ranked page, within the theme, that has very few links on it. How realistic is this or how possible is what you make of it. I really think these links are partnerships and they're only going to become more important. The better you build your site as a whole will offer you more opportunities for the best of the possible partnerships. I think hubs help make the site more appealing.

Thanks for the question. I hope people feel welcome to express their own ideas and strategies here.


11:04 pm on Jun 14, 2001 (gmt 0)

WebmasterWorld Senior Member littleman is a WebmasterWorld Top Contributor of All Time 10+ Year Member

Did you all know that Google is actively kicking out dmoz mirrors?


11:50 pm on Jun 14, 2001 (gmt 0)

Good point littleman, and a great reminder that an effective hub is much more than a dmoz mirror. I use the dmoz info as one of many tools that goes into the hub, along with the most important, great content. I also use a message board, guestbook, not to mention the themed layout and internal linking. I also have a few other tricks up my sleeve because you canít tell all your secrets.


4:57 am on Jun 15, 2001 (gmt 0)

10+ Year Member

paynt - Thanks... I actually get it now (finally)!

littleman - When you say "kicking out", do you mean banning from their index?


6:12 am on Jun 15, 2001 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Yes google removes the dmoz clones. I can't say as I really blame them. That is so much duplicate data. With an rdf dump at 500+meg, it wouldn't take long to reel off a few hundred gig in dmoz clones if they indexed them.

What I've seen is they will still allow the root page of the site in.

It's a tough one since I would love to put up a site that somewhat the same, but a little different format than dmoz to give it a unique feel.

Nice work on the program Adam and welcome to the board. Although I don't run it, I must confess to wandering through the code a few times - nice and clean.


3:48 pm on Jun 16, 2001 (gmt 0)

I wanted to add another link here to a discussion on sub-domains and navigation at [webmasterworld.com...] . Also the one on sub-domains and indexing at [webmasterworld.com...] . Just to help for referencing these ongoing discussions.

<added>Thanks to ideavirus for finding this thread on themes at [webmasterworld.com...]


5:53 pm on Nov 14, 2001 (gmt 0)

I'm not tooting my own horn here in bumping this thread. I think with the discussions going on about Yahoo and Overture and Liane's excellent post bringing up again the idea of niche portals, we may want to take a look at this information again.

Are there any new portal ideas or questions?


9:16 pm on Nov 14, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member


No offense ... but please don't assume that I endorse cartels or niche directories. Quite the opposite! I feel the whole concept is disgusting and I will fight it tooth and nail to the bitter end!

If I have to supply the best niche directory out there for free ... then that is what I will do to bust these damned cartels!


10:09 pm on Nov 14, 2001 (gmt 0)

Oh, I can tell that from what youíve written. Your strong and clearly expressed opposition certainly doesnít stop me from promoting something that I have found to work. In fact, in this ever-changing business climate if I want to make money I need to ensure my clients are found. That means many different things these days. Niche portals are one. Your comments, although in strong opposition again reminded me that others may once again want to review the information gathered here.

I'm glad you brought your concerns to the table. Fortunately, we don't all have to agree. Sometimes it's the opposition that stimulates the most.

I'm someone who strongly believes in the positive movement with niche portals. I certainly donít run from or fight something that obviously works well for me. Fear and change only motivates me to action. In my experience niche portals are a positive action, for my clients and myself. I suppose if youíre not creating them or working as a team with other similar niche markets to develop them, then you probably have to pay to get on one and I agree that wouldn't make me smile.


1:46 am on Nov 17, 2001 (gmt 0)

10+ Year Member

I read through most of the links listed, but did not see Liane's thread mentioned. Which thread was that?? ...when niche portals/vortals are mentioned, I want to read it.

This may be an old thread, but still a good one. I'll see if maybe a few questions come up from this stuff. Many know who I am associated with, most do not, so I'll just mention I am in the "companies with products" category. (hosting/software provider) If this gets into a real specific discussion, I can always pass questions on to portal owners if it does not get answered here. As of today we are up to 3,203 of them hosted now.

I would like to just pass on a couple things about portals from several different positions.

1. From a position of a web site owner looking to promote a my site...

Topic specific/niche sites with directories and smaller portals are where I start looking for links even before I submit to search engines. As most of the big SE's have gone to links relationship to rank sites, these seem to be the biggest source of getting some link pop quickly with low or no cost.

These smaller directories many times bring some very high quality traffic to the site early. This gives me a quick view of how my site will function with the right traffic and I can quickly fine tune structures/wording/paths so when the traffic does start to show up in quantity, I know I'm ready for it.

When submitting to these smaller directories, I have a file of titles/descriptions I use and will use many different combinations depending on the categories available, the type of site it is, and to spread different combos around. The different combinations gives me the ability to see what works good together, either to pre-qualify my visitors better or just get more hits. Plus, it puts a larger number of 'keywords' in links pointing to my site.

2. From a serfer point of view.

Depending on your market, you may have an experienced web serfer group visiting your site. If you do, I would not use ODP data. They have seen it a hundred times and they are looking for something new (IE, their not impressed). In this instance you need to do the work and properly build your own data for a directory. If you have just the average internet novice type of traffic, you can use just about anything and they will think your a genious.

You had better have a search feature or I am going to your competitor. If I can't search within sub-categories, I am frustrated but I'll at least try it.

Some applications give you many abilities to add supporting data with descriptions in the listings, such as submitted data, rate/vote links, etc... Keep these to just what is needed, don't clutter your listings with a bunch of stuff unless you have to. I hate waiting... affiliate graphic links many times will slow your pages down if servered from the remote server. This compounded by the search process can make a directory unusable.

Keep it simple and remember I am looking for unique content.

3. From a portal owner's view.

You will have a greater respect for Yahoo, Dmoz, Looksmart, and others after a few months, trust me.

Maintenance!!! If you have never had the pleasure of checking a directory for 404's, get ready for it, and do it often. If you allow submissions then you have to review them. I would not suggest allowing any submission to pass without at least a quick review. You will be amazed at what spammers will do. Get ready for requests for title/description changes, category suggestions, the standard 'I hate you and the site you first visited' emails from rejected links.

Automate everything you can, such as STATIC page generation, link rot(404) checking, algo rotation/tweaks, submission/rejection emails, targeted web crawling, etc...

Study this page:
..live it, learn it, luv it.
This is the only place I have seen this explained properly, well done Brett (just found it couple weeks ago, when was it written?). I stumbled upon this pyramid thing a couple years ago when I was building Yahoo Stores for people, then Google hit the market and all these sites hit the top of the charts, a little study and common sense led to exactly what is explained on that page. Yes, building a theme-pyramid and proper link stucture is the sercret to 'hub' status. ( why did he make it public... ;) )

On sub-sub-pages, you know the ones targeting those specific terms, use your keyword phrase ONLY in all the common places, such as title of the page, heading of the page, in description, in a paragragh early in the page, in link text on the page, in link on parent page, etc... Get creative with keywords used, but study to see which ones work. On dead terms that don't work, keep the page just add more. Never replace a bad keyword page, just move links...

Each sub-directory may give you another link to submit to other places if the content is different enough. And don't forget, category names are great for keywords just as well as the page names.

4. From the server side of things.

Don't run dynamic, publish your directory to static pages always. If your software can't do that, get new software.

Have a fast hard drive and lots of memory. You will be amazed at how many more page views a directory/search engine user will go through compared to other type of sites. They search many different combos, they play, sometimes they try to break your search, and yes they will search for porn no matter what your site is.

**TIP: study your search logs!! you will be given some real juicy phrases you never thought of.

Few things to consider about ODP data.

Yes it is the easiest DB to get a hold of, but just remember we have way too many of them out there. If you are just putting a directory up to get hub status for ranking, then use whatever is easy to do, but if you expect visitors to use it, I would suggest you forget ODP data and do the work yourself. You can't compete with the big boys when you have the same product they do, be better! ODP is a place to get some data to start, but don't settle for it. I am not down on ODP. I think it is a great resource, just over used.

Thousands of sites constantly hit dmoz.com parsing the ODP site live to serve to their own page or DB. Such as a small script that just grabs dmoz.com pages and strips the results out and places them in your container page. This has put a major strain on the ODP servers and has cost them editors/staff from the overhead cost. This in turn cost the web the quality and data lost from this loss of persons. Don't use a program which accesses a site like that, do it the right way ok, get the ODP dump.


For those that are serious about respect in their field and get noticed by the press, yes a properly implemented portal can do wonders. I'm not just talking about putting up a searchable directory, more is needed to be a portal. But, for those that have implemented all the tools and have built a true vertical portal, it will get noticed more quickly than most other site strutures.

...so what is a portal. Think of every source of information/product/service in a specific field. What type of data/news/content feeds are available, what type of services/tools, what can you develope/write... put all this under one roof, then you have a portal. (get creative)


paynt... I think 40 per page is a little too high, I have found 20-30 in a better range. Though many variables in that one, your linkage may differ.


Portals are here for the long term. Many things in the near future in this area. Yes, localized content will be a massive movement very soon. Global advertising has hit botton and will not recover very soon, local is where it is at. The business model follows the profit...


2:55 pm on Nov 17, 2001 (gmt 0)

Ah netcommr,

Thanks for bring new life into the portal/hub discussion. You have made some terrific points here and I appreciate your willingness to share. I have a few answers for you and comments. I hope others will join in.

>ÖLiane's threadÖ>

Sheís going to be ticked at me for bringing up her name. The comment was from the discussion weíre having on
Yahoo teams up with overture [webmasterworld.com]. The only reason I used her comment to jumpstart this discussion again was because she was so against them and probably because I am so for them. Plus she expressed her opposition so fluidly. Iíll hope she comes back in again to this discussion because I think a lot of healthy growth can come from disagreements. I want to publicly state that I think Liane is absolutely terrific and is a valuable member of this community. I expect many to disagree with my theories and ideas. Itís definitely not a surprise.

I donít develop all of my projects around portals although I do include many of the principals as often as possible. Portals have many things about them that search engines find attractive and we often discuss those pieces, particularly in terms of Google. For me the portal is a creative way of bringing these pieces together and enjoying the benefits.

Topic specific/niche sites with directories and smaller portals are where I start looking for links even before I submit to search engines. As most of the big SE's have gone to links relationship to rank sites, these seem to be the biggest source of getting some link pop quickly with low or no cost.

I absolutely agree. Weíve had some terrific discussion about this. Here are just a few for those, for you looking at the idea of portals for the first time or like me, revisiting the topic with a fresh eye.

Methods to research specialized search and vortals links [webmasterworld.com]
Inbound links - pointers please [webmasterworld.com]

I have a file of titles/descriptions I use and will use many different combinations depending on the categories available, the type of site it is, and to spread different combos around. The different combinations gives me the ability to see what works good together, either to pre-qualify my visitors better or just get more hits.

I think thatís a terrific tip.

I also agree with you tips and points of view from the surfer perspective. Clutter isnít better. I like a clean and simple look, with pertinent and viable information provided upfront. I also like to develop specific pages for each new topic or product. Make each page count and when possible stand-alone. Someone else said that recently and I really agree. A hub isnít about putting a mess of stuff on each page.

I do like each page to pay for itself in terms of all itís taken to create the page and submit it to directories and search engines plus the ongoing linking campaigns. And, as youíve mentioned, the maintenance because if you are going to have a portal you have to maintain it. In terms of how the page is perceived by the visitor though, much consideration should go into this.

Your >>>Öportal owner's viewÖ>>> points are well taken.

On Brettís theme-pyramid page, what can be said except heís brilliant and itís so incredible that he shares that brilliance with the world. I never tire of reading about this stuff.

Don't run dynamic, publish your directory to static pages always. If your software can't do that, get new software.

I absolutely agree. When you see the affects of a well spidered portal thatís been set up with static pages, indexed in the search engines, particularly Google and the traffic that comes in from that, itís amazing. Tie that in with the results you see in your referrer logs and like youíve mentioned, the keywords and keyword combinations you begin to see and can build upon is unbelievable.

On the ODP data Iím only going to point out the obvious. Itís free and to jumpstart a directory, for some with low-end pockets but high-end dreams, the ODP data may be the best place to start. Iíve given pointers regarding my experience with it and Iíll make just a few more points before we move on.

First, if you use ODP data, make it your own and this takes work. No mirrors. I use it as a jumpstart. It gives me a whole lot of links to work from to develop a page or even a category. I pick which sites I want to list and I modify titles and descriptions. Many an email Iíve received thanking me for listing a site and for the title and description Iíve used with a request that could I change it to that in the ODP. Unfortunately if itís not a category I already edit then, no.

I have moved away from using the ODP search box and really only use the info for the data already generated as filler for my own information. These listings are also great leads for further requests for their participation. I give them a free listing as a beginning, I write and let them know that and request an exchange, and then I suggest they can become involved further with an article or paid sponsorship links or maybe they have an affiliate opportunity of some sort I can develop. I figure if they are already in ODP then theyíve made some strides to be seen or found and are more likely to be interested in further development.

As to links on a page, 40 are my maximum. I have a little top-40 thing I play off of on a portal I worked on and itís a fun idea and worked well. I personally liked Brettís idea once where he developed a page around a link. Man, thatís a powerful exchange. I agree that 15-20 is better. Lately I find 10 good ones, along with all my internal navigation is more than enough. As often as I can lead to that one page per link on topic and for a specific keyword, the better.

Thanks again for your ideas and input. Very helpful.


5:02 pm on Nov 21, 2001 (gmt 0)

10+ Year Member

There's a lot of good stuff here and in those related discussions. Thanks Paynt, and others who have driven this. It's been very helpful and informative.

I spent quite a bit of time looking for high quality links to add to my site as outgoing links, and I think that definitely improves my site. However, these are links to related sites, not to competitive sites. And some of them are .edu sites. In most cases, it doesn't make sense for any of these sites to have a link back to me. It would detract from the value of their site, or dilute their value.

So, incoming links to my site are more difficult. I have a few questions regarding the value of reciprocal links on vortals. Is it possible that my link will be "diluted" by being associated with dozens of other similar links? Some web-rings are full of links to really cheezie sites. Also, is there a concern that the user will jump ship to the competition by placing my link right next to a competitive link?


3:45 pm on Dec 4, 2001 (gmt 0)

Those of you that have created a hub with dmoz. The cange will come very quick, so I`m interested in hearing how big changes you saw in the ranking of your page? Does the change to being classified as a hub reflect in the PR (more than it naturally would caused by new backlinks), or is it invissible and only shows when comparing rankings?


10:50 pm on Dec 5, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

paynt, netcommr and everybody,

This has been an incredibly useful thread for me and I thank you all for the thoughtful posts. It has made me take the plunge and finally switch from dynamic to a built mode on my directory.

You also lay out an excellent case for making most web sites into a hub site.

Anyway, even though I don't know enough to contribute to this discussion - I thought it important to let you know that your efforts are appreciated and have influance.

Thanks again.


12:39 am on Dec 6, 2001 (gmt 0)

Thanks Brad, itís exciting to hear feedback. I hope people keep popping back to let us know the results of their efforts. Weíre all learning from each other.

Sorry I missed your question Haakon. With my dmoz sites I do continually improve on PR but I think thatís to do mainly with the folks linking to them, especially the very large niche site which is rich in content. I never saw a change in ranking in the sense that for me when I launched it was with the dmoz dump included. Any improvement in ranking came with time as more folks linked to the site, as the content improved, and particularly as I refined the search feature.

I got a ton of traffic after my first spider and index from Google and mostly from the dynamic search, much of it off topic. I had a pet site getting tons of traffic for prom hairdos. Over a period of time we refined the search. I think this is very important for folks to understand, especially as the dump includes adult material and if you are developing a kid friendly site youíll want to make sure to exclude adult categories and linking. Itís also important to stay on theme.

Using the dmoz dump though at 6 weeks out, before refining I was getting 1500 unique visitors a day.


2:19 pm on Dec 12, 2001 (gmt 0)

Iíve been asked to explain a hub so that a third grader can understand. Iím not sure this does it but feel free to ask as many questions as you need so that you understand. I am more than happy to discuss hubs and since I first started posting about them my theories and experiences have evolved. Thereís a whole bunch of this stuff in my head and Iím just glad there are those of you that find this useful or at least entertaining.

For projects that have a variety of domains or canonicals, ideally related in theme, a hub is incredibly useful to pull the separate yet related domains together. It's the meeting ground. A wheel is probably the best visual example. You have all of the spokes (separate domains or canonicals) and they meet at the hub (center).

Now a hub could then extend its power and usefulness if it is additionally a portal with a search feature to search the web or if itís set up as a directory. The search though canít take you off site as in using an Overture search box or something. The results or search listings must contain your URL. Thatís why I suggest modifying the ODP dump because you get the benefit of appearing to be this huge portal. Itís really important though to modify the ODP info correctly to stay on theme and not have a lot of dynamic search strings. I find the ODP info best used to develop categories that relate to your theme. Iíve written lots about that already though so Iím sure you can find most of what I think about that with the other info already posted.

I believe the ideal hub is set up with separate canonicals. Each canonical relates to a sub theme of your main theme. Each of these canonicals is then connected to a separate domain that is all about that subject. Each sub theme is then developed in the same manner (with itís own domain) and connected then to itís canonical.
There can then be crosslinking (related and on theme) between the separate domains and within the hub itself. The homepage is where this all comes together.

For really large sites you may find hallway pages necessary, if you are dealing with lots of links, but in essence they arenít needed with this ideal hub because there is a clear pattern developed for linking and there are no orphan pages. Everything is connected. Not everything to each other but everything to what it should be to make it the best that it can be. We can always talk more about crosslinking. I think we have a ways to go in explaining how I approach crosslinking.

Another very important feature with the hub is a search feature that can search the hub itself for keywords that will lead the visitor to exactly where they want to go.

A hub can also stand-alone. It doesnít have to have separate domains for each sub theme or you can grow into that as the hub grows. If youíre going to have a stand-alone hub though I think you will want to use canonicals. Each of these canonicals are mini sites, ideally and can stand alone if they must. Ideally an editor at ODP would see the usefulness of listing the canonical because it offers unique content and resources, even though it ties into this larger entity.


3:48 pm on Dec 12, 2001 (gmt 0)

I've started a new discussion about hubs in the Research forum.

Researching the ideal SEO hub [webmasterworld.com]
Hub theory for the larger SEO company


4:59 pm on Dec 12, 2001 (gmt 0)

10+ Year Member

Hey Paynt, you luv this stuff don't ya?
I like where this thread is going... Maybe you'll actually 'define' what a hub is by the time your finished. ;) Here is a little more to think about to keep the thread rolling.

developed a page around a link
This I have seen work wonders when done properly. I know of some sites which get paid nicely for these pages. I think a good example would be an affiliate link you would want to promote. If you want to sell an affiliates product then pre-sell on a good page on your site, don't just put up a link somewhere...they don't work.(usually)

Same thing goes for full page links in a hub. If you want to impress upon the visitor the quality of a site, then this is the way to go. Something similar to a site review pointing out some strong points on the site your linking to.

if you use ODP data, make it your own

Very good point! I did not want to come across as anti-ODP, I have just seen way to many 'clones', especially for directory structure and category names. (too general for hubs)

Hubs are topic specific. A category structure designed for the entire web does not usually fit on a hub. This seems to be a hard thing to come up with for the average person starting a hub/portal.

When developing a category stucture naming strategy, take your time and think it through. This is not something you want to do over in the future. THINK KEYWORDS! Not only will this help your SE ranks, but your visitors will appreciate it as well. The words and phrases people use in a search engine are also the same words that 'key' into visitors when they see links on a page. Use common verbage and get specific. I would never use a 'General' or 'Other' category name. I would also not use something as general as 'Pets', but look for a better phrase, such as 'Your Household Pets'. But, always remember to stay on your 'theme' in the wording you use.

value of reciprocal links on vortals

Hi Tilt,
Just remember recip. links are a neccessity. You need it for ranking and for quality traffic. Don't be worried about loosing traffic, this is a mythe. Visitors are 'surfers' remember, they browse sites and compare. If your link is right next to a competitor, they will usually look at both. You need to have a better 'instant appeal' than your competitor and you will win. This comes into page copy and colors, study it! About the fact 'my link will be "diluted"' is not something to really be concerned with, though it does have a little affect. A page will many links does not have the same 'weight' as a page with few, but the 'association' you have to the theme of these other sites more than outweights the negative. Get the links...

being classified as a hub reflect in the PR

Good question Haakon. But to be honest with ya, I really don't care if my PR is 1 or 10, what I am concerned with is how the page actually ranks. If it takes a 3 to get the same spot as a competitor with a 7, then so be it. I just want the placement. I really don't know if achieving hub status will increase your PR, but why would it? These are two different things is Google's algo I believe. SO, what you'll probably see is just all of a sudden your #1 for some big terms.

**When you do achieve hub status, the sites you 'link to' will get a big help in the PR dept. Links from the home page can be 'awesome'...

A hub can also stand-alone

I am glad you posted that Paynt. I see many portals which do not have any content themselves, just affiliate links or plugin content, such as a news feed. I think for the success of any type of hub/portal needs to provide something to distinguish themselves from your competitor. Too many times we see just a link farm with no real content, but those sites which actually provide something usefull end up with visitors.

Think about this to tie it all together, do you want $100.00 per month from an advertiser for that full page ad you developed, or do you want to deposit that $5.00 affiliate check you get every 3 months...


5:34 pm on Dec 12, 2001 (gmt 0)

Youíre right netcommr, I do love this stuff. I also appreciate your participation and input. Itís like a fine tennis match.

I am concerned with is how the page actually ranks.

I donít create hubs to rank necessarily although if done well (relevant and unique content) they can rank exceptionally well. I create hubs for linking opportunity and to promote either a product line, an industry, an affiliate, or as the research topic I started suggests, a group of mix/matched SEO client sites. This is my foremost reason. A benefit of this is traffic not only to the sites promoted by the hub, but to the hub itself. Not necessarily ranking for specific keywords or keyword phrases. These hubs appear to capture a multitude of alternative keywords that you may not have considered when developing them, particularly in they are in a niche and are themed.

This can be confusing to those starting out. When I started my hub quest it was purely as a tool for linking. Creating a quality themed hub rich in content gave me the side benefit of terrific traffic to the hub itself with a 80% or so click over rate to the sister site. This was amazing. At the same time I saw the sister site rise in rankings because of itís linking to the hub.

What I find is the sites attached through linking to them benefit. Especially if done as you suggest by creating a whole page for a link. Especially true and helpful with affiliate promotion. If you develop a whole little mini site around the link, via canonicals, I find these links receive even greater traffic.

You are absolutely right that you must consider your keywords. I tie in my keyword development with the theming of the site.


11:18 am on Dec 25, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

If the suspicion of a cross-linking "filter" as came up in [webmasterworld.com...] turns out to be true, would it not have serious implications for the development of a hub?


10:53 pm on Dec 30, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

Well Google has now spidered, indexed and gone live with my site since I "built" the directory.

Traffic is certainly up and I'm getting hits on obscure search phrases I never dreamed of. It looks like it has helped on my primary keywords too.

So paynt and netcommr, you were right. I'm kicking myself for not doing this sooner. Thanks again for starting/participating in this fine thread. Great stuff. :)


12:54 am on Jan 4, 2002 (gmt 0)

10+ Year Member

Well done Brad..I've seen a few good listings for your site too. Keep up the good stuff bud.

Glengara, very good question, but I honestly think it is good for a true hub/portal. If Google is really giving a bonk to spam linkers I applaud them for it.

I see no penalty for sites that just link to a few other domains properly on the page to send traffic that way.

I see the bonk to sites that lack content, a bunch of banners and link,link,link,link,link,link, link,link,link,link,link,link, link,link,link,link,link,link, link,link,link,link,link,link, link,link,link,link,link,link,link at the bottom. They should be bonked! This is spam linking. You have no desire to send any traffic that way, you just want to boost your rank with an undeserving site.

Hubs (directories and portals) will get an even stonger boost if the spam linkers are gone.

This also means your can't get away with a small wannabe hub just full of links to your own sites. You need to acually do some work and build a content rich site along with your directory.

Paynt...good stuff as always.

tie in my keyword development with the theming
The last site I wrote I spent more time in research of phrases than I did in actually writing the site (first draft of course).

[edit to fix scroll]

(edited by: paynt at 1:08 am (gmt) on Jan. 4, 2002)


1:19 am on Jan 4, 2002 (gmt 0)

With the holidays and all I've been sadly neglectful with my posts. I wanted to add my cheerio, Brad. All the hard work deserves the extra traffic. I hope you have the same success Iíve seen with the site becoming stronger every round.

I second netcommrís comment about content. Iím eager to try out the blogger too and see what contribution that makes.

Honestly, with the direction the industry is taking in regards to Yahooís latest announcement about pay to be included each year, I stand by my prediction of 2 years ago that hubs will become even more important and popular.

And with linking, donít create a closed circle. Keep the information flowing through the web and youíll continue to build your popularity.

This 33 message thread spans 2 pages: 33

Featured Threads

Hot Threads This Week

Hot Threads This Month