How do you create a big directory and keep Google happy?

[Kriss Johnson]

SEO experts,how you deal with big directories?
Hi to all #*$! seo experts,

I'm in the middle of my making my first business directory,but don't have experience with directories and need some advice from you.

I built a business directory with let's say roughly 2000 categories and subcategories + 1000 locations (cities,states).Example for the directory structure:

example.com/Chigago
example.com/Chigago/Transportation
example.com/Chigago/Transportation/Van-Hire

So in total I will end up with more then 2 000 000 category pages!
My questions are:
1. If I submit so many pages to Google,I suppose the site will be considered as spam,so how to deal with that problem?

2.Also,can you tell me how to deal with so many empty categories?Should I fill them with listings from the net,or place noindex tag until they get natural listings?

Best Regards in advance
Krasi

[tedster]

I know it's not a good idea to have too many "stump" pages with little or no unique content. The cycles that googlebot spends crawling all those essentially empty pages will detract from finding and updating your pages that do have content. Crawling and indexing is likely to be quite limited until you attract significant inbound links and PR.

I've never built up a directory from scratch. I assume that robots.txt and rel="nofollow" could be useful. I hope others who have can jump in with practical ideas for how to focus googlebot.

[Planet13]

I know this isn't what you asked, but...

I asked a few questions about directories a while back on this forum, and most people said that for SEO purposes, having a smaller, more focused directory is going to rank higher than a "general" business directory, will be easier to acquire listings, and will be easier to maintain, and will probably be easier to monetize.

I understand that there is a lot of directory software that will allow you to have 2 million categories. But the question is, will it actually be more profitable for you to do that or not?

[TheMadScientist]

The biggest thing I would focus on is architecture...

IMO You're going to have to 'direct' search engines and visitors through the site and to the information they want using the least clicks possible. IOW: It needs to be well organized.

I would recommend tons of site maps (I've been known to run xml, txt and 3 sets of html for some sites); focus link weight sectionally on the important landing pages; build / manage it as 'sites within a site' rather than 'one site fits all', so to speak.

Also, noindex is a powerful tool.

Personally, I don't use nofollow, unless it's to keep a page totally out of the ranking equation, such as a contact page. Basically, I use it as 'insurance' in case I forget the noindex tag on the specific page I don't want to rank for some reason, not to sculpt PageRank to other pages. If you want to sculpt PageRank (IMO you're going to have to) do it with the links themselves not the nofollow tag.

[Rosalind]

Having a big general directory in 2010/11 seems misguided. The smart thing to do would be to concentrate on a tight niche and expand as and when you fill up the categories. Smarter still would be to build almost any other kind of website than a directory.

Users don't want to navigate down to empty pages, so search engines don't want to see them either. They have a way of managing this, called a crawl budget, which Adam Lasnik discussed in 2007. Look it up, because it explains why big websites that lack link-worthy content will fare pretty badly in Google.

[HuskyPup]

Having run a global trade widget directory since the 90s IMHO if you have no unique content to post then don't even bother. People are very lazy these days in submitting themselves to any old directory, getting them to do it with a niche directory is hard enough.

The problem is that whilst you and the large businesses with dedicated marketing teams may see the advantage of being listed, many small businesses neither have the time and possibly not the knowledge of how to do it no matter how simple you may feel you are making things for them.

Empty pages will get you nowhere in any of the search engines however if you do go ahead TheMadScientist is correct, good architecture is the key. I custom built mine since it was years before any CMS existed and it still easily out-performs anything on the CMS market.

[scooterdude]

@Rosalind
I've been reading this forum for a while. but this is the first time I've read about "crawl budget", makes sense , explains a lot, thanks

[scooterdude]

@huskypup

Do you get traffic from the Search engines ?

[HuskyPup]

Do you get traffic from the Search engines ?

Yep, loads, even though it is principally a B2B directory for my industry from raw extraction of materials, primary and secondary production to wholesaling to retailing, it is open for all to see and to educate both the trade and the public and to provide quality images so that everyone can see what example sample is actually like.

No prices are posted on the site however I do have AdSense so that advertisers anywhere in the world can target the site if they so wish.

It would be possible to expand it further however I have no desire to provide a.n.other A-Z listing of companies in each country however I do list principal machinery manufacturers and consumable suppliers along with all trade assoiciations, magazines, forum boards, trade portals etc.